Methods and apparatus for biological fluid sample characterization using a neural network with reduced training

Abstract

A method for training a neural network (convolutional neural network - CNN) comprising a reduced graphical annotation input is provided. The training method can be used to train a "test CNN", which can be used to determine hemolysis (H), icterus (I), and / or lipemia (L), or normality (N) of the serum or plasma portion of a test sample. The training method includes: capturing training images of multiple sample containers including training samples, generating candidate regions of the serum or plasma portion of the training samples; and selecting the best match for the location, size and shape of the candidate regions for the multiple training samples. The obtained features (network and weights) from the "training CNN" can be used in the "test CNN". A quality inspection module and test device adapted to implement the training method and characterization method by using a bounding box regressor are described, as in other aspects.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application Serial No. 62 / 615,873, filed on January 10, 2018, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates to methods and apparatus for training a neural network to characterize a biological fluid sample, and more particularly to methods and apparatus, including a neural network, for determining whether a biological fluid sample includes an interfering substance. Background Art

[0004] Automated testing systems can be used to perform clinical chemistry or laboratory tests that identify analytes or other components in samples such as serum, plasma, etc. by using one or more reagents. For convenience and safety reasons, these samples can be contained in sample containers (e.g., blood collection tubes). In the case where the sample is whole blood, a gel separator can be added to the sample container to help maintain the separation of the precipitated blood portion from the serum or plasma portion during centrifugation. The test or test reaction generates various changes that can be read and / or manipulated to determine the concentration of the analyte or other component present in the sample.

[0005] Improvements in automated testing technology have been accompanied by corresponding advances in pre-analytical sample preparation and handling operations, such as sorting, batch preparation, centrifugation of sample containers for separation of sample components, cap removal for facilitating sample access, aliquot preparation, and pre-screening for HILN (hereinafter "pre-analytical operations") by automated systems that may be part of a "laboratory automation system (LAS)". A LAS may automatically transport samples in sample containers to one or more pre-analytical sample handling stations and analyzer stations, which include clinical chemistry analyzers and / or test instruments (hereinafter collectively "analyzers").

[0006] Such a sample container may have one or more labels provided thereon. The label(s) may be a manufacturer's label and / or a label provided by a phlebotomist or other location or facility where the sample is handled. In most instances, at least one label includes identifying information, such as a barcode (hereinafter "barcode label"). The label(s) may be made of paper having, for example, an adhesive backing.

[0007] The LAS can handle a large number of different specimens at one time, and can use barcode labels for tracking and routing. The barcode label can contain an accession number, which can be associated with demographic information, which can be entered into the hospital's laboratory information system (LIS) along with the test order and / or other information, wherein the LIS is configured to electronically interface with the LAS. An operator or robot can place a labeled specimen container on the LAS system, which automatically routes the specimen container for one or more pre-analytical operations; all of which can be before the specimen is actually subjected to clinical analysis or testing by one or more analyzers, which can be located on a portion of the LAS.

[0008] After fractionation and other pre-analytical operations, the sample container can be transported to an appropriate analyzer, which can extract a serum or plasma portion from the sample container via aspiration and combine the portion with one or more reagents in a reaction vessel (e.g., a test tube). Analytical measurements can then be performed, typically using, for example, a beam of interrogating radiation, or by using photometric or fluorescent absorption readings, etc. Analytical measurements allow determination of endpoint or ratio values, from which the concentration of an analyte or other component can be determined using well-known techniques.

[0009] Unfortunately, the presence of any interfering substances (e.g., H, I, and / or L) in a sample as a result of the patient's condition or sample processing may have the potential to adversely affect the test results of the analyte or component measurement obtained from the one or more analyzers. For example, the presence of hemolysis in a blood sample, which may be unrelated to the patient's disease state, may lead to different interpretations of the patient's condition. In addition, the presence of icterus and / or lipemia in a blood sample may also lead to different interpretations of the patient's condition. Thus, it is desirable to know whether a sample includes H, I, and / or L, and even its index level, prior to analytical testing via pre-analytical operations.

[0010] Regarding whether the sample includes H, I and / or L, a normal (N) serum or plasma portion has a light yellow to light amber color. A serum or plasma portion containing hemolysis (H) has a reddish color. A serum or plasma portion containing icterus (I) has a dark yellow color due to increased bilirubin, and a serum or plasma portion containing lipemia (L) has a whitish or milky appearance. Depending on the color determined by the pre-analysis operation, the interfering substance type and index value can be assigned.

[0011] In some instances, one or more of the above-described barcode labels may partially obscure and obscure certain side viewpoints of the specimen, such that there may be certain orientations that do not provide a clear opportunity for visually observing the serum or plasma portion. Thus, automation of such prescreening has included, for example, rotating the specimen in a manner that allows for automated prescreening for H, I, and / or L, or N. For example, in some systems, such as those described in U.S. Patent No. 9,322,761 to Miller, entitled "Methods And Apparatus For Ascertaining Interferents And Physical Dimensions in Liquid Samples And Containers To Be Analyzed By A Clinical Analyzer," the specimen container is rotated to find a viewing window that is not obstructed by the label, after which imaging is performed.

[0012] In other systems, such as those described in WO2016 / 133,900 by Park et al., the sample container and sample are imaged from multiple viewpoints and processed using a model-based system so that rotation of the sample container is not required. However, in such a model-based system, the advance training of the model is extremely labor-intensive.

[0013] Therefore, there is an unmet need for robust and efficient methods and devices adapted to characterize the presence of interfering substances in biological fluid samples and that can be trained more efficiently. Summary of the invention

[0014] According to a first aspect, a method for training a neural network is provided. The method for training a neural network comprises: capturing a training image of a sample container containing a training sample at an imaging position, generating a candidate region proposal for a serum or plasma portion for input to the neural network, and selecting the candidate region that provides the best match to the serum or plasma portion of the training sample.

[0015] In another aspect, a method for characterizing a sample by using a trained neural network is provided. The method includes capturing an image of a sample container containing the sample at an imaging position, generating candidate regions of a serum or plasma portion for input to the neural network, and converging the candidate regions by regression to provide a validated region.

[0016] According to another aspect, a quality check module is provided that is configured to determine the presence of interfering substances in a sample contained within a sample container. The quality check module includes: an image capture device configured to capture a plurality of images of a sample container containing a serum or plasma portion of the sample; and a computer coupled to the image capture device, the computer configured and operable to: input image data from the plurality of images into a neural network, generate candidate regions for the serum or plasma portion, converge the candidate regions by regression, and output from the neural network a classification of the serum or plasma portion as one or more of hemolytic, icteric, lipemic, and normal.

[0017] By illustrating multiple example embodiments and implementations, including being contemplated as the best mode for implementing the present invention, still other aspects, features and advantages of the present disclosure can be easily apparent from the following description. Some details of the different embodiments described herein can be modified in various aspects, all without departing from the scope of the present disclosure. The present disclosure is intended to cover all modifications, equivalents and alternatives that fall within the scope of the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described below are for illustrative purposes and are not necessarily drawn to scale. Therefore, the drawings and descriptions are to be considered illustrative rather than restrictive in nature. The drawings are not intended to limit the scope of the present invention in any way.

[0019] Figure 1 A top schematic diagram of a sample testing device including one or more quality inspection modules configured to perform pre-screening for H, I, and / or L or N is illustrated according to one or more embodiments.

[0020] Figure 2 Illustrated is a side view of a sample container including a separated sample having a serum or plasma portion containing interfering substances, and wherein the sample container includes a bar code label thereon.

[0021] Figure 3A A side view of a sample container including a barcode label, a separated sample including therein: a serum or plasma portion containing interfering substances, a precipitated blood portion, and a gel separator is illustrated.

[0022] Figure 3B A side view image of a sample container including a bar code label, a sample comprising: a serum or plasma portion containing interfering substances, a precipitated blood portion, and a gel separator, wherein the sample container is supported in an upright orientation in a holder and a rectangular imaging area is illustrated.

[0023] Figure 3C The present invention illustrates a method according to one or more embodiments. Figure 3B Side view of the imaging area including a grid superimposed with examples.

[0024] Figure 3D The present invention illustrates a method according to one or more embodiments. Figure 3B Side view of an image of with a simplified superimposed grid based on knowledge of the tube diameter.

[0025] Figure 4A Illustrated is a schematic top view of a quality inspection module (with the top plate removed) comprising multiple viewpoints and configured to capture and analyze multiple images to enable determination of the presence of interfering substances in accordance with one or more embodiments.

[0026] Figure 4B FIG. 1 illustrates a method for performing a multi-step process according to one or more embodiments. Figure 4A The section line 4B-4B is taken Figure 4A Schematic side view of a quality inspection module with the front enclosure wall removed.

[0027] Figure 5A A block diagram of an example architecture for training a CNN in accordance with one or more embodiments is illustrated.

[0028] Figure 5B A block diagram of an example architecture for testing a CNN is illustrated in accordance with one or more embodiments.

[0029] Fig. 6A A first image from a first viewpoint is illustrated in accordance with one or more embodiments.

[0030] Figure 6B A second image from a second viewpoint is illustrated in accordance with one or more embodiments.

[0031] Figure 6C A third image from a third viewpoint is illustrated in accordance with one or more embodiments.

[0032] Fig.6D A schematic top view illustrating the orientation of various viewpoints is illustrated in accordance with one or more embodiments.

[0033] Figure 7 is a flow chart of a method for training a neural network according to one or more embodiments.

[0034] Figure 8 is a flow chart of a method for characterizing a specimen according to one or more embodiments. DETAILED DESCRIPTION

[0035] In view of the above-mentioned deficiencies of the method for training a model-based system for characterizing a biological fluid, a method and apparatus are provided herein that significantly improve the efficiency of the model training process. Specifically, prior art training of a model-based system involves imaging a training sample with known properties, and then the operator will have to manually draw various parts (annotations of graphic areas, such as bounding boxes) within the image of the training sample. These bounding boxes can draw peripheral areas of serum or plasma parts, precipitated blood parts, gel separators, air, etc. These bounding boxes are provided as annotated inputs (graphic inputs) for model training so that the model can be trained to identify these areas later during sample testing. In addition, manual data annotation inputs can be provided for each manually generated bounding box, such as H, I, L class or N and / or HIL index values. It should be understood that providing and inputting such manual bounding boxes (of its graphic area) makes the overall training of the model-based system very cumbersome and extremely time-consuming, thereby requiring manual effort of a person. Therefore, a more efficient model training method that is less dependent on manual graphic input is highly desirable.

[0036] Thus, the present disclosure provides methods and apparatus configured to train models, such as neural networks (e.g., convolutional neural networks - CNNs). Specifically, the model can be configured and adapted to determine whether an interfering substance (e.g., HIL) is present in a biological fluid sample or whether it is normal (e.g., N). The training methods and apparatus described herein are extremely efficient and do not involve any manual zone / graphic annotation input in order to train the neural network (e.g., CNN). Thus, neural network training can be performed quickly and with minimal manual input by an operator.

[0037] In a first aspect, a training method involves capturing images of multiple sample containers including training samples at an imaging position using one or more image capture devices for capturing multiple training images (e.g., digital, pixelated training images). Ground truth labels for a model (e.g., a "training CNN") are input as training inputs. For each training sample, a disturbance type and possibly an index unit of measurement or verification may be provided as a ground truth label, which is expected to be an output from a test CNN. Thus, the ground truth label may be, for example, H, I, and / or L, or N, and / or in other embodiments, a number of discrete index values ​​for each of H, I, and / or L, which are expected to be an output from a test CNN. A "test CNN" is a model that, once trained, can then test samples against a HILN.

[0038] Image data preprocessing can optionally be used in some embodiments to, for example, achieve background removal or tube type detection. Such preprocessing can be used to reduce the size of the image (i.e., to the relevant area of ​​interest), reduce the amount of iterations for convergence, and reduce the overall time for training. In order to avoid the use of manual graphic input, the method and apparatus described herein include the automatic generation of candidate areas for serum or plasma parts. The candidate areas automatically generated for training are composed of sub-areas of the imaged area in an embodiment. For example, the sub-areas can correspond to sub-areas of a virtual training grid that has been superimposed on the training image. In other words, the candidate area can be a collection of sub-areas of different areas as smaller areas within the virtual grid. In an embodiment, the virtual training grid can include vertical and horizontal lines that intersect and form virtual nodes of the virtual grid at the intersection. A collection or grouping of grid elements within the virtual grid can constitute a candidate area.

[0039] For example, the candidate area may include a rectangular sub-area within the training image. For example, in some embodiments, the candidate area may be generated by selecting two virtual nodes within a virtual grid (e.g., diagonally opposite corner nodes). In some embodiments, a specific candidate area may be randomly selected. Thousands of training images are captured and input to the "training CNN", and thousands of randomly generated candidate areas are also generated and input to the "training CNN", together with ground truth information about HILN and possibly HIL index levels. Each of the various candidate areas can be checked for relevance (fit) to the position of the serum / plasma portion within the entered image data.

[0040] For example, each candidate region can be tested / checked by filtering, such as by examining the light intensity gradient within each candidate region. For example, the degree of match can be discerned by examining the gradient of light intensity within certain portions of the training image. Those portions with good correlation can be kept, while those with poor correlation can be discarded. Thus, a number of best candidates (e.g., thousands) can be selected based on the degree of "match" with the input training image, i.e., those candidate regions with the highest correlation can be used.

[0041] In some instances, the virtual grid can be composed of grid elements with rectangular shapes having width (W) and height (H) in size. A suitably fine grid should be used so that a good estimate of the serum or plasma portion or size and shape can be obtained. Each candidate region is generated (e.g., randomly) and submitted as input to a "training CNN" along with training images and ground truth labels, and convergence is tested. Convergence means that correct results are obtained in 80% of the cases, or even 90%, or even 95%, or higher. If sufficient convergence is obtained during training, training can be stopped. Consideration of appropriate training can be achieved by considering the loss for the HIL classification part (SoftMax).

[0042] Once the "training CNN" is sufficiently trained, the various features and / or model weights can be transferred and used as part of a "testing CNN" which is configured to automatically identify the serum or plasma portion of a test sample in a sample container and provide an appropriate classification, such as H, I, and / or L or N, and if the model has been trained to do so, provide its H, I, or L index.

[0043] Thus, once trained, the "test CNN" can be used to determine the presence of interfering substances, such as H, I, and / or L, or N (hereinafter "HILN") or some HIL subset thereof, in the serum or plasma portion of the sample. The method and neural network (e.g., "test CNN") can be configured to determine n types of H (e.g., H1, H2, H3 or more), n types of I (e.g., I1, I2, I3 or more), and / or n types of L (e.g., L1, L2, L3 or more), or N. In some embodiments, the "test CNN" is capable of identifying all three of n types of H, and n types of I, and n types of L. In particular, although a 10-class "test CNN" is shown, including H with n=3, I with n=3, L with n=3, and N, other numbers of results can be used, such as n=4, 5, 6, 7, 8 or more.

[0044] Distinguishing serum and plasma fractions from regions that include one or more labels is a particularly vexing problem because the one or more labels can wrap around the sample container to various degrees. Thus, the one or more labels can obscure one or more viewpoints, making it difficult to obtain a clear view of the serum or plasma fraction. Thus, classification of the serum or plasma fraction can be quite challenging due to interference from the one or more labels, the placement of which can vary considerably from one sample container to the next being prescreened.

[0045] In particular, given that the one or more tags may be present on the back side and thus affect the light transmission received on the front side, the obstruction caused by the one or more tags may severely affect the spectral response, such as from various viewpoints. With this in mind, the input to "training a CNN" may be multi-spectral, multi-exposure image data, which may be pre-processed (e.g., merged and normalized), and obtained from more than one image capture device. For example, the image capture device may include a plurality of image capture devices, which are arranged and configured to capture images from more than one viewpoint (e.g., such as three side viewpoints).

[0046] In some embodiments of the training method, some easier training examples (e.g., no labels) may first be trained at various levels of HIL and N and from multiple viewpoints. Then, more difficult training examples including various positions and various numbers of labels (e.g., up to two labels) may be input as training specimens at multiple viewpoints. As a result, more efficient training of a "training CNN" for identifying serum or plasma regions may be provided in situations where there is label obstruction.

[0047] definition

[0048] "Interfering substances", as used herein, means the presence of at least one of hemolysis (H), icterus (I), or lipemia (L) in the serum or plasma portion of the sample. Hemolysis (H), icterus (I), or lipemia (L) are collectively referred to herein as "HIL".

[0049] "Hemolysis" is defined as a condition in the serum or plasma fraction where during processing, red blood cells are disrupted, which results in hemoglobin being released from the red blood cells into the serum or plasma fraction, causing the serum or plasma fraction to take on a reddish hue. The degree of hemolysis (H) can be quantified by assigning a hemolysis index.

[0050] "Icterus" is defined as a condition of the blood in which the serum or plasma fraction is discolored a deep yellow due to the accumulation of the bile pigment (bilirubin). The degree of jaundice (I) can be quantified by assigning an icteric index.

[0051] "Lipemia" is defined as the presence of abnormally high concentrations of emulsified fat in the blood, such that the serum or plasma fractions include a whitish or milky appearance. The degree of lipemia (L) can be quantified by assigning a lipemia index.

[0052] "Normal" is defined as a serum or plasma fraction that includes acceptably low amounts of H, I, and L and is designated N herein.

[0053] The "serum or plasma fraction" is the liquid component of blood. It is present above the precipitated blood fraction after fractionation (e.g., by centrifugation). Plasma and serum differ in the content of clotting components, primarily fibrinogen. Plasma is a non-coagulating liquid, whereas serum refers to the plasma of blood that has been allowed to coagulate under the influence of endogenous enzymes or exogenous components.

[0054] The "precipitated blood portion" is a packed semi-solid, consisting of blood cells, such as white blood cells (leukocytes), red blood cells (erythrocytes) and platelets (thrombocytes), which aggregate and separate from the serum or plasma portion. After fractionation, the precipitated blood portion is present at the bottom portion of the sample container, below the serum or plasma portion.

[0055] An “image capture device” is any device capable of capturing a pixelated image (eg, a digital image) for analysis, such as a digital camera, a CCD (charge coupled device), a CMOS (complementary metal oxide semiconductor), an array of sensors or photodetectors, and the like.

[0056] A "pixelated image," as used herein, means an image composed of a single pixel or a grouping of pixels, such as a superpixel or an image patch (patch), which includes more than one pixel.

[0057] A "label" is defined as an area on the outer surface of a sample container that is adapted to contain identifying information (i.e., a marking). The label may be an opaque paper, plastic, paint, or other material applied to the outer surface of the sample container. The marking may be a bar code, alphabetic characters, numeric characters, or a combination thereof. The label may be a manufacturer's label or may be a label applied later by the phlebotomist or by a subsequent sample processing entity, and may include a bar code.

[0058] "LA" is defined as the liquid-air interface and is the dividing line between the serum or plasma portion and the air above the serum or plasma portion (viewed from the side).

[0059] "SB" is the serum-blood interface, which is the dividing line between the serum or plasma portion and the precipitated blood portion (viewed from the side).

[0060] "TC" is the tube-cap interface, which is the dividing line at the interface between air and the cap (viewed from the side).

[0061] "HT" is the height of the tube and is defined as the height from the bottom-most portion of the tube to the bottom of the cap.

[0062] "HSP", in the case where no gel separator is used, is the height of the serum or plasma fraction and is defined as the height from the top of the serum or plasma fraction to the top of the sedimented blood fraction, ie, from LA to SB.

[0063] "HSP", where a gel separator is used ( Figure 3A ) is the height of the serum or plasma portion and is defined as the height from the top of the serum or plasma portion at LA to the top of the gel separator at SG, i.e., from LA to SG.

[0064] "HSB", in the case where no gel separator is used, is the height of the sedimented blood portion and is defined as the height from the bottom of the sedimented blood portion to the top of the sedimented blood portion at SB.

[0065] "HSB", in the case where a gel separator is used, is the height of the sedimented blood portion, and is defined as the height from the bottom of the sedimented blood portion to the bottom of the gel separator at BG.

[0066] "HTOT", in the case where no gel separator is used, is the total height of the specimen and is equal to HSP + HSB.

[0067] "HTOT", in the case where a gel separator is used, is the total height of the specimen and is equal to the height of HSP + HSB + gel separator.

[0068] "Tw" is the wall thickness of the sample container.

[0069] "W" is the outer width of the sample cell.

[0070] "Wi" is the inner width of the sample container.

[0071] A "carrier" is a device that is configured to support and transport sample containers, such as within a laboratory automation system (LAS).

[0072] "VSP" is the volume of the serum or plasma portion of the sample container.

[0073] "VSB" is the volume of the settled blood portion in the sample container.

[0074] "Hemolysis index," as used herein, means a grade given to a particular sample based on the determined level (degree or amount) of hemolysis present in the serum or plasma fraction.

[0075] "Icterus index," as used herein, means a grade given to a particular sample based on the determined level (degree or amount) of icterus present in the serum or plasma fraction.

[0076] "Lipidemia index," as used herein, means a grade assigned to a serum or plasma fraction based on the determined level (degree or amount) of lipemia present in the serum or plasma fraction.

[0077] "Convolution", as used herein, means a process step that learns and applies a filter kernel. During the forward pass, the filter is applied to the input image data by computing the dot product. This results in an activation map for the filter. Thus, the network learns a filter that activates when the process detects a certain type of feature at a certain spatial location in the input image data.

[0078] "Pooling", as used herein, means a processing step that performs non-linear downsampling. Typically, max pooling is applied. Max pooling is implemented by applying a max filter to non-overlapping sub-regions of a representation.

[0079] “SoftMax,” as used in this article, is a loss for predicting a single class among N mutually exclusive classes.

[0080] The ability to prescreen for HILN just after centrifugation and prior to analysis by one or more analyzers can advantageously minimize the time wasted analyzing samples that do not have the proper quality for analysis, can avoid or minimize erroneous test results, can minimize delays in patient test results, and can avoid waste of patient samples. Additionally, in one or more embodiments, remedial action can occur after prescreening for HILN, where unacceptable levels of H, I, and / or L are identified in the serum or plasma portion of the sample.

[0081] The sample containers provided for training the model ("training the CNN") can have different sizes, including different widths and different heights, and thus can be provided for training and later pre-screened in a plurality of different physical configurations. For example, the sample containers can have sizes such as, for example, 13 mm x 75 mm, 13 mm x 100 mm, 16 mm x 100 mm, and 16 mm x 125 mm. Other suitable sample container sizes may be encountered.

[0082] The methods described herein can use high dynamic range (HDR) image processing of the sample container and the serum or plasma portion as input to the "training CNN" and "testing CNN." HDR imaging involves capturing multiple exposures while using multi-spectral illumination, as will be apparent from the following.

[0083] In one or more embodiments, the quality inspection module can be configured to implement the training methods and testing methods described herein. The quality inspection module can be implemented in any suitable area of ​​the LAS, preferably where a robotic mechanism (e.g., a gripper robot) or track can facilitate the delivery of the sample container thereto. In some embodiments, the quality inspection module can be provided on or along the track of the sample testing device, and the carrier supporting the sample container can be configured to move the sample to an imaging position in the quality inspection module so as to accurately train to distinguish HILN using the image-based quality inspection module.

[0084] However, in some embodiments, training may alternatively occur on a training module at a location different from the test location of the sample testing device, wherein the training module is an exact replica of the quality inspection module used for pre-screening. In some implementations, the track can carry the sample to the pre-screening location, and also to one or more remote locations for analysis (e.g., clinical chemistry tests or experiments) on an analyzer on a carrier. In an embodiment, the training sample container can be held in an upright position during training (and testing), such as by a retainer. The retainer may include a finger or other suitable item that holds the sample container in an approximately vertical upright orientation during capture of the corresponding image. In some embodiments, the retainer may be part of the carrier.

[0085] In some embodiments, the training images may be obtained by a plurality of image capture devices positioned to capture the training images from a plurality of viewpoints (e.g., a plurality of side viewpoints). The plurality of training images may be obtained at the quality inspection module and may be captured at a plurality of exposures (e.g., exposure times) while providing illumination (e.g., backlighting) with a plurality of spectrums, each spectrum having a different nominal wavelength.

[0086] Multiple spectra of illumination can include, for example, illumination of emitted red (R), green (G), and / or blue (B) light. In other embodiments, white (W), infrared (IR), and / or near IR (NIR) can be used for illumination. Any combination of previously mentioned illumination sources can be used. Illumination can include backlighting, wherein the image capture device is located on one side, and the backlight source is on the opposite side of the sample container. The exposure time can vary based on the intensity and spectrum of the illumination used and the characteristics of the (multiple) image capture devices. Multiple exposure times (e.g., 4-8 different exposures) can be used for each spectrum and for each image capture device (i.e., for each viewpoint). Other numbers of exposures can be used.

[0087] In the image data preprocessing operation, for each corresponding pixel (or patch, if patch-by-patch processing is used) of a plurality of captured images under a particular spectrum (under different exposure times), the pixel (or patch) exhibiting the best image intensity can be selected. These selected pixels (or patches) exhibiting the best image intensity can also be normalized. The result can be a plurality of merged and normalized color image data sets, one for each different illumination spectrum (e.g., R, G, B, W, IR and / or NIR) and for each viewpoint. These merged and normalized data sets can be provided to the "training CNN" as a layer in the form of a data matrix. Once properly trained, the "test CNN" can be operated on the images (test images) taken from each test sample to determine the HILN. In some embodiments, the determination of the interference type (HIL) is provided as an output from the "test CNN". In other embodiments, the output of the "test CNN" can be n types of HIL or normal (N).

[0088] Further details of the inventive training method and the training quality check module configured to implement the training method are herein referred to Figure 1-8 and is further described.

[0089] Figure 1 The diagram shows a method for automatically processing a plurality of sample containers 102 containing samples 212 (see, for example, Figure 2-3B) of a sample testing device 100. The sample container 102 may be provided in one or more racks 104 at a loading area 105 before being transported to one or more analyzers (e.g., first, second and third analyzers 106, 108 and / or 110, respectively) arranged around the sample testing device 100 and analyzed by the one or more analyzers. A greater or lesser number of analyzers may be used. The analyzer may be any combination of a clinical chemistry analyzer and / or a test instrument, etc. The sample container 102 may be any suitable transparent or translucent container, such as a blood collection tube, a test tube, a sample cup, a test tube, or other transparent or opaque glass or plastic container capable of containing a sample 212 and allowing imaging of the sample 212 contained therein. The sample container 102 may vary in size, as discussed above.

[0090] The sample 212 may be provided to the sample testing device 100 in a sample container 102, which may be capped with a cap 214. The cap 214 may have different cap colors (e.g., red, royal blue, light blue, green, gray, tan, yellow, or a combination of colors) which may have implications as to what test the sample container 102 is used for, the type of additives included therein, whether the container includes a gel separator, etc. Other colors may be used.

[0091] Each of the sample containers 102 can be provided with a label 218, which can include identification information 218i (i.e., indicia) thereon, such as a bar code, alphabetical, numerical, or a combination thereof. The identification information 218i can be machine-readable at various locations around the sample testing device 100. The machine-readable information can be darker (e.g., black) than the label material (e.g., white) so that it has a contrast that can be easily imaged. The identification information 218i can indicate, or can otherwise be associated with, for example, the identification of the patient and the tests to be performed on the sample 212, or other information via a laboratory information system (LIS) 147. Such identification information 218i can be provided on a label 218, which can be adhered to or otherwise provided on the outer surface of the tube 215. In Figure 2 In the depicted embodiment, the label 218 may not extend all the way around the sample container 102 or completely along the length of the sample container 102. Thus, from the particular front viewpoint shown, a large portion of the serum or plasma portion 212SP is clearly visible and can be imaged (the portion shown in dotted lines) and is not obstructed by the label 218.

[0092] However, in some embodiments, multiple labels 218 may have been provided (such as from multiple facilities that have handled the sample container 102), and they may overlap with each other to some extent. For example, two labels (such as a manufacturer's label and a barcode label) may be provided and may overlap and may obscure (block) certain viewpoints. Thus, it should be understood that in some embodiments, although (multiple) labels 218 may obscure a portion of the sample 212 (the obscured portion), a portion of the sample 212 and the serum and plasma portion 212SP may still be visible from at least one viewpoint (the unobstructed portion). Thus, according to another aspect of the present disclosure, an embodiment of a "training CNN" configured to implement a training method can be trained to identify obscured and unobstructed portions, so that improved HILN detection can be provided.

[0093] Reference again Figure 2 , the sample 212 may include a serum or plasma portion 212SP and a precipitated blood portion 212SB contained within a tube 215. Air 216 may be provided above the serum and plasma portions 212SP, and the boundary therebetween is defined as a liquid-air interface (LA). The boundary between the serum or plasma portion 212SP and the precipitated blood portion 212SB is defined as a serum-blood interface (SB). The interface between the air 216 and the cap 214 is defined as a tube-cap interface (TC). The height (HT) of the tube is defined as the height from the bottommost portion of the tube 215 to the bottom of the cap 214, and may be imaged and used to determine the tube height. The height of the serum or plasma portion 212SP is (HSP), and is defined as the height from the top of the serum or plasma portion 212SP to the top of the precipitated blood portion 212SB. The height of the precipitated blood portion 212SB is (HSB), and is defined as the height from the bottom of the precipitated blood portion 212SB to the top of the precipitated blood portion 212SB at SB. HTOT is the total height of the sample 212 and is equal to HSP plus HSB.

[0094] In the case where the gel separator 313 is used ( Figure 3A), the height of the serum or plasma portion 212SP is (HSP) and is defined as the height from the top of the serum or plasma portion 212SP at LA to the top of the gel separator 313 at SG, where SG is the interface between the serum or plasma portion 212SP and the gel separator 313. The height of the precipitated blood portion 212SB is (HSB) and is defined as the height from the bottom of the precipitated blood portion 212SB to the bottom of the gel separator 313 at BG, where BG is the interface between the precipitated blood portion 212SB and the gel separator 313. HTOT is the total height of the sample 212 and is equal to the height of HSP plus HSB plus the height of the gel separator 313. In each case, Tw is the wall thickness, W is the outer width, which can also be used to determine the size of the sample container 102, and Wi is the inner width of the sample container 102.

[0095] In more detail, the sample testing device 100 can include a base 120 (e.g., a frame, a base plate, or other structure) on which a track 121 can be mounted. The track 121 can be a railed track (e.g., a single track or multiple tracks), a collection of conveyor belts, a conveyor chain, a movable platform, or any other suitable type of conveying mechanism. The track 121 can be circular or any other suitable shape, and in some embodiments can be a closed track (e.g., an endless track). The track 121 can be operated to transport individual ones of the sample containers 102 in the carrier 122 to various locations spaced around the track 121.

[0096] The carrier 122 may be a passive, non-motorized puck that may be configured to carry a single sample container 102 on the track 121, or alternatively may be an automated carrier that includes an on-board drive motor, such as a linear motor, that is programmed to move around the track 121 and stop at a pre-programmed position. Other configurations of the carrier 122 may be used. The carriers 122 may each include a holder 122H ( Figure 3B ), the holder is configured to hold the sample container 102 in a defined upright position and orientation. The holder 122H may include a plurality of fingers or leaf springs that secure the sample container 102 to the carrier 122, but some may be movable or flexible to accommodate different sizes (diameters) of the sample container 102. In some embodiments, the carrier 122 may be removed from the loading area 105 after being unloaded from the one or more racks 104. The loading area 105 may serve the dual function of also allowing the sample container 102 to be reloaded from the carrier 122 to the loading area 105 after pre-screening and / or analysis is completed. The delivery and routing of training samples may use the same approach.

[0097] A robot 124 may be provided at the loading area 105 and may be configured to grab a sample container 102 from the one or more racks 104 and load the sample container 102 onto a carrier 122, such as onto an input channel of a track 121. The robot 124 may also be configured to reload the sample container 102 from the carrier 122 to the one or more racks 104. The robot 124 may include one or more (e.g., at least two) robot arms or assemblies capable of X (lateral) and Z (vertical—out of the paper, as shown), Y and Z, X, Y and Z, and r (radial) and theta (rotational) motion. The robot 124 may be a gantry robot, an articulated robot, an R-theta robot, or other suitable robot, wherein the robot 124 may be equipped with a robot gripper oriented, sized, and configured to pick up and place the sample container 102.

[0098] When loaded onto the track 121, the sample container 102 carried by the carrier 122 can proceed to the first pre-processing station 125. For example, the first pre-processing station 125 can be an automated centrifuge configured to implement fractionation of the sample 212. The carrier 122 carrying the sample container 102 can be diverted to the first pre-processing station 125 via an inflow channel or other suitable robot. After being centrifuged, the sample container 102 can exit on the outflow channel, or be otherwise removed from the centrifuge by the robot, and continue along the track 121. In the depicted embodiment, the sample container 102 in the carrier 122 can then be transported to the quality inspection module 130 for training or pre-screening, as will be further described herein. The degree of HILN for each of the samples 212 and the container size in the rack 105 are known during training, and can thus be provided to the "training CNN" via correlation with the label 218.

[0099] The quality inspection module 130, once trained, is configured to pre-screen, to automatically determine the presence and possibly extent of H, I, and / or L contained in the sample 212, or whether the sample is normal (N). If found to contain substantially low amounts of H, I, and / or L so as to be considered normal (N), the sample 212 may continue on track 121 and may then be analyzed by the one or more analyzers (e.g., the first, second, and / or third analyzers 106, 108, and / or 110). Thereafter, the sample container 102 may be returned to the loading area 105 for reloading to the one or more racks 104.

[0100] The sample testing device 100 may include a plurality of sensors 116 at one or more locations around the track 121. The sensors 116 may be used to detect the position of the sample container 102 on the track 121 by reading identification information 218i, or information (not shown) such as provided on each carrier 122. Any suitable means for tracking the position may be used, such as proximity sensors. All sensors 116 may be docked with a computer 143 so that the position of each sample container 102 may always be known. The sensors 116 may be used during training to identify the sample 212 that has been routed to the quality inspection module 130 for training.

[0101] The pre-processing stations and analyzers 106 , 108 , 110 may be equipped with a robotic mechanism and / or an inflow channel configured to remove the carrier 122 from the track 121 , and a robotic mechanism and / or an outflow channel configured to re-enter the carrier 122 to the track 121 .

[0102] The sample testing device 100 can be controlled by a computer 143, which can be a microprocessor-based central processing unit CPU, with suitable memory and suitable regulating electronics and drivers for operating various system components. The computer 143 can be housed as part of the base 120 of the sample testing device 100 or separate from the base 120. The computer 143 can be operated to control the movement of the carrier 122 to and from the loading area 105, the movement around the track 121, the movement to and from the first pre-processing station 125 and the operation of the first pre-processing station 125 (e.g., a centrifuge), the movement to and from the quality inspection module 130 and the operation of the quality inspection module 130, and the movement to and from each analyzer 106, 108, 110 and the operation of each analyzer 106, 108, 110 for implementing various types of tests (e.g., tests or clinical chemistry). During training, the computer 143 can be used to complete the training of "training CNN". Optionally, a separate computer can implement model training.

[0103] For all but the quality check module 130, the computer 143 may control the sample testing device 100 according to software, firmware and / or hardware commands or circuits, such as those used on the Dimension® clinical chemistry analyzer sold by Siemens Healthcare Diagnostics of Tarrytown, New York, and such control is typical to one skilled in the art of computer-based electromechanical control programming and will not be further described herein. However, other suitable systems for controlling the sample testing device 100 may be used. Control of the quality check module 130 may also be provided by the computer 143, but according to the methods described in detail herein. Specifically, methods and apparatus for achieving rapid training of the sample testing device 100 are provided.

[0104] The computer 143 for image processing implemented for the characterization / testing and training methods described herein may include a CPU or GPU, sufficient processing power and RAM, and suitable storage. In one example, the computer 143 may be a PC equipped with a multiprocessor, which has one or more GPUs, 8GB of Ram or more, and terabytes or more of storage. In another example, the computer 143 may be a PC equipped with a GPU, or a PC equipped with a CPU that optionally operates in a parallelized mode. MKL, 8GB of RAM or more, and suitable storage may also be used.

[0105] The test embodiments of the present disclosure may be implemented using a computer interface module (CIM) 145 that allows a user to easily and quickly access various control and status display screens. These control and status display screens may display some or all aspects of a plurality of related automated devices for preparation and analysis of the specimen 212 and enable control of the plurality of related automated devices. The CIM 145 may be employed to provide information about the operational status of a plurality of related automated devices, as well as information describing the location of any specimen 212, and the status of the test to be performed on the specimen 212 or being performed on the specimen 212. The CIM 145 is thus adapted to facilitate interaction between an operator and the specimen testing device 100. The CIM 145 may include a display screen adapted to display a menu including icons, scroll bars, boxes, and buttons through which an operator may interface with the specimen testing device 100. The menu may include a plurality of functional elements programmed to display and / or operate functional aspects of the specimen testing device 100.

[0106] Before explaining the rapid training method in detail, the configuration of the quality check module 130 which can also be used for training will be described. Figure 4A-4B As shown in , a quality check module 130 configured to implement the HILN characterization and training method is provided. The quality check module 130 can be configured to image the sample 212 and be adapted to pre-screen for the presence of interfering substances (e.g., H, I and / or L or N) in the sample 212 (in its serum or plasma portion 212SP) prior to analysis by the one or more analyzers 106, 108, 110.

[0107] Reference now Figure 1 , 4A 4B, a first embodiment of the quality inspection module 130 is shown, which includes a plurality of image capture devices 440A-440C. Three image capture devices 440A-440C are shown and are preferred, but a single image capture device, two or more, or four or more may alternatively be used. The image capture devices 440A-440C may be any suitable device for capturing well-defined digital images, such as a conventional digital camera capable of capturing pixelated images, a charge coupled device (CCD), an array of photodetectors, one or more CMOS sensors, and the like. For example, in Figure 4A The image capture devices 440A, 440B, 440C illustrated in FIG. 4 are configured to capture images from three different side viewpoints (viewpoints labeled 1, 2, and 3). For example, the captured image size may be, for example, approximately 2560×694 pixels. In another embodiment, the image capture devices 440A, 440B, 440C may capture an image size that may be, for example, approximately 1280×387 pixels. Other image sizes and pixel densities may be used.

[0108] Each of the image capture devices 440A, 440B, and 440C can be configured and operable to capture a side image of at least a portion of the specimen container 102 and at least a portion of the specimen 212. For example, the image capture devices 440A-440C can capture an image 315 such as Figure 3B , which includes portions of the label 218 and portions or all of the serum or plasma portion 212SP. For example, in some instances, portions of viewpoints 1-3 may be partially obscured by the label 218. In some embodiments, one or more of the viewpoints 1-3 may be completely obscured, i.e., no clear view of any serum or plasma portion 212SP is possible. However, even in a case where one side (front or back) of a viewpoint 1-3 is completely obscured by one or more labels 218, the characterization method may still be able to distinguish the boundaries of the serum or plasma portion 212SP by the one or more obscured labels 218 when properly trained.

[0109] In the illustrated embodiment, the plurality of image capture devices 440A, 440B, 440C are configured to capture side images of the sample container 102 and the sample 212 at the imaging position 432 from a plurality of viewpoints 1-3. The viewpoints 1-3 may be spaced so that they are approximately equally spaced from each other, such as approximately 120° from each other, as shown. As depicted, the image capture devices 440A, 440B, 440C may be arranged around the track 121. Other arrangements of image capture devices may be used. In this way, when the sample container 102 resides in the carrier 122 at the imaging position 432, an image of the sample 212 in the sample container 102 may be taken. The fields of view of the plurality of images obtained by the image capture devices 440A, 440B, and 440C may overlap slightly in the range of the circumference.

[0110] In one or more embodiments, the carrier 122 may be stopped at a predetermined position in the quality inspection module 130, such as at the imaging position 432, i.e., such as at a point where the normal vectors from each of the image capture devices 440A, 440B, 440C intersect each other. A gate or linear motor of the carrier 122 may be provided to stop the carrier 122 at the imaging position 432 so that a plurality of quality images may be captured there. Training images may be captured at the imaging position 432 during training. In embodiments where a gate is present at the quality inspection module 130, one or more sensors, such as the sensor 116, may be used to determine the presence of the carrier 122 at the quality inspection module 130.

[0111] Image capture devices 440A, 440B, 440C may be provided in close proximity to imaging location 432 and focused to capture an image window having the dimensions of image 315 at imaging location 432, wherein the image window is an area that includes the expected location of specimen container 102. Thus, in some embodiments, specimen container 102 may be stopped so that it is approximately in the center of the image window, and thus at the center of image 315.

[0112] In the operation of the quality inspection module 130, each training and test image can be triggered and captured in response to a trigger signal provided in the communication lines 443A, 443B, 443C and sent by the computer 143. Each of the captured images can be processed by the computer 143. In a particularly effective method, high dynamic range (HDR) processing can be used to capture and process image data from the captured images. In more detail, when illuminated sequentially with one or more different spectrums, multiple images of the sample 212 are captured at the quality inspection module 130 with multiple different exposures (e.g., with different exposure times). For example, each image capture device 440A, 440B, 440C can take 4-8 images of the sample container 102 including the serum or plasma part 212SP with each of the multiple spectrums and at different exposure times. For example, when the sample 212 is backlit by a light source 444A with a red spectrum, 4-8 images can be taken by the image capture device 440A at viewpoint 1. Each image can be at a different exposure. Additional similar images may be taken sequentially, for example at viewpoints 2 and 3 .

[0113] In some embodiments, multiple spectral images can be realized by using different light sources 444A-444C that emit different spectrum illumination. Light sources 444A-444C can backlight the sample container 102 (as shown). Multiple different spectrum light sources 444A-444C can be RGB light sources, such as LEDs emitting nominal wavelengths of 634nm + / - 35nm (red), 537nm + / - 35nm (green) and 455nm + / - 35nm (blue). In other embodiments, light sources 444A-444C can be white light sources. In the case where the label 218 blurs multiple viewpoints, IR backlighting or NIR backlighting can be optionally used. In addition, RGB light sources can be used in some instances, even when label shielding exists. In other embodiments, light sources 444A-444C can emit one or more spectra with a nominal wavelength between about 700nm and about 1200nm.

[0114] By way of non-limiting example, to capture images at a first wavelength, three red light sources 444A-444C (wavelength of approximately 634 nm + / - 35 nm) may be used to sequentially illuminate the specimen 212 from three side positions. Red illumination by the light sources 444A-444C may occur while capturing multiple images (e.g., 4-8 images or more) with different exposure times from each viewpoint 1-3 by each image capture device 440A-440C. In some embodiments, the exposure time may be between approximately 0.1 ms and 256 ms. Other exposure times may be used. In some embodiments, each of the corresponding images for each image capture device 440A-440C may be captured, for example, sequentially. Thus, for each viewpoint 1-3, a set of images is sequentially obtained with red spectrum backlighting and multiple (e.g., 4-8 exposures, such as with different exposure times). For example, images may be captured in a round-robin fashion, where all images from viewpoint 1 are captured, followed sequentially by viewpoints 2 and 3, and stored in the memory of controller 143. This sequence may be implemented for training images as well as for later testing of captured images.

[0115] In each embodiment, the quality inspection module 130 may include a housing 446 that may at least partially surround or cover the track 121 to minimize external lighting effects. The sample container 102 may be located inside the housing 446 during the image capture sequence. The housing 446 may include one or more doors 446D to allow the carrier 122 to enter the housing 446 and / or exit the housing 446. In some embodiments, the top plate may include an opening 446O to allow the sample container 102 to be loaded into the carrier 122 from above by a robot including movable robotic fingers.

[0116] Once in Figure 4A-4B In an embodiment where an image of red illumination is captured, another spectrum, such as a green spectrum light source 444A-444C, can be sequentially turned on (a nominal wavelength of approximately 537nm with a bandwidth of approximately + / -35nm), and multiple images (e.g., 4-8 or more images) with different exposure times can be sequentially captured by each image capture device 440A, 440B, 440C. This can be repeated for each image capture device 440A, 440B, 440C using a blue spectrum light source 444A-444C (a nominal wavelength of approximately 455nm with a bandwidth of approximately + / -35nm). The different nominal wavelength spectrum light sources 444A-444C can be implemented by a light panel that includes a library of different desired spectrum light sources (e.g., R, G, B, W, IR and / or NIR) that can, for example, be selectively turned on and off. Other means for backlighting can be used.

[0117] Multiple images taken at multiple exposures (e.g., exposure times) for each corresponding wavelength spectrum may be obtained in rapid succession, such that a complete set of backlit images of the sample container 102 and the sample 212 from multiple viewpoints 1-3 may be obtained, for example, in less than a few seconds. In one example, four different exposure images for each wavelength at three viewpoints 1-3 using image capture devices 440A, 440B, 440C and backlighting with RGB light sources 444A-444C would result in 4 images × 3 spectra × 3 image capture devices = 36 images. In another example, four different exposure images for each wavelength at three viewpoints using image capture devices 440A, 440B, 440C and backlighting with R, G, B, W, IR, and NIR light sources 444A-444C would result in 4 images × 6 spectra × 3 cameras = 72 images. This sequence may be used for training and also for testing, as will be apparent.

[0118] According to an embodiment of the characterization method, the processing of the image data may involve a pre-processing step, which includes, for example, selecting the best-exposed pixels from a plurality of captured images at different exposure times under each wavelength spectrum and for each image capture device 440A-440C, so that best-exposed image data is generated for each spectrum and for each viewpoint 1-3. This is a pre-processing step, referred to herein as "image merging".

[0119] For each corresponding pixel (or patch), for each of the images from each of the image capture devices 440A-440C, a pixel (or patch) exhibiting an optimal image intensity may be selected from each of the different exposure images for each viewpoint 1-3. In one embodiment, the optimal image intensity may be, for example, a pixel (or patch) that falls within a predetermined intensity range (e.g., between 180-254 on a scale of 0-255). In another embodiment, the optimal image intensity may be, for example, between 16-254 on a scale of 0-255. If more than one pixel (or patch) in the corresponding pixel (or patch) position of the two exposure images is determined to be optimally exposed, the higher of the two may be selected.

[0120] The selected pixels (or patches) exhibiting the optimal image intensity can be normalized by their corresponding exposure times. The result is a plurality of normalized and merged spectral image data sets for the illumination spectrum (e.g., R, G, B, white light, IR, and / or IR, depending on the combination used) and for each image capture device 440A-440C, wherein all pixels (or patches) are optimally exposed (e.g., one image data set per spectrum) and normalized. In other words, for each viewpoint 1-3, the data preprocessing implemented by the computer 143 results in a plurality of optimally exposed and normalized image data sets, one for each illumination spectrum employed. The data sets obtained in the described manner can be used for both training and testing (once the system is trained).

[0121] exist Figure 5A 1 shows functional components of one embodiment of a training device 500TR configured to implement a training method. The training device 500TR may be embodied as a quality inspection module 130 and configured as a training device. As discussed above, at the imaging position 432 ( Figure 4A-4B ), a specimen container 102 including a specimen 212 is provided at a location 104. Multiple training images of each specimen container type may be captured while containing training specimens 212. The training images are captured at an imaging location (e.g., at imaging location 432) or at a similar location that includes a replication of the lighting conditions of imaging location 432.

[0122] In 504, a multi-spectral, multi-exposure image is captured by one or more image capture devices (e.g., image capture devices 440A-440C or a subset thereof). Image data for each of the multi-spectral, multi-exposure images may be pre-processed in 506, as discussed above, to provide a plurality of combined, optimally exposed, and normalized image datasets (hereinafter, "image datasets"). During testing following the training method, a similarly generated image dataset from the test specimen 212 may be fed as a layer (as a matrix) and input to the "test CNN 535 (TE)" of the "test CNN 500TE". Figure 5B )” which has been previously trained to identify HILNs and to classify between HILNs. As a result, a determination of the HILN for the serum or plasma portion 212SP of any imaged sample 212 can be provided.

[0123] During the training method (before testing), multiple sets of training samples 212 are used to train the "training CNN535TR". According to the training method, the "training CNN 535TR" is trained by imaging many sample containers 102 containing training samples 212 (for example, using the quality inspection module 130 or a substantial copy thereof). The number of sample images may include thousands (for example, 5000 or more, 10,000 or more, or even 15,000 or more). The image is obtained in 504, as discussed above, and pre-processing of the image data may be included in 506 to provide image merging and normalization. Ground truth labels 508 are provided as training inputs to the "training CNN535TR", which, together with the captured images from the image capture 504, are used to establish corresponding HILN values ​​(for example, interfering substance types and / or HIL index levels) for each training sample 212. The size of the sample container 102 may also be a training input.

[0124] In the depicted embodiment, “Test CNN 535” ( Figure 5B ) is n-type hemolysis, n-type icteric, n-type lipemia, and normal (N). Thus, the ground truth label 508 as input will include n values ​​of HIL and the expected range of values ​​across HIL and N. The HILN training value input as the ground truth label 508 has been predetermined for each training sample 212 via a separate HILN test, for providing the interfering substance type or N, and if there is an interfering substance, then possibly providing its index. In the "test CNN 535", the output of the "test CNN 535" can be mapped to n set point values ​​(where n is an integer value) for each of hemolysis, icteric, and lipemia and one set point value for N normal. The value of n can be, for example, between 3 and 7. However, other values ​​of n can be used. For example, the "test CNN 535" result can be assigned to a specific result based on being within a certain pre-established range for each n (e.g., for n=3, the range can then be + / - 16.5%). Any suitable fusion scheme may be employed for mapping to one of the available outputs of the "Test CNN 535".

[0125] Reference again Figure 5A , “Train CNN 500TR” can use the candidate region generator 511 to generate the candidate region 319 ( Figure 3C). The candidate region 319 may be a randomly generated region that is a sub-region of a region of the image 315. The candidate region 319 is an attempt to provide an estimate of a region within the image 315 that corresponds in area and position to the serum or plasma portion 212SP of the training sample 212. The candidate region 319 generated in the candidate region generator 511 may be generated by and suitable means. In one embodiment, the candidate region 319 may be randomly generated by the candidate region generator 511. For example, the candidate region 319 may be generated by randomly selecting a sub-region (e.g., a rectangular box) from a virtual grid 317 superimposed on the image 315 by any suitable means, such as Figure 3C as shown in .

[0126] The virtual grid 317 may include a plurality of grid elements 317G of thickness T arranged along the X direction and height H arranged along the Y direction. For example, the number of levels of T in the X direction of the virtual grid 317 may be between about 10 and about 20 T units wide along the width of the sample container 102, and the number of levels of H in the Y direction may be between 20 and 40 units along the length / height of the sample container 102. Other numbers of grid elements 317G in the X and Y directions may be used. The levels in X and Y should be fine enough to enable a good representation (e.g., 90% match area or greater) of the area of ​​the serum or plasma portion 212SP of each of the training samples 212 to be found. Reference Figure 3C , an example of a poor candidate region 319B is shown as a rectangular box in the upper left region of the image 315, while a good candidate region 319G is shown as closely corresponding in position and size to the region of the serum or plasma portion 212SP. If enough random candidates are finally generated in 511, each of the training images 315 will approximately match a suitably close candidate region 319, i.e., approximately match the region, position, and shape of the serum or plasma portion 212SP.

[0127] Candidate areas can be generated by any suitable means. In some embodiments, the diagonal corners of candidate areas 319 can be generated by randomly selecting two nodes 317N in the virtual grid 317. The selected nodes are the diagonally opposite corners of the candidate areas 319. Any suitable selection scheme can be used to select the nodes / corners of the candidate areas 319. For example, the candidate area 319B of the reference difference, the lower left node (X1, Y1) and the upper right node (X2, Y2) can be randomly selected and define the lower left and upper right corners of the candidate area 319 in the virtual grid 317. Multiple such candidate areas 319 (e.g., thousands) can be generated. Thus, candidate areas 319 of various rectangular shapes (width and height) and positions in the virtual grid 317 will be generated and include multiple different sub-areas in the virtual grid 317. Some of the candidate regions 319 will be good fits (e.g., good candidate regions 319G) to regions of the serum or plasma portion 212SP of a particular training sample 212, while others will not be (e.g., poor candidate regions 319B). Thousands of images 315 will be captured and processed, and thousands of candidate regions 319 will be generated by the candidate region generator 511.

[0128] As will be appreciated, the fit of the candidate regions 319 will be tested / verified, and the best of the candidate regions 319 (e.g., the top 2000 or more candidate regions) will be selected as being the best fit ("match") to the region of the serum or plasma portion 212SP of the training specimen 212. In some embodiments, the candidate region generator 511 may be configured so that the same rectangular candidate region is never generated twice. Other means for selecting the rectangular shape of the candidate regions 319 within the virtual grid 317 may be employed. For example, a center node 317N may be selected, and then a random width and a random height may be randomly selected to generate each candidate region 319.

[0129] A suitable intelligent convergence scheme can be used to guide the candidate regions 319 toward the relevant regions of the serum or plasma portion 212SP. For example, various means can be used to reduce or limit the area within the virtual grid 317 where the candidate regions 319 will be generated, or otherwise reduce the size of the virtual grid 317. For example, Figure 3D As shown in , having knowledge of the width W of the sample container 102 can limit the size of the virtual grid 317 to only the region that includes the width W. For example, the width W of the sample container 102 can be provided as a ground truth label in 508.

[0130] Candidate area 319 is then Figure 5AThe training period shown in is provided as input to the neural network (eg, to "training CNN 535"). The generation of candidate regions 319 is automatic and does not require graphical input (eg, graphical contour annotation) by an operator, as required in the prior art.

[0131] In some embodiments, historical data on known sizes and locations of regions of serum or plasma 212SP can be used to further constrain the starting size of the virtual grid 317 from which candidate regions 319 are generated by the candidate region generator 511. In addition to thousands of training images 315, thousands of candidate regions 319 of varying sizes and locations are fed into the "training CNN 535" during training. After a suitable number of images 315 and candidate regions 319 are input, the convergence of the neural network (e.g., "training CNN 535") can be tested to determine whether the candidate region 319 inputs have provided sufficient performance of the neural network (e.g., CNN 535) so far, as discussed above.

[0132] To test / verify whether a candidate region 317 is a good match to the serum or plasma portion 212SP of the training sample 212, the intensity gradients within the various candidate regions 319 may be analyzed. For example, each grid element 317G in each of the training images 315 may include a plurality of rows and columns of pixels, and each pixel may have an associated intensity value for each spectrum. Certain portions of the candidate regions 317 may include grid elements 317G with relatively high intensity gradients. Such an intensity gradient may be the average intensity of the pixels (or patches within a particular grid element 317G).

[0133] Each candidate region 319 can be tested / verified as follows: by filtering, such as by examining the light intensity gradient within each of the candidate regions 319. For example, the degree of match can be discerned by examining the gradient of the light intensity within only certain regions of the training image 315. For example, the gradient along the outer boundary of each candidate region 317 can be quantified first. A relatively high gradient within one or more sub-regions of the candidate region 317 indicates a change in intensity that can be correlated with the presence of an interface, such as, for example, a liquid-air interface (LA), a serum-blood interface (SB), or a serum-gel interface (SG). Thus, the upper interface (LA) and / or the lower interface (SB) or the serum-gel interface (SG) can be discerned based on the change in the identification intensity within the sub-region and the ground truth label 508 input with the image 315.

[0134] In some embodiments, the intensity gradients within individual grid elements 317G within the virtual grid 317 positioned along the periphery of the generated candidate region 319 (which has, for example, a rectangular shape) can be quantified. These intensity gradient values ​​(e.g., average intensity gradients) for each of the grid elements 317G located at the periphery of the candidate region 319 can be summed for each candidate region 319, and / or summed and averaged for each candidate region 319. A large sum or average value can mean that there is a good correlation (e.g., a good match) between the imaged serum or plasma portion 212SP and the candidate region 319. A relatively small sum or average value can mean a poor correlation (e.g., a poor match). Thus, from the plurality of candidate regions 319, a plurality of best candidate regions (e.g., thousands of best matching candidate regions) can be selected based on the degree of "matching" with the input training image 315, i.e., those candidate regions 319 with the highest sum or average value are selected, and the others can be discarded or not used.

[0135] For example, in image 315, the liquid-air interface LA is expected to be located between a certain range of rows of the grid 317G (e.g., based on historical data) and the known dimensions of the sample container 102. Similarly, the serum-blood interface SB is expected to exist between another range of rows of the grid 317G (e.g., again based on historical data). In the case where a gel separator 313 is provided, the serum-gel interface SG is expected to exist between yet another range of rows of the grid 317G. Along the horizontal dimension, a change in intensity for a grid element 317G may mean that there is an edge of a tube 215 (tube / air interface), or there is an edge of a label 218. "Training CNN 535TR" automatically determines candidate regions 319 that may correspond to a serum or plasma portion 212P by checking the bounding box hypothesis using available gradient intensity information visible in the HDR image. Due to the characteristics of the various regions visible in the sample 212, the transition between a fluid region (e.g., a serum or plasma portion 212SP) and air 216 results in a clear intensity gradient, which can be quantified and used to check whether the candidate bounding box captures the transition. Regions such as precipitated blood portion 212SB, gel portion 313, or air 216 may be selected as candidate generated portions during training, however, convergence of the learning process will not be satisfied because the content is not discriminative enough to reduce the loss of the SoftMax classifier during training.

[0136] The change in intensity within the grid elements 317G in these regions can indicate that such an interface or side exists. The existence of the interface can be confirmed in addition by: these intensity gradients exist as patterns within the grid 317, such as along the rows of the grid 317 or along the columns of the grid 317. The intensity gradient can be determined by comparing the average intensity values ​​of the grid elements 317G along the periphery. In order to further restrict to only feasible areas, the grid sub-areas of the virtual grid 317 identified as the areas where the serum or plasma part 212SP may be present can be checked, and those grid elements 317G outside the expected sub-areas can be ignored. For example, if the ground truth label 508 includes the tube height HT, then the area that can only contain the cap 214 or the precipitated blood part 212SB can be ignored, and the candidate area 319 can only be selected from the sub-areas of the grid 317 that do not include these areas. For example, a virtual mask or other virtual means can be used to exempt certain areas of the grid 317 as being able to be selected. In other embodiments, knowledge of the width W of the sample container 102 as the ground truth label 508 can be used to reduce the size of the virtual grid 317. Thus, due to the fewer options for candidate areas 319 that can be used for testing / verification, convergence should be faster. The diameter and / or tube height input can be provided once as the ground truth label 508 during the training method, and then training using a large number of samples with the same size can be started before switching to another size. Thus, when the size is changed, only the diameter and / or height input need to be entered as the ground truth label 508.

[0137] exist Figure 5B In the current embodiment, the outputs from the "test CNN 535" at output 529 are n categories of each hemolysis (H1-H3), n categories of icterus (I1-I3), and n categories of lipemia (L1-L3). An output from the "test CNN 535" at 529 that is normal (N) is also available. As with training, during testing, the quality inspection module 130 may include an image capture device (e.g., 440A-440C) configured to capture multiple images of a sample container 102 containing a serum or plasma portion 212SP of a test sample 212. A computer 143 coupled to the image capture device may be configured and operable to input image data from the multiple images into a neural network (e.g., Figure 5BThe "test CNN 535" is used to generate candidate regions for the serum or plasma portion. The candidate regions may be generated by the candidate region generator 511. The bounding box regressor 515 may function to converge the candidate region 319 to the actual size, shape, and position of the serum or plasma portion 212SP of the test sample 212. The bounding box regressor 515 converges by geometrically comparing the input candidate region 319 with the response of the regressor 515 (the output of the network). If a suitable region match is obtained, the regressor 515 has converged. Region matches greater than 80% may be considered a match, but even higher percentages (e.g., greater than 90%, greater than 95%, greater than 97%) may provide even more accurate results. The output from the neural network ("test CNN 535") is a classification of the serum or plasma portion 212SP as one or more of hemolytic, icteric, lipemic, and normal.

[0138] Each (pixel or patch) within the candidate region 319 generated by the candidate region generator 511 is processed by the "test CNN 535" and has an output for one of the HILNs. These per-pixel (or per-patch) results can be mapped to output channels (e.g., 1-10) and summed or fused by any suitable method, and the HILN is determined based on which of the HILNs has the highest count within the final candidate region.

[0139] To provide an improved final vote for the HILN, in some embodiments, post-processing may be provided to iterate over the counts from other viewpoints (e.g., viewpoints 1-3). Thus, a histogram over all occurrences (counts) may be generated, and an average HILN for sample 212, considering all viewpoints, may be generated.

[0140] In more detail, as before, during training, during testing, a multi-spectral, multi-exposure merged and normalized image dataset is input into the "test CNN 535" as an image dataset, and operated on and processed by the "test CNN 535". The output of the processing using the "test CNN 535" can be multiple output possibilities (n classes) for each of the HIL or N, and of course for each viewpoint.

[0141] For the test device 500TE, a per-channel confidence level (CL) for the HILN may be used, where each channel 1-10 is associated with a type of a particular class at 529 (e.g., H1, H2, H3, I1, I2, I3, L1, L2, L3, and N). Thus, for each pixel (or patch) in the final candidate region (matching candidate) determined by the bounding box regressor 515, an output class (H1-N) is provided. These per-pixel (or per-patch) outputs may be fused by any suitable post-processing routine to aggregate or otherwise process the results for each channel (e.g., channels 1-10 as shown), and arrive at an overall determination of the HILN from the available classes (e.g., H1, H2, H3, I1, I2, I3, L1, L2, L3, and N) at output 529. Any suitable voting scheme may be used, such as final pixel class = max CL, and then summing the number of maximum CLs for the serum or plasma portion 212SP. This same process can be implemented per viewpoint. The classification results for the various viewpoints used can be summed or otherwise combined or averaged.

[0142] exist Figure 5A-5B An example architecture of a "training CNN 535TR" and a "testing CNN 535" that can be used is shown in FIG. The CNNs described herein can be encoded using any suitable scientific computing framework, program, or toolbox, such as, for example, Caffe, Theano, a Python framework for fast computation of mathematical expressions, TensorFlow, Torch, etc. available from the Berkeley Vision and Learning Center (BVLC).

[0143] In more detail, CNN 535 TR, 535 may include a suitable number of operating layers to provide deep learning. For example, CNN 535TR, 535 may include an architecture including at least two layers, including convolution (Conv1, Conv2) and pooling (Pool 1 and Pool 2), and at least two additional full convolution (FC) layers. In the depicted embodiment, two operating layers 538, 540 and an additional operating layer 542 may be provided. The full convolution (FC) layer may be provided after the at least two operating layers 538, 540 and before each of the bounding box regressor 515 and the SoftMax 548. The loss layer 544 may be provided in the "training CNN 535TR" and may include a bounding box regressor 515 and a SoftMax 548. The described architecture of CNN 535TR, 535 may be used to find a suitable candidate region 319 for each corresponding image. As will be appreciated, the regressor 515 provides a potential bounding box annotation for the serum / plasma portion 212SP. During training, candidate bounding boxes (candidate regions 319) are provided to the regressor 515. The results are given by the network as a consequence of taking into account the image content. If the deviation between the input to the regressor 515 and the converged bounding box reaches a given minimum error (e.g., a suitably low geometric difference between the four corner points or a % area match greater than some suitable percentage), which can be robustly calculated over a given set of input images (e.g., 10,000 training samples 212), then the regressor 515 can be assumed to have converged.

[0144] In the "test CNN 535", once a suitable candidate region 319 is obtained, each layer of the image data of the candidate region can be scanned using a moving window approach to determine the output at channels 1-10, and post-processed to fuse the output at 529, which can be fused based on any suitable voting scheme. The moving window of the input patch can be, for example, a 64×64 patch (64×64 pixels). However, patches of other sizes can be used. For example, three main operation layers (Conv 1 + Pool 1, Conv 2 + Pool 2, and Conv 3 + Pool 3) are shown. The first layer 538 can extract very local structural edges; the second layer 540 can learn textures, which are combinations of edges; and the third layer 542 can form a part. Each of the layers 538, 540, 542 of the CNN 535, 535TR benefits from the multi-channel input being processed (e.g., multi-spectral, multi-exposure information). These operations on various input layers, and in particular three input layers (e.g., RGB), can be easily handled and represented by a deep learning network. The framework naturally integrates low, medium, and high-level features and leads to multi-layer classification. The features obtained from the "training CNN535TR" (e.g., a network with training weights) are stored in the computer 143 and can be used for later testing. Training can be until a suitable confidence level in the "training CNN 535TR" is achieved.

[0145] Reference again Figure 5A and 5B, CNN 535TR, 535 may include a first layer 538, which includes a convolutional layer (Conv 1), which may include, for example, 10 filters of size 5×5×12. Other numbers of filters and filter sizes may be used. The resulting 10 feature maps are then fed to a maximum pooling (Pool 1), which may be separated for each channel and take the max (maximum value) over a 2×2 spatial neighborhood with a stride of 1. The purpose of this layer is to extract low-level features, especially simple edges. This is followed by a second layer 540, which includes a convolutional layer (Conv 2) and a maximum pooling layer (Pool 2), which may have 10 filters of size 5×5×20, and the maximum pooling layer (Pool 2) may take the max (maximum value) over a 3×3 spatial neighborhood with a stride of 2. The purpose of the second layer 540 is to learn different combinations of simple edges to form textures. Thereafter, the resulting feature map is fed into the third layer 542, which includes a convolutional layer (Conv 3) and a maximum pooling layer (Pool 3), wherein the convolutional layer (Conv3) may have 20 filters of size 3×3×20, and the maximum pooling layer (Pool 3) may take the max (maximum value) over a 2×2 spatial neighborhood with a stride of 2 for learning combinations of textures to form parts. The previously mentioned maximum pooling layer makes the output of the convolutional network more robust to local translation. Finally, the top layer is a fully convolutional (FC) layer, where each output unit is connected to all inputs. These layers are able to capture correlations between parts. The output of the last fully convolutional layer (FC) is fed into an n-way SoftMax (where n=3 is the number of possible output classes), which produces a distribution over n-class output channels. In short, a moving window of input patches, where each patch is classified using the trained CNN535, gives a response toward one of the n classes. The final candidate region found after regression through the bounding box regressor 515 identifies the region to be classified. All regions outside this region are ignored. Classification for the corresponding image data set can be based on a majority vote result from these outputs. Additional steps can be applied, where the results per viewpoint can be fused over multiple viewpoints to obtain a confident decision, such as by averaging the results of multiple viewpoints.

[0146] Fig. 6A and 6B The diagram shows images from image capture devices 440A, 440B (see Fig.6D ) of the first viewpoint 1 and the second viewpoint 2 of the front semantic image 615A, 615B. Fig. 6A and 6BIn the embodiment, some of the serum or plasma portion 212SP is shielded by the label 218 and is not directly visible from the light sources 444A, 444B ( Fig.6D ) is blocked by the rear view label 218B (i.e., the portion of the serum or plasma portion 212SP that is obscured is shown by dotted lines).

[0147] Figure 6C The image is captured from the image capture device 440C ( Fig.6D ) of the viewpoint 3. Figure 6C , from viewpoint 3, all serum or plasma portions 212SP are occluded by label 218. For each of these viewpoints 1, 2, 3, the "test CNN 535" can output a HILN for the serum or plasma portions 212SP. In addition, the characterization method can fuse the results for each viewpoint 1, 2, 3 and provide an overall determination of the HILN. In each case, the "test CNN 535" considers the visible serum or plasma portions 212SP, but also considers the area occluded by the back view label 218B.

[0148] Thus, the "test CNN 535TE" outputs an n-class output or N at 529. For example, if the majority of the pixels (or patches) from the final candidate area are classified as N, then the serum or plasma portion 212SP can be classified as normal (N). If the majority of the pixels (or patches) are classified as H, then the serum or plasma portion 212SP can be classified as containing hemolysis (e.g., mapped to H1, H2, or H3). Similarly, if the majority of the pixels (or patches) of the final candidate area are classified as I or L, then the serum or plasma portion 212SP can be classified as icteric (e.g., mapped to I1, I2, or I3) or lipemia (e.g., mapped to L1, L2, or L3) respectively. In some embodiments, a weighted majority voting scheme can also be used to classify the sample 212 by using the probability (CL) from the HILN result as a weight. Other means for characterizing the serum or plasma portion 212SP as a whole can be used.

[0149] Figure 7A flow chart of a training method 700 according to an embodiment of the present disclosure is illustrated. The training method 700 may be implemented by the quality inspection module 130 (or a substantial copy thereof) and may be operable to train a neural network (e.g., "training CNN535TR"), as described herein. Specifically, according to one or more embodiments, the training method 700 may be used to establish features of a test neural network (e.g., "test CNN 535") to determine the presence or normality N of interfering substances (e.g., H, I, and / or L) of a serum or plasma portion 212SP of a sample 212. Advantageously, the training method 700 does not require any manual graphical annotation (e.g., graphical contouring of the serum or plasma portion 212SP or other regions). The training method 700 includes, at 702, capturing a training image of a sample container (e.g., sample container 102) including a training sample (e.g., sample 212) at an imaging position (e.g., imaging position 432). The training sample 212 includes a serum or plasma portion (e.g., serum or plasma portion 212SP). Capturing multiple images may be from multiple viewpoints (e.g., viewpoints 1, 2, and 3). Additionally, the sample container 102 may include one or more labels thereon (e.g., label 218). The imaged initial sample container 102 may be free of label 218 to aid in training efficiency. The captured training images may be digital, pixelated images captured using one or more image capture devices (e.g., image capture devices 440A-440C).

[0150] The training method 700 may further optionally include background removal or tube type detection in 704. This image data preprocessing 506 may occur by capturing images of an empty sample container 102 at the imaging location 432 under various lighting and exposure conditions used to train the CNN, and then subtracting those pixel intensity values ​​to leave only the intensities corresponding to the sample 212 and label 218. Optionally, if the width W of the sample container 102 is known from the ground truth label 508 input, then the width of the virtual grid 317 may be reduced. As discussed above, the raw image data comprising the multi-spectral, multi-exposure images captured by the image capture devices 440A, 440B, 440C in 504 may also be merged and normalized in the image data preprocessing 506.

[0151] The training method 700 further includes generating candidate regions (e.g., candidate regions 319) of the serum or plasma portion (e.g., serum or plasma portion 212SP) for input to a neural network (e.g., CNN 535TR) at 706. Image data (e.g., a merged and normalized image data set) from a plurality of captured images (e.g., image 315) along with the candidate regions 319 are submitted to a neural network (e.g., "training CNN 535TR") and processed thereby. The processing may be accomplished by the computer 143 described herein.

[0152] The training method 700 further includes determining which of the candidate regions (e.g., candidate regions 319) provide the best match with the serum or plasma portion 212SP of the training sample 212 at 708. The best match can be selected by taking the top percentage or number of the candidate regions 319 based on the peripheral intensity gradient data, as described above. However, other means for selecting the best match can be used.

[0153] According to the method, once the "training CNN 535TR" is sufficiently trained, the obtained features (e.g., networks and weights) from the "training CNN535TR" can then be used in a test neural network (e.g., "testing CNN535") in 710. Therefore, based on the foregoing, it should be apparent that an improved training method 700 is provided that facilitates training a neural network (e.g., a CNN) by substantially eliminating manual graphical annotation input of the serum or plasma portion 212SP of the training sample 212.

[0154] Figure 8 A flow chart of a testing method 800 according to an embodiment of the present disclosure is illustrated. The testing method 800 may be implemented by the quality inspection module 130 as described herein. Specifically, according to one or more embodiments, the testing method 800 may determine the presence of interfering substances in the sample 212. The characterization method 800 includes capturing multiple images of a sample container (e.g., sample container 102) containing a serum or plasma portion (e.g., serum or plasma portion 212SP) of a sample (e.g., sample 212) in 802. Capturing the multiple images may be from multiple viewpoints (e.g., viewpoints 1, 2, and 3). In addition, the sample container 102 may include one or more labels (e.g., label 218) thereon. The one or more images may be digital, pixelated images captured by using one or more image capture devices (e.g., image capture devices 440A-440C).

[0155] The characterization method 800 may optionally include, at 804, background removal or tube type detection as described herein. As will be apparent, this may allow sub-regions within the virtual grid 317 to be used to select candidate regions 319 that will be region-restricted, as outlined herein. The characterization method 800 further includes, at 806, inputting image data from a plurality of images (e.g., a merged and normalized image data set) into a neural network (e.g., a convolutional neural network—test CNN 535).

[0156] The characterization method 800 further includes generating candidate regions (e.g., candidate regions 319) for the serum or plasma portion 212SP in 808 and converging the candidate regions 319 by regression to provide a validated region in 810. A validated region is one that closely matches the size, shape, and position of the serum or plasma portion 212SP. Convergence is determined as discussed above (e.g., based on geometric measurements), where the input candidate regions 319 from the candidate region generator 511 should substantially correspond to the output of the regressor 515 within a defined error corridor.

[0157] Finally, the method 800 includes characterizing the validated zone and one or more HILNs using a convolutional neural network (e.g., "test CNN 535") at 812. Specifically, the characterization method 800 can output from the convolutional neural network (e.g., test CNN 535) a classification of the validated zone corresponding to the serum or plasma portion 212SP as one or more of hemolysis, icterus, lipemia, and normal (i.e., H, I, L, H and I, H and L, I and L, H, I, and L or N).

[0158] The multiple images may include multiple images captured from each viewpoint at different exposure times and / or at different spectral illuminations (e.g., R, G, B, white light, IR, and / or near IR). For example, for each viewpoint, there may be 4-8 different exposures or more taken at different exposure times under different spectral illumination conditions. Other numbers of exposures and spectral illuminations may be used.

[0159] Therefore, based on the foregoing, it should be apparent that an improved characterization method 800 is provided that easily characterizes the serum or plasma portion 212SP by using a candidate region (e.g., candidate region 319). As should be apparent, the above characterization method 800 can be implemented by using a quality inspection module (e.g., quality inspection module 130), the quality inspection module comprising a plurality of image capture devices (e.g., image capture devices) 440A-440C and a computer (e.g., computer 143), the image capture devices being arranged around an imaging position (e.g., imaging position 432) and configured to capture a plurality of images of a sample container 102 including one or more labels 218 and containing the serum or plasma portion 212SP of the sample 212 from a plurality of viewpoints (e.g., a plurality of viewpoints 1-3), the computer being coupled to the plurality of image capture devices and being adapted to process the image data of the plurality of images. The computer (e.g., computer 143) can be configured and operable to process the plurality of images from a plurality of viewpoints (e.g., viewpoints 1-3) to provide a HILN determination.

[0160] Various selected components, features or embodiments can be described separately in this article.It should be noted that such components, features or embodiments can replace other components, features or embodiments described separately or even be used in combination with other components, features or embodiments described herein, as practical.Although the embodiments of the present disclosure are prone to various modifications and alternative forms, specific devices, systems and methods have been shown as examples in the accompanying drawings and are described in detail in this article.However, it should be understood that it is not intended to limit the present disclosure to the disclosed specific devices, systems and methods, but on the contrary, the present invention will cover all modifications, equivalents and alternatives falling within the scope of the claims.

Claims

1. A method for training a neural network, comprising: capturing a training image of a specimen container containing a training specimen at an imaging location; automatically generating candidate regions of the serum or plasma fraction for input to a neural network, wherein the candidate regions are generated by randomly selecting sub-regions from a virtual grid superimposed on the training image, and the sub-regions are rectangular sub-regions within the virtual grid; and The candidate region that provides the best match to the serum or plasma portion of the training sample is selected, where The best match for the serum or plasma portion of the training sample is based on the intensity gradient within each candidate region in the candidate region, or the best match for the serum or plasma portion of the training sample is based on the intensity gradient outside each candidate region in the candidate region, or the best match for the serum or plasma portion of the training sample is based on the sum of the intensity gradients outside each candidate region in the candidate region. 2 . The method of claim 1 , wherein the virtual grid comprises a plurality of grid elements having a size of width W×height H. 3 . The method of claim 1 , wherein selecting candidate regions comprises selecting the best 2,000 or more of the generated candidate regions.

4. The method of claim 1 , wherein the neural network comprises an architecture comprising at least two layers configured to implement convolution and pooling, and at least two additional fully convolutional layers.

5. The method of claim 1, wherein the neural network comprises an architecture comprising a loss layer having a bounding box regressor.

6. The method of claim 1, wherein the neural network comprises an architecture comprising a loss layer having a bounding box regressor and a SoftMax.

7. The method of claim 1, wherein the neural network comprises an architecture comprising: Include at least three layers of convolution and pooling, and at least two fully convolutional layers, and a loss layer with a bounding box regressor and softmax. The method of claim 1 , wherein capturing training images comprises a different exposure for each of a plurality of spectrums.

9. The method of claim 1, wherein capturing training images comprises providing different exposure times for each spectrum of red, green, and blue.

10. The method of claim 1, wherein capturing training images involves capturing images from multiple viewpoints, with a multi-spectral, multi-exposure image for each viewpoint.

11. A method for characterizing a sample using a trained neural network trained by the method according to any one of claims 1 to 7, comprising: capturing an image of a specimen container containing a specimen at an imaging position; generating candidate regions of the serum or plasma fraction for input to a neural network; as well as The candidate regions are converged by regression to provide a match to the serum or plasma portion of the sample to provide a validated region.

12. The method of claim 11, comprising utilizing a trained neural network to characterize a validated zone as containing one or more of hemolysis, icterus, lipemia, or as normal.

13. A quality inspection module, comprising: an image capture device configured to capture a plurality of images of a sample container containing a serum or plasma portion of the sample; as well as a computer coupled to the image capture device, the computer being configured and operable to: inputting image data from the plurality of images into a neural network trained by the method according to any one of claims 1 to 7, Generating candidate regions of the serum or plasma fraction, converging the candidate regions by regression to provide validated regions, and Output from the neural network is a classification of the validated zone as one or more of hemolytic, icteric, lipemic, and normal.

14. The quality inspection module of claim 13, wherein the plurality of images are captured as multi-spectral, multi-exposure images.

15. The quality inspection module of claim 13, wherein the plurality of images are captured from multiple viewpoints, with a multi-spectral, multi-exposure image for each viewpoint.

Images