Method, components and software for detecting a systematic error in a protein detection system

Abstract

The present invention relates to quality control (QC) of data produced from a protein detection system and in particular to methods, components and software for detecting systematic errors in a protein detection system. The methods comprise receiving (S502) an amount of the protein in each sample; determining (S504) a pattern of the array for the protein by thresholding the determined amount for each sample, such that the determined amount of the protein for each sample is represented by one of a plurality of predetermined discrete values in the pattern; comparing (S506) the pattern with a plurality of predetermined patterns, each predetermined pattern corresponding to a systematic error in the protein detection system, and determining a similarity value for each of the predetermined patterns; upon the similarity value exceeding (S508) a pattern specific similarity threshold, indicating (S510) the systematic error in a data structure of candidate systematic errors.

Description

[0001] METHOD, COMPONENTS AND SOFTWARE FOR DETECTING A

[0002] SYSTEMATIC ERROR IN A PROTEIN DETECTION SYSTEM

[0003] Technical Field

[0004] The present invention relates to quality control (QC) of data produced from a protein detection system and in particular to methods, components and software for detecting systematic errors in a protein detection system.

[0005] Background

[0006] Modern proteomics methods require the ability to detect a large number of different proteins (or protein complexes) in a small sample volume. To achieve this, multiplex analysis may be performed. Common methods by which multiplex detection of proteins in a sample may be achieved include dual -recognition immunoassays. Dualrecognition immunoassays build on a concept developed by Ulf Landegren and coworkers and described in Fredriksson et al., Nature Biotechnology, vol. 20, 2002, pp. 473-477 and W001 / 61037.

[0007] Dual -recognition immunoassay methods include Proximity Extension Assay (PEA), commercially available from Olink Proteomics AB (Uppsala, Sweden). Various aspects of PEA is further described in WO 03 / 044231, WO 2004 / 094456, WO 2005 / 123963, WO 2006 / 137932, WO 2013 / 113699, WO 2021 / 191442, WO 2021 / 191448, WO 2021 / 191449, WO 2021 / 191450, and WO 2022 / 112300; Assarsson et al., PLoS 1, 2014, 9, 4, e95192; Lundberg et al., Molecular & Cellular Proteomics 10: 10.1074 / mcp.Ml 10.004978, 1-10, 2011; and Wik et al., 2021, Mol Cell Proteomics 20, 100168 (https: / / doi.Org / 10.1016 / j.mcpro.2021.100168), all incorporated herein by reference in their entirety.

[0008] The Proximity Ligation Assay (PLA) methodology has primarily been used in combination with Rolling Circle Amplification (RCA) for detection of protein-protein interactions and post-translational modifications and is commercially available from Merck under the trademark Duolink®. A PLA-based method for multiplex detection of proteins is described in WO 2021 / 113290.

[0009] PEA and PLA are dual -recognition assays, which rely on the principle of “proximity probing”. In these methods an analyte is detected by the binding of multiple (i.e., two or more, generally two or three) probes, which when brought into proximity by binding to the analyte (hence "proximity probes") allow a signal to be generated. Typically, at least one of the proximity probes comprises a nucleic acid domain (or moiety) linked to the analyte-binding domain (or moiety) of the probe, and generation of the signal involves an interaction between the nucleic acid moieties and / or a further functional moiety which is carried by the other probe(s). Thus, signal generation is dependent on an interaction between the probes (more particularly between the nucleic acid or other functional moieties / domains carried by them) and hence only occurs when the necessary probes have bound to the analyte, thereby lending improved specificity to the detection system.

[0010] In PEA, nucleic acid moieties linked to the analyte-binding domains of a probe pair hybridise to one another when the probes are in close proximity (i.e., when bound to the same target molecule), and are then extended using a nucleic acid polymerase. The extension product forms a reporter nucleic acid, detection of which demonstrates the presence of a particular analyte (the analyte bound by the relevant probe pair) in a sample of interest. In PLA, nucleic acid moieties linked to the analyte-binding domains of a probe pair come into proximity when the probes of the probe pair bind their target, and may be ligated together, or alternatively they may together template the ligation of separately added oligonucleotides which are able to hybridise to the nucleic acid domains when they are in proximity. The ligation product is then amplified, acting as a reporter nucleic acid. Multiplex analyte detection using PEA or PLA may be achieved by including one or more unique barcode sequences in the nucleic acid moiety of each probe. Oligonucleotides comprising a barcode sequence unique to a specific sample may further be added to the respective sample and incorporated into all reporter molecules generated from that sample. A reporter nucleic acid molecule corresponding to a particular analyte, and optionally a particular sample, may then be identified by the barcode sequences it contains. The methods of the present invention find particular utility in multiplex PEA and PLA methods.

[0011] Panels of proximity assays (also referred to as array of reaction containers herein), as described above, are commercially available from Olink Proteomics AB (Uppsala, Sweden) under the trademark Olink® Target, Olink® Focus, Olink® Explore, and Olink® Flex. These are panels of up to 92 assays (Olink® Target, Olink® Focus and Olink® Flex) or up to -3,000 assays split over eight different panels (Olink® Explore). A panel may further be divided into a number, typically four, of “abundance blocks” and the samples may be diluted based on their predicted abundance prior to being incubated with the assay probes of the respective abundance block. Each panel generally includes assays for proteins that have known functions within certain biological or physiological areas, pathways or organs in the body, such as inflammation, organ-specific proteins, cardiovascular, neurology etc. It is also possible for a user to select a specific combination of protein assays that are of particular interest to create a tailor-made panel.

[0012] Proximity based assays produce a number of reporter molecules with a certain barcode set-up, which number correlates to the amount of protein with the corresponding barcode. This number can be quantified by either quantitative Polymerase Chain Reaction (qPCR) or sequencing, preferably Next Generation Sequencing (NGS). qPCR produces a Ct value that corresponds to the amount of protein in the sample and NGS produces an actual number, called “counts”, of reporter molecules and the counts correlate with the amount of protein in the sample.

[0013] Based on Ct values and counts, it is possible to calculate both absolute and relative amounts of protein in the sample. Olink uses an arbitrary relative quantification unit called NPX that may be calculated with software adapted for use with the above panels. A measure of relative protein quantity, such as NPX, can be calculated from counts as described in Wik et al., cited above.

[0014] It is further known to perform Quality Control (QC) on data produced from proximity based assays, as e.g., described in Wik et al., cited above. To facilitate QC, internal and external controls are included in the protocol. The internal controls used for QC are an incubation control and an amplification control. The incubation control comprises PEA probes measuring a fixed concentration of nonhuman green fluorescent protein (GFP), added to each sample. The amplification control consists of a synthetic double-stranded DNA template and is used in QC to monitor the PCR steps in the protocol. External controls used for QC comprise a negative control (buffer only) run in triplicate, and a biological sample control, run in duplicate.

[0015] The QC assessment is performed at two levels; run QC and sample QC. At the run QC level each of the abundance blocks for each panel and sample plate should fulfil the following criteria: (i) mean absolute deviation (MAD) in internal controls may not exceed a certain threshold of relative protein quantity (such as 0.3 NPX) and (ii) deviation on the sample QC level is allowed for a maximum of one out of six samples. Further, in each panel, the median of at least 90% of the assays in plate and negative control samples must be in the accepted range from predefined values set during validation. Apart from run QC, the performance of each sample is assessed individually by the internal controls that should be within a predefined range of relative protein quantities (such as ±0.3 NPX) from the median level across the abundance block. Additionally, the mean assay count for a sample may not be below 500 counts. Abundance blocks and samples that do not fulfil their respective QC criteria will receive a QC warning. Another QC metric is precision evaluated as the coefficient of variation (CV). CV is a measure of technical variation for individual assays both within a plate (intra-CV) and across multiple plates (inter-CV) and may be calculated as described in Wik et al. It is possible to set upper limits on intra-CV and inter-CV among all assays as additional QC criteria.

[0016] However, the current QC evaluations may not always be able to automatically detect quality issues that arise from malfunctioning equipment or, more frequently, from human errors. These types of quality issues are often categorized as systematic effects or systematic errors. Examples of such systematic errors include instrument calibration issues and reagent composition issues.

[0017] There is thus a need for improvements within this context.

[0018] Summary

[0019] In view of the above, solving or at least reducing one or several of the drawbacks discussed above would be beneficial, as set forth in the attached independent patent claims.

[0020] According to a first aspect of the present invention, there is provided a computer implemented method for detecting a systematic error in a protein detection system configured to determining an amount of each protein of a plurality of proteins in a plurality of samples, wherein each sample is provided to the protein detection system in an array of reaction containers at a random position of the array, the method comprising: for at least some proteins of the plurality of proteins: receiving, from the protein detection system, an amount of the protein in each sample; determining a pattern of the array for the protein by thresholding the determined amount for each sample, such that the determined amount of the protein for each sample is represented by one of a plurality of predetermined discrete values in the pattern; comparing the pattern with a plurality of predetermined patterns, wherein each predetermined pattern corresponding to a systematic error of a plurality of predetermined systematic errors of the protein detection system, and determining a similarity value for each of the predetermined patterns;

[0021] - upon the similarity value between the pattern and a predetermined pattern from the plurality of predetermined patterns meeting or exceeding a pattern specific similarity threshold, indicating the systematic error corresponding to the predetermined pattern in a data structure of candidate systematic errors.

[0022] The method further comprise, upon the data structure of candidate systematic errors indicates a first threshold number of instances of a first systematic error of the plurality of systematic errors, indicating, to a user of the protein detection system, that the array exhibits the first systematic error.

[0023] The term “an array of reaction containers” should, in the context of present specification, be understood as a set of individual vessels or wells each of which can hold a separate reaction mixture. One example of such an array is a microtiter plate (microplate), which has a grid of wells. These plates come in various formats, with the number of wells typically being 6, 12, 24, 48, 96, 384, or even 1536. Each well in the plate acts as a separate reaction container, and the grid format allows many reactions to be conducted simultaneously, for example to determine an amount of each protein of a plurality of proteins (for example 1500-3000 proteins) in a plurality of samples (reaction mixtures). Using an array of reaction containers may be particularly useful for high-throughput screening, large-scale studies, or when sample volume is limited.

[0024] The amounts of protein may for a sample may be received from the protein detection system as measure of relative protein quantity for the samples in the array, such as NPX. Relative protein quantity can be calculated from counts as described in Wik et al., cited above.

[0025] Since the samples are provided at random positions in the array, an expected pattern of a measurement of an amount of protein in the samples provided in the array is a random pattern. However, technical errors in the workflow, such as instrument calibration issues and reagent composition issues, may result in that such a pattern deviate from a random pattern and instead is similar to a predefined pattern associated with a technical error in the workflow.

[0026] The inventors have realized that that by determining a pattern of the array for a currently analysed protein by thresholding the determined amount for each sample, such that the determined amount of the protein for each sample is represented by one of a plurality of predetermined discrete values, a pattern of the protein measurements (i.e., the amount of the currently analysed protein for each of the samples in the array) may be determined. The pattern thus represents the underlying biological concentration of the protein and the randomized position of samples on the array. This pattern may be automatically compared with a plurality of predetermined patterns, wherein each predetermined pattern corresponding to a systematic error of a plurality of predetermined systematic errors of the protein detection system. The method thus uses a library of patterns, e.g., represented as integer matrices, specific to systematic effects that may occur in the analysis workflow. Each library pattern thus represents one systematic effect arising from a problem / error / malfunctioning in the workflow.

[0027] In case the pattern is similar enough (according to a suitable similarity metric) to a certain predetermined pattern, the systematic error corresponding to the similar predetermined pattern is marked as a candidate systematic error for the entire analysis of the array. This process is repeated for a plurality of proteins until either a threshold number of proteins have resulted in a pattern that is similar enough to a certain systematic error, which means that the array exhibits this systematic error, or otherwise until all patterns for all proteins measurable in the samples have been analysed (which means that the array does not exhibits any systematic error). In case it is determined that the array exhibits a systematic error, this is indicated to a user of the protein detection system, that may then take any suitable actions to e.g., verify the systematic error, remedy the systematic error, exclude the array, perform a rerun of the lab workflow for the array, etc.

[0028] The term “data structure of candidate systematic errors” should, in the context of present specification, be understood as any suitable data structure for keeping track of which systematic errors that are candidate systematic errors for the array (i.e., possible systematic errors, determined based on the pattern similarities as discussed herein), and for how many proteins that each systematic error has been assessed as being a candidate systematic error. The data structure may for example be implemented as a table with each row corresponding to a systematic error of the plurality of systematic errors, and having a number representing a count of how many proteins whose patterns has been determined similar enough to the pattern corresponding to the systematic error.

[0029] Advantageously, the present method provides an efficient and automatic way of analysing an array of samples for systematic errors. For example, by implementing the first threshold, computational resources and time may be saved since not all proteins of the plurality of proteins may need to be analysed before indicating, to a user of the protein detection system, that the array exhibits the first systematic error. Moreover, by determining a pattern of the array for the protein by thresholding the determined amount for each sample, such that the determined amount of the protein for each sample is represented by one of a plurality of predetermined discrete values in the pattern, a low complexity way of determining a pattern for measurement values of amounts of proteins may be achieved. A further advantage may be that using historical data, it may be possible to determine a library of predetermined patterns, where each predetermined pattern corresponds to a certain systematic error.

[0030] In some examples, the first threshold number is between 5-15% of the number of proteins. Advantageously, the first threshold may be set to reliably indicate that the array exhibits a certain systematic error. Moreover, such an interval may result in an advantageous balance between certainty (reliability) of the assessment that the array exhibits a certain systematic error and saving computational resource and / or analysis time.

[0031] In some embodiments, the step of thresholding comprises thresholding the determined amount of the protein for each sample, such that each determined amount of the protein is represented by one of: -1, 0, and +1 in the pattern, wherein each predetermined pattern consists of values selected from -1 and +1. Advantageously, determining a similarity value between the pattern and a predetermined pattern may be simplified. Moreover, using relatively few possible values in the pattern may still provide the possibility to determine predetermined patterns for systematic errors arising from the workflow of a protein detection system.

[0032] In examples, the step of thresholding comprises calculating a median of the determined amount of the protein for each sample, wherein an amount exceeding the median with more than a threshold amount is set to +1, wherein an amount falling below the median with more than the threshold amount is set to -1, and an amount being within the threshold amount from the median is set to 0.

[0033] By representing a value (amount of protein) that is close to the median (according to the threshold amount) by a zero, noise in the pattern for the protein may be reduced. The threshold amount may be calculated based on the median, such that the threshold is larger for a larger value indicating the median (generally larger amounts of the measured protein in the array), compared to for a lower value indicating the median (generally lower amounts of the measured protein in the array). In other embodiments, a fixed threshold, e.g., 0.45 NPX, 0.5 NPX, 0.63 NPX, etc., may be employed. The amounts are thus “normalized” before being represented by 1, 0 or -1 in the pattern. In some embodiments, average may be used instead of median. In yet other embodiments, standard deviation may be used as a cutoff point for 1, 0 and -1.

[0034] In some embodiments, the method further comprises, for a determined pattern, counting the number of samples represented by -1 or 1, wherein the step of comparing the pattern with a plurality of predetermined patterns is only performed for patterns where the number of samples represented by -1 or 1 meets or exceeds a second threshold number. Advantageously, computer resources may be saved. Moreover, reliability of the pattern comparison and the result thereof may be increased since a large enough number of protein measurements (i.e., amount of a certain protein in the samples of the array) must result in a 1 or -1 in order for the pattern comparison step to be performed.

[0035] In some examples, the second threshold number is between 40-66% of the number of samples. For example, the second threshold may correspond to 40 / 96 samples being represented by a -1 or 1 for a certain protein for the pattern comparison for that protein to be performed. If less samples than the threshold number are represented by -1 or 1, further analysis (e.g., pattern matching) of the protein may be skipped and thus not resulting in any candidate systematic errors.

[0036] According to some embodiments, the array of reaction containers consists of columns and rows, wherein at least a subset of predetermined patterns among the plurality of patterns have a same number of columns and rows as the number of columns and rows of reaction containers in the array, wherein the step of calculating the similarity value comprises, for each predetermined pattern from the subset of predetermined patterns, an elementwise multiplication between the pattern and the predetermined pattern to determine a similarity matrix, such that each element in the similarity matrix having the value 1, 0 or -1, and wherein the similarity value indicates a difference between a count of elements in the similarity matrix having the value 1 and a count of elements in the similarity matrix having the value -1. An elementwise multiplication between a matrix having 1, 0 or -1 as values (i.e., the pattern) and a matrix having -1 and 1 as values (i.e., the predetermined pattern) will produce a result matrix (similarity matrix) with a value 1 for indexes where the value of the pattern and the predetermined pattern have the same value (i.e., 1x1, or -lx-1), a value -1 for indexes where the value of the pattern and the predetermined pattern have the opposite value (i.e., lx-1, or -1x1), and zero otherwise (for indexes where the pattern has 0 as a value). The similarity value may thus advantageously indicate a difference between a count of elements in the similarity matrix having the value 1 and a count of elements in the similarity matrix having the value -1 which thus compares the number of corresponding values with the number of opposite values in the two patterns. The zeros in the similarity matrix may advantageously be ignored when calculating the similarity value to reduce noise as discussed above.

[0037] According to some embodiments, the similarity value is calculated by dividing an absolute value of the difference with a count of elements in the similarity matrix having the value 1 or -1. Consequently, also a pattern that is (similar enough to) an inverted version of a predetermined pattern will give a high similarity score, which advantageously reduces the number of predetermined patterns needed to represent possible systematic errors. Generally, a predetermined pattern and an inverted version of the predetermined pattern may both represent a certain systematic effect. In examples, a predetermined pattern from the subset of predetermined patterns is one of: a vertically striped pattern with alternating columns having values 1 and -1, a horizontally striped pattern with alternating rows having values 1 and -1, a diagonally striped pattern with alternating diagonals having values 1 and -1, or, a regional pattern, wherein the pattern consist of two regions, wherein one region of the pattern has 1 as values and the other region of the pattern has -1 as values.

[0038] In some embodiments, the array of reaction containers consists of columns and rows, and wherein a first predetermined pattern among the plurality of patterns indicates a row in the pattern having at least a third threshold number of 1 :s as values, or at least the third threshold of -1 :s as values, wherein the step of calculating the similarity value between the pattern and the first predetermined pattern comprises: counting a number of rows in the pattern having at least the third threshold number of 1 :s as values, or at least the third threshold of -l :s as values.. Advantageously, also systematic errors resulting in a row-based pattern which not necessarily extends over the full array may be captured by the method, resulting in an increased flexibility.

[0039] In some examples, the array of reaction containers consists of columns and rows, and wherein a second predetermined pattern among the plurality of patterns indicates a column in the pattern having at least a fourth threshold number of l :s as values, or at least the fourth threshold of -1 :s as values, wherein the step of calculating the similarity value between the pattern and the second predetermined pattern comprises: counting a number of columns in the pattern with values having at least the fourth threshold number of 1 :s as values, or at least the fourth threshold of -1 :s as values. Advantageously, also systematic errors resulting in a column-based pattern which not necessarily extends over the full array may be captured by the method, resulting in an increased flexibility.

[0040] According to a second aspect of the invention, the above object is achieved by a protein detection system configured to determining an amount of each protein of a plurality of proteins in a plurality of samples, wherein each sample is provided to the protein detection system in an array of reaction containers at a random position of the array, the protein detection system comprising an error detection component comprising: one or more processors; and one or more non-transitory computer-readable media storing computer executable instructions that, when executed by the one or more processors, cause the system to perform actions comprising: for at least some proteins of the plurality of proteins: receiving, from the protein detection system, an amount of the protein in each sample; determining a pattern of the array for the protein by thresholding the determined amount for each sample, such that the determined amount of the protein for each sample is represented by one of a plurality of predetermined discrete values in the pattern; comparing the pattern with a plurality of predetermined patterns, wherein each predetermined pattern corresponding to a systematic error of a plurality of predetermined systematic errors of the protein detection system, and determining a similarity value for each of the predetermined patterns;

[0041] - upon the similarity value between the pattern and a predetermined pattern from the plurality of predetermined patterns meeting or exceeding a pattern specific similarity threshold, indicating the systematic error corresponding to the predetermined pattern in a data structure of candidate systematic errors;

[0042] The instructions may further cause the system to perform the action of upon the data structure of candidate systematic errors indicates a first threshold number of instances of a first systematic error of the plurality of systematic errors, indicating, to a user of the protein detection system, that the array exhibits the first systematic error.

[0043] In some examples, the protein detection system is configured to determining an amount of each protein of a plurality of proteins in a plurality of samples using an immunoassay based technology. It should be noted that the QC techniques described herein may be used for other suitable technologies within proteomics, such as other affinity-based methods.

[0044] According to a third aspect of the invention, the above object is achieved by a non-transitory computer-readable storage medium having stored thereon instructions for implementing the method according to the first aspect when executed on a device having processing capabilities.

[0045] The second and third aspects may generally have the same features and advantages as the first aspect. It is further noted that the disclosure relates to all possible combinations of features unless explicitly stated otherwise. Further features and advantages of the invention will become apparent from the following description of preferred embodiments of the invention, given by way of example only, which is made with reference to the accompanying drawings.

[0046] Brief Description of the Drawings

[0047] The above, as well as additional objects, features, and advantages of the present invention, will be better understood through the following illustrative and non-limiting detailed description of embodiments of the present disclosure, with reference to the appended drawings, where the same reference numerals will be used for similar elements, wherein:

[0048] Figure 1 shows a protein detection system according to embodiments,

[0049] Figure 2 shows a plot of protein amounts for an array of reaction containers according to embodiments,

[0050] Figure 3 shows four examples of predetermined patterns, each corresponding to a systematic error, according to embodiments,

[0051] Figure 4 shows a comparison between a pattern of the array for a protein and a predetermined pattern from figure 3, resulting in a similarity matrix that can be used to calculate a similarity score, according to embodiments,

[0052] Figure 5 shows a flow chart of a method for detecting a systematic error in a protein detection system, according to embodiments.

[0053] Detailed Description

[0054] Quality control (QC) is critical in all areas of laboratory research, including immunoassay based technologies. Multiplexed immunoassays is a group of high- throughput techniques used for analysing multiple proteins in small sample volumes, often used in proteomics studies. It involves multiple steps, as exemplified above. Each of these steps has the potential to introduce variability or errors, which could affect the final results. Therefore, careful QC is necessary to ensure that the assay is working as expected. For example, given its high-throughput nature, small systematic errors in multiplexed immunoassays can affect a large number of measurements, leading to potentially significant impacts on the final results. The present disclosure describes the present invention primarily with reference to its application in Proximity Extension Assay (PEA), but the invention is equally applicable to other types of multiplexed protein detection assays performed in an arrayed set of reaction containers, such as multiplexed Proximity Ligation Assay (PLA). Furthermore, the present disclosure uses the applicant’s unit NPX as a measure of relative protein quantity. While this is the present standard unit for relative protein quantification, other measures of relative protein quantity may be used as appreciated by the skilled person.

[0055] Figure 1 shows by way of example a protein detection system 102, for example using immunoassay based technologies to determine an amount of each protein of a plurality of proteins in a plurality of samples, the plurality of samples provided to the protein detection system 102 using an array of reaction containers. In the example of figure 1, the protein detection system 102 comprises a protein counting component 104, which may be configured to measure a concentration (count) of each protein of the plurality of proteins for all samples provided in the array. The counts for a protein thus correlate with the amount of that protein in the samples.

[0056] The protein detection system 102 may further comprise a NPX calculation unit 106 configured to determine NPX values (arbitrary relative quantification unit) for the proteins for each sample provided in the array. In other embodiments, the NPX calculation unit is not part of the protein detection system, and QC is performed directly on the counts from the protein counting component 104.

[0057] The protein detection system 102 further comprises an error detection component 108. The purpose of the error detection component 108 is to perform QC on data produced from the protein counting component 104 and / or the NPX calculation component 106. In particular, it may be a purpose of the error detection component 108 to identify systematic errors that may have influenced the protein counts of the samples. This may be done by separately analysing the amounts of each protein, i.e., protein by protein from the plurality of proteins, as outputted by the multiplex detection of proteins in a sample.

[0058] In case the error detection component 108 identifies that the array may exhibit a certain systematic error, this may be indicated to a user / operator of the protein detection system 102 as a QC warning using an error indication component 110. The error indication component 108 may for example log the systematic error in a QC log file, show the QC warning on display of the protein detection system, indicate the QC warning using audio or light (using speakers, light emitters, etc.), or by any other suitable means.

[0059] The relative protein quantity (e.g., NPX values) or protein count for each protein (assay) can be presented for all samples in an array in a plot, herein referred to as a plate plot. Figure 2 shows by way of example a plate plot representing a measured concentration of a protein over a 96-well plate, meaning that the array of reaction containers consists of 12 columns and 8 rows. Of note is that while every plate displays a pattern, the pattern is random and represents the underlying biological concentration of the protein and the randomized position of samples 202i...n(n = 96) on the plate. This type of pattern will vary for every array and protein. In this example, the darkness of each square indicates the NPX (darker means a higher NPX value), and the position of the square indicates the sample.

[0060] Systematic errors (technical errors) in the workflow may affect the protein counts and result in systematically erroneous NPX values in the Plate Plot. This disclosure provides an automatic method for detecting if such systematic effects have occurred. The inventors have realized that pattern that occurs for plate plots influenced by some technical errors may be derivable from the technical error (i.e., a certain pattern may indicate a certain systematic error) and deviate from a random pattern as seen in Figure 2.

[0061] A method for identifying arrays that exhibits one or more systematic error and indicate this to a user of the protein detection system 102 will now be described in conjunction with figures 3-4 and the flow chart shown in figure 5.

[0062] Figure 5 shows computer implemented method 500 for detecting a systematic error in a protein detection system (e.g., the protein detection system 102 in figure 2). Parts of the method 500 may be performed protein by protein until it can be determined whether the array do or do not exhibit any of a set of predetermined systematic errors. In other words, if enough assays / proteins are affected, the array may get a systematic effects warning corresponding to that effect.

[0063] Consequently, for a protein of the plurality of proteins detectable by the protein detection system, the method 500 comprises receiving S502, from the protein detection system (e.g., from the protein counting component 104 or from the NPX calculation unit 106 in figure 1), an amount of the protein in each sample. The amount may be received in the form of a NPX value, calculated as set out in Wik et al as described above. The NPX values may be visualized in a plate plot as shown in figure 2.

[0064] To improve and simplify the detection of a systematic error, the received amounts are thresholded. The thresholded amounts may be used to determine S504 a pattern if the array for the protein, such that determined amount of the protein for each sample is represented by one of a plurality of predetermined discrete values in the pattern. In one example, the predetermined discrete values are -1, 0 and +1, such that each determined amount of the protein is represented by one of: -1, 0, and +1 in the pattern, but other scales and granularity of the predetermined discrete values may be possible.

[0065] In some embodiments, the thresholding comprises calculating a median of the determined amount of the protein for each sample. In this example, the amounts determined for a protein are thus median centred. The median centred NPX, used specifically for systematic effects calculations, is below denoted mcNPXa s(median centered NPX). plate _mediana s= medlan^ExtN PXa s,- mcNPXa s= ExtNPXa s— plate _mediana s; a e {assay}, s e {SAMPLE}

[0066] The NPX values for a certain sample and protein (assay) is denoted ExtNPXa s. It should be noted that within a plate, ExtNPXa shas the same properties as NPX and is in effect the same as NPX. a denotes all assays (proteins), and s denotes all samples on the plate.

[0067] If the amount is exceeding the median with at least a threshold amount, the corresponding thresholded value is set to +1. If the amount is falling below the median with more than the threshold amount, the corresponding thresholded value is set to -1. In all other cases, when the amount being within the threshold amount from the median, the corresponding thresholded value is set to 0. The above may be accomplished according to the following, in which SYS_EFFECT_SIZE denotes the threshold amount and stdNPXa sdenotes the thresholded value: if abs(mcNPXa>s) < SYS_EFFECT_SIZE, stdNPXaiS= 0 else if (mcNPXa s) > 0, stdNPXa s= 1 else if (mcNPXa s) < 0, stdNPXa s= — 1

[0068] The threshold amount SYS_EFFECT_SIZE may for example be set to 0.5 NPX, 0.3 NPX, etc. Any other suitable threshold may be used depending on the requirements of the implementation of the techniques described herein.

[0069] Using the thresholded values, stdNPXa s, the pattern of the array for the amounts of a certain protein may thus be determined S504. Figure 4 shows an example of such a pattern 402 where black squares represents the thresholded value 1, white squares represent the thresholded value -1, and grey squares represents the thresholded value 0.

[0070] In some embodiments (not shown in figure 5), the method comprises counting the number of samples represented by -1 or 1 (samples where a systematic effect / error may be suspected), and if the number (denoted by # samples below) does not meet or exceed a second threshold number (such as 38, 40, 45, 52 for an array of 96 reaction containers all holding a sample, denoted by NSAMPLESbelow), it may be determined that the protein as such does not seems to exhibit a systematic effect in the array and the method may proceed by analysing the next protein.

[0071] # samples > N_S AMPLE

[0072] Figure 3 shows a plurality of predetermined patterns 302, 304, 306, 308. Each pattern 302, 304, 306, 308 corresponds to a systematic error of a plurality of predetermined systematic errors of the protein detection system. It should be noted that the patterns in figure 3 is shown by way of example, and that more patterns, a combination of the patterns in figure 3, or less patterns may be implemented. All patterns 302, 304, 306, 308 in figure 3 corresponds to arrays of reaction containers with 12 columns and 8 rows. For protein detection systems using arrays with other sizes, the size of the patterns needs to be changed accordingly. In the patterns 302, 304, 306, 308, black represents a value of 1, and white represents a value of -1. Figure 3 shows a horizontally striped pattern 302 with alternating rows having values 1 and -1. In some embodiments, a horizontally striped pattern may comprise alternating sets of rows having values 1, and -1, such as 2 rows with value 1, and then 2 rows with value -1, etc.

[0073] Figure 3 shows a vertically striped pattern 304 with alternating columns having values 1 and -1. In some embodiments, a vertically striped pattern 304 may comprise alternating sets of columns having values 1, and -1, such as 2 columns with value 1, and then 2 columns with value -1, etc.

[0074] Figure 3 further shows a diagonally striped pattern 306 with alternating diagonals having values 1 and -1. In some embodiments, a diagonally striped pattern 303 may comprise alternating sets of diagonals having values 1, and -1, such as 2 diagonals with value 1, and then 2 diagonals with value -1, etc.

[0075] Finally, figure 3 shows a regional pattern 308, wherein the pattern consists of two regions, wherein one region of the pattern has 1 as values and the other region of the pattern has -1 as values. A regional pattern may be divided into the two regions in a vertical direction (like shown in figure 3), in a horizontal direction, and in a diagonal direction.

[0076] Each of the patterns 302, 304, 306, 308 thus corresponds to a systematic error of a plurality of predetermined systematic errors of the protein detection system. It should be noted that it is possible for a single effect to be represented by several patterns, and that a single array can exhibit more than one systematic error. For example, a systematic effect resulting in a diagonally striped pattern may result in any of four different patterns due to the shifts in the diagonal. A protein (assay) may be considered to display a diagonal effect if any of the four patterns is determined to be similar enough to the pattern determined for the protein.

[0077] Figure 4 shows by way of example how a similarity between a predetermined pattern 302, 304, 306, 308 and a pattern 402 of a protein may be determined. This is done by comparing S506 the pattern 402 of the protein with the predetermined patterns 302, 304, 306, 308 and determining a similarity value for each of the predetermined patterns. In the example of figure 4, the array of reaction containers analysed by the protein detection system consists of 12 columns and 8 rows. The predetermined patterns 302, 304, 306, 308 each has a same number of columns and rows as the number of columns and rows of reaction containers in the array.

[0078] Comparison S506 may be accomplished by an elementwise multiplication between the pattern and the predetermined pattern to determine a similarity matrix 404. In the below, the similarity matrix is denoted by mNPXa sand the predetermined pattern is denoted by mask. mNPXa s= stdNPXa s* mask

[0079] An elementwise multiplication between the pattern 402 (in the form of a matrix having 1, 0 or -1 as values) and a predetermined pattern 302 (in the form of a matrix having -1 and 1 as values) will produce a result matrix (similarity matrix 404) with a value 1 for indexes where the value of the pattern 402 and the predetermined pattern 302 have the same value (i.e., 1x1, or -lx-1), a value -1 for indexes where the value of the pattern 402 and the predetermined pattern 302 have the opposite value (i.e., lx-1, or -1x1), and zero otherwise (for indexes where the pattern 402 has 0 as a value). In figure 4, the similarity matrix 404 is visualized using black to represent values being 1, white to represent values being -1, and grey to represent values being 0.

[0080] The similarity value may then be determined based on a difference between a count of elements in the similarity matrix 404 having the value 1 and a count of elements in the similarity matrix 404 having the value -1.

[0081] In some embodiments, the similarity value is calculated by dividing an absolute value of the difference with a count of elements in the similarity matrix having the value 1 or -1. Advantageously, this means that also predetermined patterns that are inverted in respect to the pattern of the protein will result in high similarity values, which advantageously reduces the number of predetermined patterns.

[0082] The similarity value may be computed as follows, where effects_ratio denotes the similarity value. abs (sum(mNPXa s) effects_ratioa s= - - '■ — ; sum ( abs(mNPXas) ]

[0083] The similarity value is then compared to a pattern specific similarity threshold. If the similarity value does not meet or exceed the similarity threshold, it is determined that the amounts of the protein as determined by the protein detection system for the samples in the array does not indicate the systematic error that the predetermined pattern 302 of the comparison represents.

[0084] In some embodiments, not shown in figure 3-4, a predetermined pattern may include the value 0 as well, to indicate areas of the array which for any reason is not interesting or will obscure the effect that is looked for. A zero in the predetermined pattern will result in a zero in the similarity matrix no matter what value the array (and thus the pattern representing the array) has for the corresponding parts. As such, the above equation for calculating the similarity value effects_ratioa sstill applies.

[0085] It should be noted that in some embodiments, a predetermined pattern used to identify a systematic error do not necessarily represent a full plate array. For example, a predetermined pattern may indicate a row in the pattern having only 1 or only -1 as values, or alternatively at least a threshold number (such as 7, 8, 10 etc. for an array with 12 columns) of l:s or -l:s. In this example, the step of calculating the similarity value between the pattern and such a row-based predetermined pattern comprises counting a number of rows in the pattern with values having only 1 or only -1 as values, or rows in the pattern having at least the threshold number of l:s or -l:s. The same strategy may be used for column-based pattern, such that a predetermined pattern may indicate a column in the pattern having only 1 or only -1 as values, or alternatively at least a threshold number (such as 6, 7, 8, etc. for an array with 8 rows) of 1 :s or -1 :s. In this example, the step of calculating the similarity value between the pattern and the second predetermined pattern comprises counting a number of columns in the pattern with values having only 1 or only -1 as values, or columns in the pattern having at least the threshold number of 1 :s or -1 :s.

[0086] The below table shows by way of example pattern specific similarity thresholds. It should be noted that these values are just examples, and that other thresholds may apply, depending on the use case and the QC requirements of the protein detection system.

[0087] Table 1

[0088] If it is determined that the similarity value for a predetermined pattern meet or exceeds S508 a pattern specific similarity threshold, the systematic error corresponding to the predetermined pattern may be indicated S510 in a data structure of candidate systematic errors.

[0089] The data structure of candidate systematic errors may be compared to a log of all systematic errors that has been indicated for proteins as described above. The data structure may be represented by a table or a list, or any other way of keeping track of how many proteins for which the amounts measured by the protein detection system indicate a certain systematic error.

[0090] This data structure is used to determine if it should be indicated to a user / operator of the protein detection system that the array exhibits a systematic error. The decision is taken based on a threshold, such as for example 7%, 10%, 12%, 15% of the proteins have been identified as indicating a certain systematic error (e.g., the similarity value between the pattern of the protein and the predetermined patterns corresponding to the certain systematic error have met or exceeded the pattern specific similarity threshold for at least the threshold number of proteins).

[0091] The data structure may be checked regularly, for example after each protein have been analysed, or after every second, third etc. protein such that the analysis process of the remaining proteins may be aborted as soon as it is determined that the array exhibits a systematic error. In other embodiments, all proteins are analysed, and the data structure is examined for systematic errors meeting or exceeding the threshold at the end.

[0092] The method 500 shown in figure 5 may be implemented by the protein detection system shown in figure 1, or in a separate device connected to a protein detection system. In some examples, the method 500 is implemented in the cloud, e.g., as a SaaS (Software as a Service) solution.

[0093] Generally, the device / devices implementing the method 500 and other functionality described herein may comprise circuitry which is configured to implement the method, and some / all of the components 104, 106, 108, 110 and, more specifically, their functionality. The features described herein can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor. Suitable processors for the execution of a program of instructions include, by way of example, both general and special purpose microprocessors, and the sole processor or one of multiple processors or cores, of any kind of computer. The processors can be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).

[0094] The above embodiments are to be understood as illustrative examples of the invention. Further embodiments of the invention are envisaged. For example, a priority order of the predetermined systematic errors may be used. An example of such priority order is that full plate (mask) patterns (as shown in figure 3) may take precedence over the row / column effects. If the amounts of a certain protein are determined to show a pattern corresponding to a mask, no analysis of row / column effects for the amounts of that protein may be done.

[0095] It is to be understood that any feature described in relation to any one embodiment may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may also be employed without departing from the scope of the invention, which is defined in the accompanying claims.

Claims

CLAIMS1. A computer implemented method (500) for detecting a systematic error in a protein detection system (102) configured to determining an amount of each protein of a plurality of proteins in a plurality of samples, wherein each sample is provided to the protein detection system in an array of reaction containers at a random position of the array, the method comprising: for at least some proteins of the plurality of proteins: receiving (S502), from the protein detection system, an amount of the protein in each sample; determining (S504) a pattern (402) of the array for the protein by thresholding the determined amount for each sample, such that the determined amount of the protein for each sample is represented by one of a plurality of predetermined discrete values in the pattern; comparing (S506) the pattern with a plurality of predetermined patterns (302, 304, 306, 308), wherein each predetermined pattern corresponding to a systematic error of a plurality of predetermined systematic errors of the protein detection system, and determining a similarity value for each of the predetermined patterns; upon the similarity value between the pattern and a predetermined pattern from the plurality of predetermined patterns meeting or exceeding (S508) a pattern specific similarity threshold, indicating (S510) the systematic error corresponding to the predetermined pattern in a data structure of candidate systematic errors; upon the data structure of candidate systematic errors indicates (S512) at least a first threshold number of instances of a first systematic error of the plurality of systematic errors, indicating (S514), to a user of the protein detection system, that the array exhibits the first systematic error.

2. The computer implemented method of claim 1, wherein the first threshold number is between 5-15% of the number of proteins.

3. The computer implemented method of any one of claims 1-2, wherein the step of thresholding comprises thresholding the determined amount of the protein for each sample, such that each determined amount of the protein is represented by one of: -1, 0, and +1 in the pattern, wherein each predetermined pattern consists of values selected from -1 and +1.

4. The computer implemented method of claim 3, wherein the step of thresholding comprises calculating a median of the determined amount of the protein for each sample, wherein an amount exceeding the median with more than a threshold amount is set to +1, wherein an amount falling below the median with more than the threshold amount is set to -1, and an amount being within the threshold amount from the median is set to 0.

5. The computer implemented method of any one of claims 3-4, further comprising, for a determined pattern, counting the number of samples represented by - 1 or 1; wherein the step of comparing the pattern with a plurality of predetermined patterns is only performed for patterns where the number of samples represented by -1 or 1 meets or exceeds a second threshold number.

6. The computer implemented method of claim 5, wherein the second threshold number is between 40-66% of the number of samples.

7. The computer implemented method of any one of claims 3-6, wherein the array of reaction containers consists of columns and rows, wherein at least a subset of predetermined patterns among the plurality of patterns have a same number of columns and rows as the number of columns and rows of reaction containers in the array, wherein the step of calculating the similarity value comprises, for each predetermined pattern from the subset of predetermined patterns, an elementwise multiplication between the pattern and the predetermined pattern to determine a similarity matrix (404), such that each element in the similarity matrix having the value 1, 0 or -1, and wherein the similarity value indicates a difference between a count of elements in the similaritymatrix having the value 1 and a count of elements in the similarity matrix having the value -1.

8. The computer implemented method of claim 7, wherein the similarity value is calculated by dividing an absolute value of the difference with a count of elements in the similarity matrix having the value 1 or -1.

9. The method of any one of claims 7-8, wherein a predetermined pattern from the subset of predetermined patterns is one of: a vertically striped pattern (304) with alternating columns having values 1 and -1, a horizontally striped pattern (302) with alternating rows having values 1 and - 1, a diagonally striped pattern (306) with alternating diagonals having values 1 and -1, or, a regional pattern (308), wherein the pattern consists of two regions, wherein one region of the pattern has 1 as values and the other region of the pattern has -1 as values.

10. The computer implemented method of any one of claims 3-9, wherein the array of reaction containers consists of columns and rows, and wherein a first predetermined pattern among the plurality of patterns indicates a row in the pattern having at least a third threshold number of 1 :s as values, or at least the third threshold of -1 :s as values, wherein the step of calculating the similarity value between the pattern and the first predetermined pattern comprises: counting a number of rows in the pattern having at least the third threshold number of 1 :s as values, or at least the third threshold of -1 :s as values.

11. The computer implemented method of any one of claims 3-10, wherein the array of reaction containers consists of columns and rows, and wherein a second predetermined pattern among the plurality of patterns indicates a column in the pattern having at least a fourth threshold number of 1 :s as values, or at least the fourth thresholdof -1 :s as values, wherein the step of calculating the similarity value between the pattern and the second predetermined pattern comprises: counting a number of columns in the pattern with values having at least the fourth threshold number of 1 :s as values, or at least the fourth threshold of -1 :s as values.

12. A protein detection system configured to determining an amount of each protein of a plurality of proteins in a plurality of samples, wherein each sample is provided to the protein detection system in an array of reaction containers at a random position of the array, the protein detection system comprising an error detection component comprising: one or more processors; and one or more non-transitory computer-readable media storing computer executable instructions that, when executed by the one or more processors, cause the system to perform actions comprising: for at least some proteins of the plurality of proteins: receiving, from the protein detection system, an amount of the protein in each sample; determining a pattern of the array for the protein by thresholding the determined amount for each sample, such that the determined amount of the protein for each sample is represented by one of a plurality of predetermined discrete values in the pattern; comparing the pattern with a plurality of predetermined patterns, wherein each predetermined pattern corresponding to a systematic error of a plurality of predetermined systematic errors of the protein detection system, and determining a similarity value for each of the predetermined patterns; upon the similarity value between the pattern and a predetermined pattern from the plurality of predetermined patterns meeting or exceeding a pattern specific similarity threshold, indicating the systematic error corresponding to the predetermined pattern in a data structure of candidate systematic errors; upon the data structure of candidate systematic errors indicates at least a first threshold number of instances of a first systematic error of the plurality of systematicerrors, indicating, to a user of the protein detection system, that the array exhibits the first systematic error.

13. The protein detection system of claim 12, configured to determining an amount of each protein of a plurality of proteins in a plurality of samples using an immunoassay based technology.

14. A non-transitory computer-readable storage medium having stored thereon instructions for implementing the method according to any one of claims 1-12 when executed on a device having processing capabilities

Images