Improved immunoassay

Abstract

The present invention relates to improved immunoassays. Disclosed is a method for determining the position of microspheres in an image of an immunoassay using microspheres, where the image comprises a rendered volume of a plurality of microspheres (110), where the method is characterized in that the method comprises determining a distribution (P) by deconvolution based on observed luminescence (Od) and a mathematical representation (b) of microspheres (beads) observed by a microscope, the location of the microspheres is determined, wherein the distribution (P) provides a likelihood that the microspheres (110) are present at a given location in the image (140).

Description

Technical Field

[0001] The teachings of this article relate to immunoassays using microspheres, and in particular immunoassays that can improve the localization of microspheres. Background Technology

[0002] Immunoassay is a biochemical assay that uses capture molecules attached to a suitable surface to measure the presence or concentration of a biomarker. Depending on the assay, the capture molecule can be an antibody or an antigen, and the suitable surface is typically a highly binding plastic, a membrane in various forms, or a microsphere (also referred to below as a bead) containing appropriate chemical groups that allow the antibody or antigen to bind. The molecule detected by an immunoassay is referred to herein as the analyte, and in many cases is a protein, such as a cytokine or hormone, but can also be an antigen-specific antibody.

[0003] Detection and quantification of analytes require secondary reagents, typically generating a detectable signal via fluorescence emission from a detection complex (in the form of a sandwich complex) bound to the captured analyte. The signal intensity is proportional to the concentration of the analyte in the sample, and a two-step detection process using a biotinylated detection antibody and a streptavidin fluorescent complex is usually required. Immunoassays are frequently used for medical and research purposes to measure analytes in biological fluids such as serum, plasma, or cell supernatants.

[0004] Microspheres are commonly used as suitable surfaces for chemically attached trapping molecules because they are easy to produce and can be chemically stained into distinguishable subgroups using appropriate concentrations of suitable fluorophores (subset-identity dyes). The dye or dye mixture is permanently trapped within the microspheres and allows for precise multiplexing in immunoassays. In some cases, the microspheres are magnetic, which facilitates all necessary washing steps in bead-based immunoassays. Once the sandwich complex is established, the microspheres need to be washed and typically analyzed on a flow cytometer. In flow cytometry, each bead is passed through a flow cell one by one and excited by multiple lasers. One laser measures the emission from the detection complex, while another laser identifies any fluorescent dyes trapped within the microspheres.

[0005] While flow cytometry is the standard method for microsphere analysis, it also has some significant drawbacks. One is its overall complexity, requiring annual maintenance to keep the laser aligned, but more importantly, the microsphere samples themselves can contaminate the instrument and may carry potentially dangerous pathogens, ultimately causing flow cytometer clogging.

[0006] Therefore, existing technologies have many drawbacks.

[0007] Figure 1A and Figure 1B The schematic diagram illustrates a general workflow for immunoassay using beads. Multiple microspheres (hereinafter referred to as beads 110) are located in one well of a plurality of wells 121 of an analysis plate 120, which may be, for example, a 96-well or 384-well plate. Beads 110 belonging to different subgroup identifiers are coated with different capture molecules (also referred to as binders 111), such as antibodies or antigens depending on the application. Analytes 112 are introduced to beads 110, and after the addition of detection antibodies 113, the analytes 112 bind to the binders 111 in the form of a sandwich complex.

[0008] Figure 1A A side view of orifice 121 is also shown, indicating the surface of test solution 122 (magnified to show a wavy shape, indicating it is a fluid). The figure also shows that most of the beads 110 had settled to the bottom of the orifice before analysis.

[0009] By coating different subgroups of fluorescently stained beads 110 with different capture molecules 111 (combined with different analytes 112) and placing them in the same well 121, several analytes can be tested in a single well 121, thus saving time, analytical plates and test solutions.

[0010] After the analyte 112 is bound to the beads 110 (via binder 111), and after other steps such as washing, a scanning microscope or other imaging device 130 can capture an image 140 using various fluorescence wavelengths (e.g., 488 nm, 532 nm, 638 nm, 642 nm, 640 nm provided by multiple lasers and / or multiple emission filters—as is known in the art), wherein the image contains a plotted volume 110' of the beads 110, such as... Figure 1B As shown. Image 140 is then analyzed to determine the number of beads, their individual intensities, and most importantly, their correct positions in the test solution, such as the location of the bead center. It should be noted that while methods for determining the bead center are discussed in this paper, this is only one approach to determining bead location; other methods are also feasible, and these are considered equivalent methods in this paper. However, in some specific implementations, it is the bead center that is determined. Figure 1B In this configuration, the imaging device 130 is positioned below the aperture 121, but the imaging device 130 can also be positioned at a different location relative to the aperture 121, such as above it. The placement of the imaging device 130 can depend on the type of imaging device 130 used.

[0011] Since the beads 110 have been deposited at the bottom of the holes, they will be randomly scattered (or distributed) in the image.

[0012] These beads 110 (or microspheres) each possess a bead center associated with the subgroup identifier dye in one fluorescence channel (hence also referred to as the bead center location), and a co-located analyte signal in another fluorescence channel. However, due to various factors such as bead aggregation and the formation of random clusters, images 140 are often difficult to analyze, making it (very) difficult to determine the correct center point of each bead 110. This is necessary to avoid erroneous readings when determining the microsphere subgroup identifier and its co-located analyte signal.

[0013] Figure 1B yes Figure 1A The magnified view of a portion of the image shows bead 110 clearly visible. As described herein, bead 110 and its bead center 110 can sometimes be difficult to identify in clusters, and some beads may be of low intensity, making them difficult to identify in complex multiplex analysis, as some beads 110 may aggregate together and obscure or mask the low-intensity bead center.

[0014] In order to achieve accurate quantitative and qualitative analysis, it is important to correctly identify or locate bead 110 (in many cases, the center of the bead) in image 140.

[0015] Existing technologies demonstrate various methods for improving the quality of image 140 through different image processing techniques, thereby making the bead center corresponding to bead 110 easier to identify. However, these existing image processing methods are limited by the inherent problems of the image itself and cannot satisfactorily handle all situations. Summary of the Invention

[0016] As mentioned above, existing technologies have many drawbacks, and therefore the inventors recognized the need to develop a more efficient new technique for microsphere analysis. Consequently, the inventors proposed an image-based method for analyzing stained microspheres using the transparent bottom of commonly used multi-well plates (e.g., 96-well or 384-well plates). This method enables non-contact reading of microspheres, thus cleverly solving many of the aforementioned problems associated with flow cytometry.

[0017] For image-based methods to succeed, the bead locations (e.g., bead centers) of randomly distributed microspheres must be accurately identified to determine their correct microsphere subgroup identifiers and co-localized analyte signals. This can be achieved through a novel form of mathematical analysis of multichannel fluorescence microscopy images, which images specific fluorophores by selecting filters for excitation and emission light.

[0018] Through profound and groundbreaking reasoning, the inventors recognized an improved method for identifying or locating the center of beads in an image, which overcomes some of the shortcomings of existing technologies.

[0019] The improvement discussed herein is achieved through the following method: a method according to the invention for determining the location of microspheres in an image of an immunoassay utilizing microspheres, wherein the image comprises a depiction of a plurality of microspheres (110), wherein the method is characterized in that the method includes determining the location of microspheres based on observed luminescence (O d The deconvolution of the microspheres (beads) and the mathematical representation of the microspheres (beads) observed by the microscope (b) determine the distribution (P), thereby determining the location of the microspheres, where the distribution (P) provides the probability that a microsphere (110) exists at a given location in the image (140).

[0020] It is important to note that the term "microsphere" is mentioned in this description and should be understood as a particle commonly used in immunoassays, also known as a microsphere, particle, microparticle, or bead. Therefore, this term does not refer to the common definition of a sphere in mathematics.

[0021] This paper describes a method for determining the location of microspheres in images of an immunoassay utilizing microspheres, which uses a mathematically model-based deconvolution to generate a probability distribution that indicates the likelihood that the center of the microsphere is located at a specific position during imaging.

[0022] “Blurred dots (blobs)” (using the language of prior art patent application publication US2014120556A1) have indistinct (undefined, vague, indistinct) shapes and are difficult to distinguish from each other when these indistinct dots are side by side or partially overlap.

[0023] Simple intensity-based filtering (as proposed in US2014120556A1) will be unable to distinguish such “fuzzy points”.

[0024] Compared to existing convolution-based technologies, this invention is based on deconvolution.

[0025] As is well known, convolution operations are based on known quantities. For example, in US2014120556A1 or US2012268584A1, convolution primarily provides edge detection. Deconvolution, on the other hand, attempts to return the original image, which is the opposite of convolution.

[0026] In this invention, if the same “fuzzy point” is generated when one, two or more overlapping particles are unknown, it is impossible to perform direct deconvolution on shape-based convolution (convolution of US2014120556A1) because it would be impossible to distinguish one or more particles.

[0027] Therefore, this invention proposes an ingenious solution: calculating the probability of a particle center existing at a given point during imaging using deconvolution. Thus, this invention makes a series of guesses about the presence of particles at a given point and observes the degree to which these guesses match the actual situation. This is clearly different from the intensity-level filtering method in US2014120556A1.

[0028] In some embodiments, the method is characterized in that it further includes determining the position of the microsphere by receiving the observed luminescence (O) d ), determine the initial distribution (P) n Based on the initial distribution (P) n ) provides prediction of luminescence (L n Update the distribution (P) using the update rule (U). n+1 ), to determine the prediction of luminescence (L n ) and observed luminescence (O d The distance (d) between the two distributions is defined, where the update rule aims to reduce the distance (d) for a predetermined number of iterations, or until the distance (d) falls below an acceptance threshold level, and if this is achieved, the final distribution (P) is determined to be the distribution (P). n+1 ).

[0029] In some implementations, the method further includes: if the distance (d) is not lower than an acceptance threshold level, then based on the updated distribution (L... n+1 Provides updated luminescence predictions (P) n+1 ), to determine the updated luminescence prediction (L n+1 ) and observed luminescence (O d The distance (d) between the two is used to update the emission prediction (L) by determining whether the distance (d) is below an acceptance threshold level. d ) and observed luminescence (O d (Compare)

[0030] Using the mathematical model described herein, which includes representations of the physical properties and image distribution of microspheres, it is possible not only to identify beads (or the location of beads) but also to identify individual beads within a cluster, even if the light intensities of the beads differ. The center of each bead can also be located. The teachings of this paper can be used to locate beads and then determine their centers, or to directly locate the center of each bead by simply adjusting the distribution to the center of any given point. For the purposes of this teaching, both methods will be discussed simultaneously without much distinction, as those skilled in the art will be able to differentiate between them and know how to adapt them to each other.

[0031] It is important to note that this does not represent an improvement in the image, but rather an application of image enhancement to improve the results. Instead, it is a mathematical method for determining the location of microspheres based on a model that includes a mapping between the actual representation of the microspheres, the assumed distribution of the bead centers, and the observable image.

[0032] According to one aspect, a computer program product is also provided, comprising program instructions that, when executed by one or more processors in a processing device, are used to perform the method according to the invention.

[0033] According to one aspect, a processing apparatus is also provided, including a memory and a processing unit, wherein the processing apparatus is configured to determine the location of microspheres in an image of an immunoassay utilizing microspheres, wherein the image includes a plurality of spots, by means of observed luminescence (O) d The deconvolution of (b) and the mathematical representation of the microspheres observed by the microscope determine the distribution (P), thereby determining the location of the microspheres, where the distribution (P) represents the probability that a microsphere is present at a given location in the image.

[0034] It is important to note that when referring to an image, it should be understood to include the digital representation of the image, i.e., image data. In some embodiments, image data may be stored in a format provided by the imaging device, such as image sensor data. In some embodiments, image data may be stored in a low-level or minimally refined format (e.g., compression or filtering). In some embodiments, image data may be stored in RAW format. In some embodiments, image data may be stored in any known or unknown image format. Attached Figure Description

[0035] This article will discuss these teachings with reference to the accompanying diagrams, in which:

[0036] Figure 1A A schematic diagram of the immune analysis system is shown.

[0037] Figure 1B This illustrates how an image to be analyzed is presented in an immune analysis system.

[0038] Figure 2 A schematic diagram of the processing system based on the teachings of this article is shown.

[0039] Figure 3A A schematic diagram of a microsphere representation based on the teachings of this paper is shown.

[0040] Figure 3B A schematic diagram of a microsphere representation based on the teachings of this paper is shown.

[0041] Figure 3C A schematic diagram of the distribution according to the teachings of this article is shown.

[0042] Figure 4 A schematic diagram of the algorithm taught in this paper is shown.

[0043] Figure 5 A flowchart illustrating the general method taught in this paper is shown, and

[0044] Figure 6 A schematic diagram of a computer-readable medium carrying computer instructions in accordance with the teachings herein is shown. Detailed Implementation

[0045] Figure 2 A schematic diagram of a processing system 200 is shown, which is configured to determine, for example... Figure 1A The position of the microspheres (also referred to as beads 110) shown in image 140 (showing multiple plotted volumes 110' of beads 110). See reference... Figure 1A It should be noted that there is no distinction between beads and bead-drawn volumes in this article.

[0046] Processing system 200 includes processing device 210. Processing device 210 includes one or more processing units (A) configured to control some or all of the functions of processing system 200. Processing device 210 includes (or is connected to) memory B carrying computer-readable instructions that, when loaded into at least one of the one or more processing units, enable processing device 210 to perform the teachings herein. Those skilled in the art will understand that the one or more processing units may include, but are not limited to, microcontrollers, microprocessors, central processing units (CPUs), graphics processing units (GPUs), complex instruction set computing (CISC) processors, application-specific integrated circuit (ASIC) processors, field-programmable gate arrays (FPGAs), reduced instruction set computing (RISC) processors, very long instruction word (VLIW) processors, and other processors or control circuits. Those skilled in the art will also understand that memory is a computer-readable storage medium, and examples of implementations of computer-readable storage media include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), flash memory, secure digital card (SD) cards, solid-state drives (SSDs), and / or CPU cache memory.

[0047] Processing device 210 receives an image 140 of the aperture 121 in the analysis plate 120 captured by scanning device or other imaging device 130. In some embodiments, scanning device 130 is included in processing system 200. In some embodiments, scanning device 130 is located outside processing system 200, and processing system receives image 140 directly or indirectly from imaging device 130.

[0048] In some embodiments, the imaging device 130 is a digital camera. In some embodiments, the imaging device 130 is a microscope equipped with a digital camera.

[0049] In some embodiments, the imaging device 130 is a confocal microscope. A confocal microscope (most commonly a confocal laser scanning microscope (CLSM) or laser scanning confocal microscope (LSCM)) is an optical imaging technique that obtains photomicrographs with high optical resolution and contrast by using a spatial pinhole to block defocused light during the imaging process. A confocal microscope focuses a beam of light to a narrow depth level in a single pass. CLSMs enable controlled and highly limited depths of focus, thereby shielding against defocused light.

[0050] Such as combination Figure 1B As described, image 140 shows randomly distributed beads 110, which sometimes overlap. The beads may also be scattered or difficult to detect. If the beads are magnetic, they also have an increased tendency to aggregate. This makes it difficult to determine which objects in image 140 correspond to the beads, as in combining... Figure 1B As briefly discussed, the problem can also be defined as determining the location of a bead. In some embodiments, the location of the bead to be determined is the center of the bead. The output 222 of system 200 is a list (or other representation) of the bead locations (xy coordinates) in the image. In some embodiments, output 222 also includes the intensity of the bead, such as peak (pixel) intensity or integral reception of bead emission. In some embodiments, output 222 also includes other properties of the bead, such as features or characteristics derived from other mathematical derivations, such as depth of focus. Output 222 may be provided as an image, a mathematical representation, a modified version of image 140, or a table.

[0051] To address this problem, the inventors proposed that instead of simply applying various image processing techniques, the beads should be represented mathematically (in some embodiments, the physical properties of the beads as observed under a microscope). Then, an algorithm (discussed in detail below) is applied to fit an observation model, which is a mapping from the mathematical representation of the beads to the volume of the beads in image 140. Therefore, the inventors propose a model-based source point localization algorithm (hereinafter referred to as the "algorithm"), where the source point corresponds to the center of the bead. This algorithm is based on three main concepts, namely...

[0052] Actual physical observation model M,

[0053] The inverse problem formulation, and

[0054] Optimization methods.

[0055] The actual physical observation model M comprises two parts. The first part (i.e., the mathematical representation discussed above) is intended to represent a bead, which is the source point of the volume drawn in image 140. Therefore, in some embodiments, this mathematical representation is intended to simulate or represent the form of a bead as observed by a microscope. In some embodiments, the bead may be represented as a sphere.

[0056] It should be noted that although the detailed description in this article focuses on confocal microscopy, these teachings are equally applicable to other types of microscopes.

[0057] The inventors further recognized that the observation of beads depends on the microscope used, and beads observed using a confocal microscope will not be observed as spheres—because a confocal microscope can only focus on a plane (or near a plane). However, the inventors also further recognized that the observation of beads does not depend on the beads themselves, but rather on the fluorophores within the beads, or more precisely, the emission of the fluorophores. Therefore, the observed beads will appear denser in the center. Thus, through pioneering and profound reasoning, the inventors designed a hemispherical mathematical representation to represent beads.

[0058] Figure 3A A schematic diagram shows an example of the mathematical representation (or function) (b of a bead), where b represents a hemisphere (proportional to it). It is important to note that... Figure 3A (and Figure 3B The diagram illustrates a function as a variable (i.e., the x-axis), whose mathematical representation is actually a function of two variables representing an xy-coordinated graph. In some implementations, the hemisphere is a (semi)circle. In some implementations, the hemisphere is a (semi)circle, but the circle is not a perfect circle.

[0059] The inventors further recognized that the observation in this case was conducted by receiving images from a microscope via a processor, and that the microscope added optical (or other) distortions to the observed images, and therefore these distortions should be taken into account when designing the representation of the beads.

[0060] As the inventors recognized, this distortion can be modeled as a convolution between the microscope's point-spread function or an approximation thereof and the mathematical representation of the bead (a hemisphere or other shape). In some embodiments, the point-spread function (an approximation thereof) can be modeled as an Airy disk or a Gaussian kernel; in some embodiments, the mathematical representation b of the hemisphere also includes (Gaussian, Airy, or other) blurring. Figure 3B A schematic diagram shows an example of a hemispherical mathematical representation b with Gaussian blur added.

[0061] The second part of the actual physical observation model (hereinafter also referred to as physical model M) aims to model the observation results as a function of possible (hypothetical) bead centers. In some implementations, physical model M is a convolution of the (bead center) distribution P with a kernel b, where kernel b is a mathematical representation of the physical properties of bead 110 as described above. Through profound and pioneering reasoning, the inventors realized that the mathematical representation of the bead can be used as a kernel for deconvolution of the observation results to obtain a density function representing the bead at any given point, thus providing a solution to the aforementioned inverse problem, which will be discussed in detail below. Figure 3C A schematic diagram of this type of distribution P is shown. This distribution P gives p i This indicates the probability that a bead exists at position i, and... Figure 3C In the example, the positions are arranged in a rectangle. In some implementations, the rectangle may correspond to an image, where each position i corresponds to a pixel, a set of pixels, or a point on a sub-pixel grid. In some implementations, each position i corresponds to a portion of the image that is the size of a bead. In some implementations, each position i corresponds to a portion of an image scan, where that portion may be the size of a bead or smaller. Portions with smaller dimensions are able to handle bead distributions that may be unevenly distributed.

[0062] It should be noted that the teachings in this paper deconvolve not only the point spread function (PSF) of the microscope, but also the entire bead model. Therefore, the deconvolution taught in this paper is fundamentally different from existing deconvolution practices. It should also be noted that it is precisely the (precise) bead model disclosed in this paper that enables the discarding of out-of-focus (floating) beads and other non-bead artifacts.

[0063] In this diagram, the representation is shown. Figure 1A The dashed lines of the rectangle in image 140 provide an indication of the image.

[0064] It is important to note that although this paper presents the distribution P in a vectorized format (one index), where each row is processed sequentially, other formats can also be used. For example, a two-dimensional (matrix) representation can be used, where the first index represents a row and the second index represents a column. It is also important to note that the numbering used can be changed, and any numbering, for example, starting with 0 or 1, can be used.

[0065] As is well known, for those skilled in the art, i.e., those with high mathematical skills, a set of observed data O d It can be represented as a function M (hereinafter referred to as the forward operator or simply the model) applied to the distribution (hereinafter referred to as P), that is, M(P) = O d Therefore, the forward problem simplifies to finding O given P. d That is, to determine M(P).

[0066] As stated above, the inventors have recognized that by observing the data (O d The image is set as a bead, and a series of previously established algorithms for solving inverse problems can be used to give the solution P.

[0067] Therefore, in the observed data (O d Given an image of bead 110, the inverse problem is given O d Find P, i.e., the observed beads 110 of image 140, wherein the inventor proposes to use M as described above, i.e., to deconvolve the observed data to obtain the distribution P in the case of M being a convolution, or P in general being the solution to the inverse problem.

[0068] To solve this inverse problem, the inventors proposed a mathematical model M for iterative algorithms, such as... Figure 4 As shown, the mathematical model M is applied to the hypothesis, yields a prediction, and is compared with the observed data. If the prediction does not match the observed data, the hypothesis is updated, and model M is applied again to the updated hypothesis. This process is repeated until the (updated) prediction matches the observation.

[0069] In some implementations, the initial assumption is that the beads (or analytes) are uniformly distributed in the wells. In this case, when the prediction matches the observation, the value at position i is proportional to the probability that a bead is present at position i. This distribution gives the locations of the beads in image 140, where the bead locations are those with "high" values.

[0070] Below, one of the many solutions will be discussed in more detail. As those skilled in the art will know, there are many ways to solve this inverse problem, and the work of providing such a solution is quite straightforward, considering that the modeling of the observations uses the bead center density function P, combined with a representation M of the mapping between bead positions and microscopic observations.

[0071] In this embodiment, the algorithm employed is based on a (proximal) gradient algorithm. Other implementations may utilize other known algorithms for solving inverse problems, such as sparsity-enhanced regularization, alternating direction multiplier method (ADMM), multiplicative update rule, and forward-backward split algorithm. For example, sparsity-enhanced regularization can be implemented in the form of L1-regularization, or in a group sparsity regularization based on a (non-square) form of L2-regularization on the image. In another embodiment, multiplicative update rule and / or forward-backward split algorithm can facilitate the optimization of variables during such deconvolution processes. Examples of proximal gradient algorithms include, but are not limited to, iterative shrinkage thresholding algorithm (ISTA), multiplicative update rule, and accelerated proximal gradient algorithm (FISTA).

[0072] Those skilled in the art will understand that the (proximal) gradient algorithm is an example of a deconvolution algorithm that can be used to provide a solution. Those skilled in the art will understand how to provide solutions based on other known algorithms, and especially after reading the details provided herein, they will understand how to adapt and modify these algorithms in a conventional manner to suit different algorithms.

[0073] It should be noted that when observing beads by light emitted from the fluorophore (after being excited by the excitation light), the distribution P actually becomes a quantity proportional to the amount of light emitted causally induced in the observation results due to the beads centered at a given point i.

[0074] The update rule used when updating hypotheses depends on the chosen algorithm. Those skilled in the art will understand that many alternatives exist, all of which are well known to them. For example, using an update rule based on (proximal) gradients, the update rule for updating the distribution at each point would be based on the difference between the predicted and observed values ​​(or other comparison metrics). If the predicted luminescence value at a given point is higher than the actual observed value, the luminescence value at that point will be reduced accordingly.

[0075] Figure 5 A flowchart illustrating the general method taught herein shows the reception of observed luminescence 510. In some embodiments, the observed luminescence is received as an image of aperture 140. Image 140 can be received from and processed by a processor of the microscope, thus the microscope also functions as a processing device 210. Image 140 can be received from outside the microscope and processed by a processor of a processing device 210 external to the microscope. Examples of such a processing device 210 may be a processing device 210 connected to the microscope, or a processing device 210 that receives images via a storage medium and / or connection (e.g., an internet connection) for subsequent or remote processing. The images show various beads 110. As mentioned above, the images can be received in any format. In some embodiments, as mentioned above, the images are received in RAW format or in a format with little or no preprocessing. Receiving images in this format allows for more accurate identification of beads, as otherwise formatting may filter out subtle details in the image.

[0076] Image 140 is then optionally processed to provide an illumination map of 515 pixels, which records the illumination of each point (pixel, group of pixels, or other point, as described above). The illumination map will be referred to as O below. d , representing the observed image O d The emission vector (or matrix) L at any point i in the image. Optionally, the image can be used as an illumination map O. d Therefore, the illumination diagram is the observed emission of O. d One possible representation.

[0077] refer to Figure 4 Hypothesis 520 is described. In some implementations, the hypothesis is set that all points i are uniformly distributed (with the same value). In some implementations, this value is determined based on the analyte being studied, wherein analytes that are generally easy to bind or find will have a higher (initial) value than analytes that are generally difficult to bind or find. In some implementations, this value is set to, for example, 0 or 1.

[0078] Subsequently, prediction 530 is generated based on this assumption, wherein, in some implementations, the luminescence at each point i is predicted; L n =M(P n ), where M is the observation model described above, which is the distribution P (a vector or matrix p of the possible bead values ​​at point i) after n iterations. i The emission L (the vector or matrix of emission at point i) after n iterations and the emission at point i. i Mapping between ).

[0079] It should be noted that uppercase letters are used below to represent vectors (or matrices), where P is a vector with value p.

[0080] The update rule U is applied to the prediction to update the 540 distribution P, i.e., P n Updated to P n+1 , where P n+1 =U(P n ,L n O d ), where O d This is the observed luminescence as described above. The update rule U aims to reduce the observed luminescence O. d Compared with the predicted luminescence L n The difference or distance d between them. This can be expressed as d(L) n+1 O d ) <d(L n O d As mentioned above, the update rule depends on the chosen algorithm. The scientific optimization literature provides a large number of update rules U to guarantee monotonicity, i.e., the convergence of the sequence d(L) in n. n O d Theories guaranteeing this are readily available in the scientific literature, although a rigorous proof of monotonicity is technically difficult, the update operator is relatively simple.

[0081] Therefore, a distance d of 550 is also determined. In some implementations, this distance d is based on the observed luminescence O captured in the image. d This distance is determined. In some implementations, this distance is determined as the predicted emission level. i Compared with the observed luminescence o iThe squared (Euclidean) distance d between points i is the sum of all distances at all points i; d = ∑ i (l i -o i ) 2 .

[0082] Then determine or compare whether the distance d is small enough. If the distance is not small enough, generate a new hypothesis based on the updated distribution. n+1 =M(P n+1 ), perform a new update 540 and determine a new distance 550. If the distance d is below the acceptable threshold level, then the distance d is small enough.

[0083] If the distance is small enough (561), then the final distribution (570) is determined and P = P n+1 The (final) distribution P is given, and the positions of 580 beads are located using the (final) distribution P.

[0084] In some implementations, a comparison is made for a predetermined number of repetitions 560, and if the predetermined number of repetitions is not completed, the process continues 561, and if the repetitions 562 have been completed, the final distribution is determined 570 and P = P n+1 Provided.

[0085] As described above, the bead position is represented by points with "high" distribution values. In some implementations, the bead position is determined in subsequent processing, possibly in another processing device; therefore, actually determining 580 is optional, as the final model is the primary result of the algorithm presented in this paper. This is in Figure 5 The image is represented by a dashed box. Similarly, the actual reception of the image and the determination of the illumination map are optional, as they can be performed by another processing unit, since the only required input is the illumination map O. d This is also Figure 5 The dashed box indicates the area.

[0086] In some implementations, it is determined whether the distance d is small enough by comparing it to a threshold distance value. Those skilled in the art will understand that the threshold distance value can be selected in a variety of different ways, including by the operator, to investigate different scenarios.

[0087] In some implementations, the distance is determined to be below an acceptable threshold level based on the number of iterations, where it is assumed that the distance is small enough after a given number of iterations, such as 40, 80, 120 or any range therebetween.

[0088] In some implementations, the determination that the distance is below the acceptance threshold level is based on L. n Changes and when L n Changes (i.e., |L) n+1 -Ln If the distance is below the threshold level, it is considered to be small enough.

[0089] As stated above, those skilled in the art only based on the relationship with Figure 5 The flowchart-related information can be used to formulate or implement the algorithm according to the teachings of this article, but a more detailed description will be given for the convenience of those who are not skilled in the art.

[0090] Therefore, the teaching in this paper utilizes a mathematical (functional) relationship or model M, which maps the (usually) two-dimensional (relative) distribution P representing possible bead centers to the expected fluorescence image L of a given fluorophore, i.e., L = M(P), taking into account a specific P. The algorithm obtains the measured illumination O. d Alternatively, an illumination map, which may be derived from the microscope's image sensor (e.g., inside the microscope or via a camera attached to the microscope), can be manipulated to recover or determine the final distribution P such that L = M(P) is as close as possible to O in some sense. d Through the distance function d(L,O) d To measure.

[0091] In some implementations, the measured distance is the predicted illumination L. n With the measured illumination O d The sum of the squared differences between the values ​​is calculated at all pixel locations. Finding a distribution P that satisfies this property is often difficult to do in a single step, so iterative methods are typically used. Therefore:

[0092] 510. Acquire illumination image 140, which represents the observed data O. d It can come from the camera of a microscope.

[0093] 520. Create an initial hypothesis (usually a uniform model) P0. Let n = 0.

[0094] 530. Current model P n Used to predict luminescence L n =M(P n ).

[0095] 540. Update rule U for P n Updated to P n+1 , where P n+1 =U(P n ,L n O d )

[0096] 550. Also determine the distance d and update it so that d(L) n+1 O d ) <d(L n O d )

[0097] 560. Repeat this operation until d(L) n+1 O d The value is small enough or the predetermined number of iterations has been reached. Repeated operations are affected by repetitions at n = n + 1 and 530.

[0098] 570. If d(L) n+1 O d If the value is sufficiently small or the predetermined number of iterations has been reached, then P = P n+1 As the final model, and to continue...

[0099] 580. Use P to identify possible bead locations.

[0100] As mentioned above, different update rules can be used, and in some implementations, this refers to the P update rule based on (proximal) gradients. In implementations where the mapping M is linear, this means it can be written as L = AP, where A is a matrix of appropriate dimensions, and P and L are vectorized representations of the model and the expected luminescence, respectively. For the squared difference distance metric d, where...

[0101] d(L,O d )=‖LO d || 2 =||AP-O d || 2

[0102] And among them |·| 2 It is the square of the Euclidean norm, and the update rule is in the form of:

[0103] P n+1 =[P n -ηA T (AP n -O d )] +

[0104] Where A T It is the transpose of A, where η is the appropriately chosen step size, [·] + It indicates element-wise positive.

[0105] The alternative update rule follows the use of near-end optimization and operator splitting, and has a slightly more complex form.

[0106] As described above, and refer to Figure 3A Under a microscope, the shape of a bead can be proportional to a hemisphere. Mathematically, this can be represented as follows. Assume a bead of radius r is located at the center of a coordinate system. Mathematically, if the three-dimensional coordinates (x, y, z) satisfy x 2 +y 2 +z 2 ≤r2 If the coordinates are inside the bead, then the coordinates are located inside the bead; otherwise, the coordinates are located outside the bead.

[0107] In some implementations, the mathematical representation of the bead (center) position is: B(x,y,z)

[0108] B = 1, given x 2 +y 2 +z 2 ≤r 2

[0109] B = 0, given x 2 +y 2 +z 2 >r 2

[0110] The mathematical representation B(x,y,z) is an indicator function that indicates the position of the bead in three-dimensional space, from which the intensity distribution b observed under a microscope can be derived. Therefore, the intensity distribution b is the mathematical representation of the microsphere observed under a microscope.

[0111] In some implementations, the intensity distribution b of the observed beads in the sample is proportional to the hemispherical mass H = H(x,y), representing the thickness of the bead at position (x,y). Note that this can differ from the intensity distribution of the beads in an image, for example, due to the disregard for the PSF or fluorescence brightness of optical elements. In the xy-plane, the number of photons g emitted per unit area and time is determined by… Given, where α is a proportionality constant, representing the number of photons emitted per unit bead volume and time. Figure 3A A schematic diagram of b is shown, which is proportional to H, representing the observation of beads without distortion (including impurities).

[0112] As the context suggests and referenced Figure 3B In some implementations, Gaussian blur can be added by assuming a Gaussian approximation of the microscope's point spread function, thereby calculating the intensity distribution b of bead B in the image.

[0113] It should be noted that this derivation b is merely an example, and other representations can or are alternatively used. In some implementations, the intensity distribution b of the bead observation results is based on empirical observation data.

[0114] The mathematical derivation of the beads taking into account the distortion characteristics of microscopes will be discussed below.

[0115] In some implementations, the mathematical representation of the microscope point spread function (PSF), f(x,y,z), is approximated by a Gaussian approximation.

[0116]

[0117] Where σ x =σ y <<σ z , where σ x σ y σ z These are the focal lengths in the x, y, and z directions, respectively, where z is the direction of the microscope (i.e., the viewpoint of the microscope), and x... m and y m It is the position of the microscope focal point in the xy-plane, where It is a well-known normalization constant for a three-dimensional multivariate Gaussian distribution.

[0118] This physical representation assumes that the concentration of fluorophores inside the three-dimensional bead is constant, and that the three-dimensional (perpendicular to the image plane) focal length is very long relative to the resolution of the imaging plane. Based on these assumptions, the luminous intensity L emitted by a single bead located at the origin of the xy-plane, when focused at the xy-plane position, is the luminous intensity (x...) picked up by the microscope. m ,y m )for

[0119] L(x m ,y m )=∫B(x,y,z)f(x,y,z)dxdydz=∫(∫B(x,y,z)f(x,y,z)dz)dxdy

[0120] As the inventors recognized, it can be assumed that σ z >> The internal integral of r (i.e., the focal length is longer than the bead radius) becomes

[0121]

[0122] Where H(x,y) is the mass of the hemisphere, given as follows:

[0123]

[0124] And where q1 = 2(2π) 1 / 2 σ z Another scaling constant is the product of all constant factors. The hemisphere H(x,y) used in this paper—as the inventors have already recognized—represents only the number (half) of fluorophores along the z-dimensional (focusing) line of the bead.

[0125] In embodiments where the amount of fluorophore is non-uniform throughout the sphere, or if fluorophores deep within the sphere are obscured by surrounding spheres, the function B(x,y,z), and therefore H(x,y), can be replaced by a more representative measurement of the contribution of the fluorophore at a certain location (x,y,z) within the bead to the luminosity seen under the microscope.

[0126] In this case, the function f(x,y,z) can be replaced by another approximation of the Microscope Point Spread Function (PSF).

[0127] Based on the assumption made (uniform and unobstructed fluorophores), the fluorescence originates from individual beads at the microscope (focal) position (x). m ,y m ) photometric distribution L(x m ,y m ) becomes

[0128]

[0129] Where * represents the convolution operator.

[0130] In other words, the intensity observed from a sphere centered at the origin is proportional to the convolution of the Gaussian function and the hemisphere. Utilizing linear and translation invariance, the expected luminosity of all beads positioned according to the distribution P(x,y) (probability) can be obtained as follows:

[0131]

[0132] Among them, such as Figure 3B As shown,

[0133]

[0134] In other words, the intensity distribution b is used as the kernel in the convolution.

[0135] In some implementations, it must be discretized (rather than continuous) to make it suitable for use in a processing device.

[0136] In summary, the representation of the expected luminosity L is given by the convolution of P with the (bead) nucleus b. Therefore, in this implementation, M is the convolution, while from the observed image O... d Obtaining the inverse of P becomes deconvolution.

[0137] The nucleus b depends on the representation of the bead at the origin of the xy-plane, and in particular on the radius r of the bead and the assumption that the fluorophore is uniformly distributed throughout the bead.

[0138] Since the convolution operator M is linear, as mentioned earlier, it can be represented by vectors of P and L, and the relation L = M(P) can be written as L = AP.

[0139] In some implementations, AP or A T P is implemented as a linear (convolution) operator (instead of explicit matrix-vector multiplication). This is to speed up processing because the computational requirements of the convolution operator are lower.

[0140] It should be noted that the teachings herein do not preclude the possibility of applying (various) image processing techniques to images prior to the determination of the teachings discussed herein.

[0141] Figure 6 A computer program product 610 comprising program instructions 620 is illustrated, which, when executed by one or more processors (A) in processing device 210, performs the methods and teachings described herein. The computer program product 610 is implemented as an algorithm embedded in software stored in a non-transitory computer-readable storage medium containing program instructions executable by one or more processors in a computer system to perform the methods and teachings described herein. The non-transitory computer-readable storage device may include, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing, such as the memory B of processing device 210. Examples of implementations of the computer-readable storage medium include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), flash memory, secure digital card (SD) card, solid-state drive (SSD), computer-readable storage media, and / or CPU cache memory.

Claims

1. A method for determining the location of microspheres in an image of an immunoassay utilizing microspheres, wherein the image comprises a plot of a plurality of microspheres (110), wherein the method is characterized in that the method comprises: The position of the microspheres is determined by the following: Based on observed luminescence (O d The deconvolution of the microspheres (110) and the mathematical representation of the microspheres observed by the microscope (b) determine the distribution (P), wherein the distribution (P) provides the probability that a microsphere (110) exists at a given location in the image (140).

2. The method according to claim 1, wherein the method is further characterized in that the observed emission (O) d The deconvolution of ) is based on model (M), where M is the convolution of the distribution (P) with kernel (b), where kernel (b) is the intensity distribution of the beads as observed by the microscope.

3. The method according to claim 2, wherein the method is further characterized in that the mathematical model (M) includes a representation of physical properties, wherein the physical properties include the luminescence of the microspheres (110) as observed by the imaging device (130).

4. The method according to claim 2 or 3, wherein the core (b) comprises a mathematical representation of the optical properties of the microscope.

5. The method according to any one of the preceding claims, wherein the method is characterized in that the method further comprises determining the position of the microspheres by: Receive (510) the observed luminescence (O) d ), Determine the initial distribution (P0) of (520). Based on the initial distribution (P0), a prediction (L) of luminescence is provided (530). n ), Update the (540) distribution (P) using update rule (U). n+1 ), Determine (550) the prediction of the emission (L) n ) and the observed luminescence (O) d The distance (d) between (560) and (560) is defined by an update rule designed to reduce the distance (d) for a predetermined number of iterations or until the distance (d) falls below an acceptable threshold level, and if this is achieved, then The final distribution (P) is determined to be (570) the distribution (P). n+1 ).

6. The method of claim 5, wherein the method further comprises, if the distance (d) is not lower than an acceptance threshold level, then Based on the updated distribution (P) n+1 ), providing (530) updated luminescence predictions (L n+1 ), Determine the updated emission prediction (L) n+1 ) and the observed luminescence (O) d The distance (d) between them. By determining whether the distance (d) is below the acceptance threshold level, the updated luminescence prediction (L) is compared (560). n ) and the observed luminescence (O) d ).

7. The method according to any one of claims 5 to 6, wherein the distribution P indicates causally generated luminescence (l) at a given point (i). i The center of the bead is used to indicate the presence of a microsphere at a given location.

8. The method according to any one of claims 5 to 7, wherein the method further comprises based on the distribution value (p) at point (i). i The final distribution (P) of the positions of the microspheres at point (i) is given, and the positions of the microspheres (580) are determined.

9. The method according to any one of claims 5 to 8, wherein the method further comprises determining the initial distribution (P0) as a uniform distribution.

10. The method according to any one of claims 5 to 9, wherein the update rule is based on a gradient-based algorithm.

11. The method according to any one of claims 5 to 9, wherein the update rule is a multiplicative update rule.

12. The method according to any one of the preceding claims, wherein the method further comprises receiving the observed emission (O) by receiving the image. d ).

13. The method according to any one of the preceding claims, wherein at least some of the microspheres are magnetic microspheres.

14. The method according to any one of the preceding claims, wherein at least some of the microspheres are stained with fluorophores so as to emit light when excited by excitation light.

15. The method of claim 14, wherein the fluorophores are uniformly distributed in the microspheres.

16. The method according to any one of the preceding claims, wherein the representation of the microsphere is given by a function proportional to the hemisphere.

17. The method of claim 16, wherein the representation of the microsphere is given by a hemisphere with added Gaussian blur.

18. The method according to any one of claims 5 to 17, wherein the method further comprises determining the distance d as a predicted emission (l) summed over each point. i ) and observed luminescence o i The squared difference between them, d = ∑ i (l i -o i ) 2 .

19. A computer program product (610) comprising program instructions (620) which, when executed by one or more processors (A) in a processing device (210), are configured to perform the method according to any one of the preceding claims.

20. A processing apparatus (210) comprising a memory (R) and a processing unit (A), wherein the processing apparatus (210) is configured to determine the position of microspheres in an image of an immunoassay utilizing microspheres by means of: The position of the microspheres is determined by the following: Based on observed luminescence (O d The deconvolution of the microspheres and the mathematical representation of the microspheres observed by the microscope (b) determine the distribution (P), wherein the distribution (P) indicates the presence of microspheres (110) at a given location in the image (140).

Images