Multi-factor digital fluorescence immunoassay method based on machine vision

Abstract

The invention relates to a multi-factor digital fluorescence immunodetection system based on machine vision. The multi-factor digital fluorescence immunodetection system comprises an upper computer such as a Windows PC / image processing and hardware control system, a camera, a microscope system and an XYZ three-dimensional electric translation stage. According to the system, detection of more than one characteristic factor is implemented in a single pore plate for detection, multi-joint detection of immune factors is realized, detection consumables are reduced, and the detection efficiency is improved. Information acquisition of a single factor can be realized only by using single brightness information in an imaging result in single-factor detection, and paired antibodies of different antigens and fluorescent microspheres of different colors are selected for sample treatment in order to realize target distinguishing of multi-factor detection; and meanwhile, classifying and counting image targets with different colors by utilizing a machine vision method. The detection system is used for photographing the substrate containing the fluorescent microspheres with multiple colors, the machine vision scheme is used for obtaining the number of the fluorescent microspheres, and the concentrations of different objects to be detected in the sample to be detected are obtained according to standard sample data.

Description

Technical Field

[0001] The present invention belongs to the field of automation control, machine vision / image recognition, and immune detection. More specifically, it is a method of using image recognition technology combined with automation technology to achieve joint detection of multiple characteristic factors on a single sample. Background Art

[0002] Compared with biochemical amplification technology, the detection of biological samples using physical light signal amplification technology has the advantages of fewer steps and faster speed. Digital immunofluorescence detection is a biological detection scheme that combines immunology and fluorescence detection. It uses the specific binding of antigens and antibodies and microscopic imaging of markers to achieve a digital detection method that is different from analog intensity signals. This scheme has the characteristics of high sensitivity, high specificity, high throughput and intuitive digital expression, and is particularly suitable for low-concentration object detection. Most of the current detection systems are designed for a single target antibody or protein, lacking efficient and accurate multi-factor and multi-target detection capabilities, low efficiency, long time consumption, and waste of detection substrates. Summary of the invention

[0003] The purpose of the present invention is to provide a multi-factor digital fluorescence immunoassay system based on machine vision.

[0004] A multi-factor digital fluorescence immunoassay system based on machine vision, the detection system includes a Windows PC, a three-dimensional translation stage, a microscope and a camera, the lens of the camera is connected to the eyepiece of the microscope; the three-dimensional translation stage control software, the camera control software and the image processing software are installed on the Windows PC, under the control of the three-dimensional translation stage control software and the camera control software, the three-dimensional translation stage is controlled to perform scanning motion, stop at a specified position and then focus and take pictures to obtain a microscopic image of the sample.

[0005] A multi-factor digital fluorescent immunoassay method based on machine vision, the specific steps are as follows:

[0006] S1: Sample preparation;

[0007] S2: imaging the sample area to be tested;

[0008] S3: Process the image to identify and count particles of different colors, thereby obtaining multi-factor detection results in different areas;

[0009] S4: Shut down the system.

[0010] Furthermore, step S1 is specifically as follows:

[0011] Paired antibodies of multiple antigens to be tested are modified on a substrate, and the antibodies are modified on fluorescent microspheres of different colors, thereby obtaining a sandwich structure sample of substrate-multiple markers to be tested-fluorescent microspheres.

[0012] Furthermore, step S2 is specifically as follows:

[0013] S2.1: Power on and perform self-test to confirm whether the camera and 3D translation stage are functioning normally.

[0014] S2.2: Place the sample prepared in S1 on the sample holder, use the three-dimensional translation stage to move the sample test area on the sample holder to a suitable position, then focus and take a picture, and use the camera to obtain a clear fluorescence image.

[0015] S2.3: Repeat S2.1 and S2.2 to image all the sample test areas.

[0016] Furthermore, step S3 is specifically as follows:

[0017] S31: Setting initial thresholds R_min and G_min for the RGB values ​​of the image.

[0018] S32: For the original image Pic_ori, select a green particle and measure its diameter Dg.

[0019] S33: (a) Construct an array G_binary with the same size as the original image, traverse the image pixels, when the R channel value is less than R_min and the G channel is greater than G_min, the corresponding position of G_binary is set to 1, otherwise it is set to 0; (b) Perform Hough transform on the variable G_binary to find the circle, and the diameter of the circle is set in the range of [Dg*0.5, Dg*1.5], that is, 0.5 to 1.5 times the size of the initially marked particles, and the specific size distribution of the particles needs to be referred to; (c) Set the particle spacing threshold to eliminate repeated recognition due to poor image capture and recognition errors, and finally obtain the number of green particles; (d) Adjust the R_min and G_min values ​​so that the green particle recognition and counting results are consistent with the results of human sensory judgment.

[0020] S34: For the original image Pic_ori, select a yellow particle and measure its diameter Dy.

[0021] S35: (a) Construct an array Y_binary with the same size as the original image, traverse the image pixels, when the R channel value is greater than R_min and the G channel is greater than G_min, the corresponding position of G_binary is set to 1, otherwise it is set to 0; (b) Perform Hough transform on the variable Y_binary to find the circle, and the diameter of the circle is set in the range of [Dy*0.5, Dy*1.5], that is, 0.5 to 1.5 times the size of the initially marked particle, and the specific size distribution of the particles needs to be referred to; (c) Set the particle spacing threshold to eliminate repeated recognition due to poor image capture and recognition errors, and finally obtain the number of green particles; (d) Adjust the R_min and G_min values ​​so that the yellow particle recognition and counting results are consistent with the results of human sensory judgment.

[0022] S36: Based on the concentration and particle quantity information in other calibration information, inversely derive the concentration data of different objects to be tested corresponding to the particle quantity in the sample to be tested.

[0023] A multi-factor digital fluorescent immunoassay method based on machine vision, the specific steps are as follows:

[0024] S1: Sample preparation;

[0025] S2: imaging the sample area to be tested;

[0026] S3: Process the image to identify and count particles of different colors, thereby obtaining multi-factor detection results in different areas;

[0027] S4: Shut down the system.

[0028] Furthermore, step S1 is specifically as follows:

[0029] Paired antibodies of multiple antigens to be tested are modified on a substrate, and the antibodies are modified on fluorescent microspheres of different colors, thereby obtaining a sandwich structure sample of substrate-multiple markers to be tested-fluorescent microspheres.

[0030] Furthermore, step S2 is specifically as follows:

[0031] S2.1: Power on and perform self-test to confirm whether the camera and 3D stage functions are normal.

[0032] S2.2: Place the sample prepared in S1 on the sample holder, use the three-dimensional translation stage to move the sample test area on the sample holder to a suitable position, then focus and take a picture, and use the camera to obtain a clear fluorescence image.

[0033] S2.3: Repeat S2.1 and S2.2 to image all the sample test areas.

[0034] Furthermore, step S3 is specifically as follows:

[0035] S31: Segment the image. Assuming that the training image size is set to 256*256 for appropriate GPU hardware (such as RTX4090 24G), the image is split.

[0036] S32: performing data annotation on the split images, that is, annotating bounding boxes of imaging photos of different fluorescent particles to obtain annotation and classification results.

[0037] S33: Training in neural network frameworks such as pytoch and tensorflow.

[0038] S34: Perform inference on the new sample image, that is, (a) split the image, (b) use the trained model in 3 to perform inference, and (c) merge the results, exclude duplicate data on the segmentation boundary, and obtain the counting result.

[0039] S35: Based on the concentration and particle quantity information in other calibration information, inversely derive the concentration data of different objects to be tested corresponding to the particle quantity in the sample to be tested.

[0040] Compared with the prior art, the present invention has the following technical effects:

[0041] By modifying the substrate with paired antibodies of various objects to be detected and fluorescent microspheres of different colors, a sandwich structure of substrate-multiple markers to be detected-fluorescent microspheres is prepared, which is placed in a digital fluorescent immunoassay system for imaging and analysis, and finally the concentration information of the object to be detected is obtained. The detection system uses software and hardware systems including microscopic imaging, cameras, translation stages, image recognition and camera / translation stage control to achieve machine vision detection of multi-characteristic factors of different antigen-antibody pairs / fluorescent ball colors on the substrate. This method achieves multi-factor monitoring in one test through multiple groups of antigen-antibody pair modifications and multi-color fluorescent labeled microspheres. It has the advantages of multi-factor detection capability, high detection efficiency, saving consumables, and simple operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of a multi-factor digital fluorescent immunoassay method based on machine vision provided in an embodiment of the present invention.

[0043] Figure 2 It is the test sample 1.

[0044] Figure 3 It is the test sample 2.

[0045] Figure 4 These are microscopic photos taken by the system of two coated antibody-analyte-antibody-fluorescent microsphere combinations. DETAILED DESCRIPTION

[0046] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more concise and clear, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] See also Figure 1-4 Now, a multi-factor digital fluorescence immunoassay system based on machine vision provided by the present invention is described.

[0048] Embodiment 1:

[0049] Figure 1 This is a schematic diagram of a multi-factor digital fluorescent immunoassay system based on machine vision provided by an embodiment of the present invention, which is used to achieve efficient automated optical scanning detection. The detection system includes a Windows PC, a three-dimensional translation stage, a microscope and a camera, and the lens of the camera is connected to the eyepiece of the microscope; the three-dimensional translation stage control software, camera control software and image processing software are installed on the Windows PC. Under the control of the three-dimensional translation stage control software and the camera control software, the three-dimensional translation stage performs scanning motion, stops at a specified position, and then focuses and takes pictures to obtain a microscopic image of the sample.

[0050] Embodiment 2:

[0051] A multi-factor digital fluorescent immunoassay method based on machine vision, the specific steps are as follows:

[0052] S1: Sample preparation

[0053] Paired antibodies of multiple antigens to be tested are modified on a substrate, and the antibodies are modified on fluorescent microspheres of different colors, thereby obtaining a sandwich structure sample of substrate-multiple markers to be tested-fluorescent microspheres.

[0054] S2: Imaging the sample area to be tested

[0055] S2.1: Power on and perform self-test to confirm whether the camera and 3D translation stage are functioning normally.

[0056] S2.2: Place the sample prepared in S1 on the sample holder, use the three-dimensional translation stage to move the sample test area on the sample holder to a suitable position, then focus and take a picture, and use the camera to obtain a clear fluorescence image.

[0057] S2.3: Repeat S2.1 and S2.2 to image all the sample test areas.

[0058] S3: Process the image to identify and count particles of different colors, thereby obtaining multi-factor detection results in different areas.

[0059] S31: Setting initial thresholds R_min and G_min for the RGB values ​​of the image.

[0060] S32: For the original image Pic_ori ( Figure 4 (a)), select a green particle and measure its diameter Dg.

[0061] S33: (a) Construct an array G_binary with the same size as the original image, traverse the image pixels, when the R channel value is less than R_min and the G channel is greater than G_min, the corresponding position of G_binary is set to 1, otherwise it is set to 0; (b) Perform Hough transform on the variable G_binary to find the circle, and the diameter of the circle is set in the range of [Dg*0.5, Dg*1.5], that is, 0.5 to 1.5 times the size of the initially marked particles, and the specific size distribution of the particles needs to be referred to; (c) Set the particle spacing threshold to eliminate repeated recognition due to poor image capture and recognition errors, and finally obtain the number of green particles; (d) Adjust the R_min and G_min values ​​so that the green particle recognition and counting results are consistent with the results of human sensory judgment.

[0062] S34: For the original image Pic_ori ( Figure 4 (a) Select a yellow particle and measure its diameter Dy.

[0063] S35: (a) Construct an array Y_binary with the same size as the original image, traverse the image pixels, when the R channel value is greater than R_min and the G channel is greater than G_min, the corresponding position of G_binary is set to 1, otherwise it is set to 0; (b) Perform Hough transform on the variable Y_binary to find the circle, and the diameter of the circle is set in the range of [Dy*0.5, Dy*1.5], that is, 0.5 to 1.5 times the size of the initially marked particle, and the specific size distribution of the particles needs to be referred to; (c) Set the particle spacing threshold to eliminate repeated recognition due to poor image capture and recognition errors, and finally obtain the number of green particles; (d) Adjust the R_min and G_min values ​​so that the yellow particle recognition and counting results are consistent with the results of human sensory judgment.

[0064] S36: Based on the concentration and particle quantity information in other calibration information, inversely derive the concentration data of different objects to be tested corresponding to the particle quantity in the sample to be tested.

[0065] S4: Shut down the system.

[0066] Particle counting can be achieved through machine vision solutions. Figure 3 Medium-grain photos and the RGB value information corresponding to particles of different colors can be used to extract the number of particles of different colors.

[0067] like Figure 4 It is the target image obtained by the dual-factor detection system. Image processing is performed on the microscopic photo to obtain the number of particles of different colors. Combined with the standard sample information, the concentration information of different analytes is obtained. The labels "RGB", "R", "G" and "B" respectively indicate that the color channels of the photo are RGB (full color) and single channels R (red), G (green) and B (blue). From the analysis of the three RGB components of the color image, it can be known that the RGB values ​​of particles that appear green or greenish are approximately (0,234,0), (221,235,98), (0,235,0), (0,164,0), and the RGB values ​​of orange-yellow particles are (235,235,0), (235,227,0), (89,95,0). RGB (221,235,98) corresponds to the orange part in the middle of the green particles that is close to the overexposed state. RGB (89,95,0) corresponds to a dark yellow-green color. At the same time, according to Figure 4 (d) The entire photo is basically black, and the blue B component in the image is basically 0. Therefore, by dividing the particles according to the RGB values, the identification and counting of particles of different colors can be achieved.

[0068] Example 3

[0069] A multi-factor digital fluorescent immunoassay method based on machine vision, the specific steps are as follows:

[0070] S1: Sample preparation

[0071] Paired antibodies of multiple antigens to be tested are modified on a substrate, and the antibodies are modified on fluorescent microspheres of different colors, thereby obtaining a sandwich structure sample of substrate-multiple markers to be tested-fluorescent microspheres.

[0072] S2: Imaging the sample area to be tested

[0073] S2.1: Power on and perform self-test to confirm whether the camera and 3D translation stage are functioning normally.

[0074] S2.2: Place the sample prepared in S1 on the sample holder, use the three-dimensional translation stage to move the sample test area on the sample holder to a suitable position, then focus and take a picture, and use the camera to obtain a clear fluorescence image.

[0075] S2.3: Repeat S2.1 and S2.2 to image all the sample test areas.

[0076] S3: Process the image to identify and count particles of different colors, thereby obtaining multi-factor detection results in different areas.

[0077] S31: Segment the image. Assuming that the training image size is set to 256*256 for appropriate GPU hardware (such as RTX4090 24G), the image is split.

[0078] S32: performing data annotation on the split images, that is, annotating bounding boxes of imaging photos of different fluorescent particles to obtain annotation and classification results.

[0079] S33: Training in neural network frameworks such as pytoch and tensorflow.

[0080] S34: Perform inference on the new sample image, that is, (a) split the image, (b) use the trained model in 3 to perform inference, and (c) merge the results, exclude duplicate data on the segmentation boundary, and obtain the counting result.

[0081] S35: Based on the concentration and particle quantity information in other calibration information, inversely derive the concentration data of different objects to be tested corresponding to the particle quantity in the sample to be tested.

[0082] S4: Shut down the system.

Claims

1. A multi-factor digital fluorescence immunoassay system based on machine vision, characterized in that: The detection system includes a Windows PC, a three-dimensional translation stage, a microscope and a camera. The lens of the camera is connected to the eyepiece of the microscope. The three-dimensional translation stage control software, camera control software and image processing software are installed on the Windows PC. Under the control of the three-dimensional translation stage control software and the camera control software, the three-dimensional translation stage performs a scanning motion, stops at a specified position, and then focuses and takes pictures to obtain a microscopic image of the sample.

2. A multi-factor digital fluorescence immunoassay method based on machine vision, the detection method is based on the detection system according to claim 1, characterized in that: The specific steps are as follows: S1: Sample preparation; S2: imaging the sample area to be tested; S3: Process the image to identify and count particles of different colors, thereby obtaining multi-factor detection results in different areas; S4: Shut down the system.

Images