An immunochromatographic test strip quantitative analysis system and detection method based on image processing

Abstract

The present invention discloses an immunochromatographic test strip quantitative analysis system and detection method based on image processing, which belongs to the technical field of immunochromatographic quantitative analysis. The analysis system includes an image acquisition device for the test strip and a data processing device. The data processing device has a built-in image processing algorithm. The detection line and the quality control line are captured by binarizing the image and finding the center of mass of the image contour, and the gray value of the center of mass is calculated. The T / C gray ratio is taken as the characteristic value, which is input into the fitting equation of the corresponding project to calculate the concentration value of the object to be tested. The present invention directly uses a mask to capture the detection line and the quality control line in the image. Compared with the image processing method based on template matching of the detection instrument currently on the market, the analysis time is greatly shortened. Using the detection method of the present invention, it only takes about 10 seconds from image input to test result output.

Description

Technical Field

[0001] The present invention relates to the technical field of immunochromatography quantitative analysis, and in particular to an immunochromatography test strip quantitative analysis system and a detection method based on image processing. Background Art

[0002] Immunochromatography is a rapid detection technology that has emerged since the 1990s. It transfers the antigen-antibody reaction from a test tube or laboratory vessel to a test strip, uses the chromatography of the test strip to move the test solution in a specified direction to complete the entire antigen-antibody specific reaction, and makes a qualitative judgment by observing the color change at a specific position of the reagent strip with the naked eye. The specific mechanism is to spray the specific antibody on the nitrocellulose membrane in a strip shape, and adsorb the marker on the nanogold binding pad. When the test liquid (usually blood or urine) is dripped onto the sample pad at one end of the test strip, it will move forward through the capillary action of the nitrocellulose membrane, reach the binding pad and dissolve the marker. Then continue to move to the detection line area coated with antibodies. At this time, if the test liquid contains the corresponding antigen, the complex of the antigen and the marker will specifically bind to the antibody on the nitrocellulose membrane and be intercepted. For example, if the marker is colloidal gold, which has the characteristics of high electron density, it will cause a large amount of complexes to accumulate on the detection line and appear red or magenta visible to the naked eye; otherwise, the detection line area of ​​the test strip will not show color.

[0003] Early visual inspection had problems such as low efficiency, strong subjective factors, and misjudgment of positive and negative, and required professional inspection personnel to conduct inspections. With the continuous development of medical image processing technology, intelligent inspection has become a social development trend, and intelligent and networked inspection equipment has become the main research focus. Immunochromatography technology is developing towards high sensitivity, quantitative, and multivariate detection.

[0004] In addition to the test strips, the immunochromatographic quantitative detection system also requires supporting detection equipment, such as using an image sensor to collect test strip image information for analysis and processing, and finally obtaining a quantitative result. Patent document CN 104020286 A discloses an immunochromatographic test strip quantitative detector, including a scanning device, a data processing device, and an output device. The scanning device performs characteristic wavelength scanning and multi-wavelength scanning of non-characteristic wavelengths on the test strip; the data processing device performs data analysis, confirms the regional position of the quality control band and the detection band, and analyzes, identifies and calculates the optical density value, and calculates the test result according to the judgment standard value table and transmits it to the output device.

[0005] However, most of the colloidal gold and other lateral immunochromatographic devices currently on the market for automatic quantification cannot avoid the impact of differences between test strips on the test results, and cannot truly achieve more accurate and stable quantitative detection. Improving image processing and analysis methods can help improve detection accuracy. For example, patent document CN111478998 A discloses a mobile phone multi-wavelength quantitative rapid detection method for immunochromatographic test strips, which uses edge detection algorithms, reference ratio algorithms, RGB to wavelength algorithms, and Fourier transforms of image sequences in image acquisition and processing modules, effectively reducing human intervention in the measurement process and reducing background light noise.

[0006] However, although the accuracy of the results has been improved, the image processing of most quantitative detectors on the market is based on template matching. This method is computationally complex and takes a long time to process images. It usually takes several minutes to output the results. Therefore, it is necessary to make substantial improvements to the image processing method and invent a lateral immunochromatography quantitative instrument that can accurately and quickly interpret quantitative results and can be combined with a mobile phone APP or WeChat applet. Summary of the invention

[0007] The purpose of the present invention is to provide an immunochromatographic test strip quantitative analysis system based on image processing, which can realize rapid and accurate quantitative analysis of the test results of immunochromatographic test strips by optimizing the image processing algorithm. By organically combining the detector with the mobile phone APP and WeChat applet, a more systematic and comprehensive intelligent management of the test data can be achieved, making it more convenient for patients and experts to view the data and make accurate judgments.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] An immunochromatographic test strip quantitative analysis system based on image processing, comprising an image acquisition device and a data processing device, wherein the image acquisition device is used to acquire an image of the test strip; the data processing device is communicatively connected with the image acquisition device, and comprises an image processing module, a data analysis module and a human-computer interaction module;

[0010] The image processing module is used to receive and process the image sent by the image acquisition device, and the image processing includes: firstly, denoising and cropping the original image to obtain a detection window image containing the detection line and the quality control line and an image copy; then, converting the detection window image from the RGB mode to the HSV mode, performing image binarization processing to distinguish the color development areas of the detection line and the quality control line, and obtaining a binary image; then, performing image masking on the image copy, outputting an image corresponding to the color development area of ​​the binary image, finding the image contours of the detection line T column and the quality control line C column, and calculating the centroid; finally, calculating the grayscale value of the centroid;

[0011] The data analysis module is connected to the image processing module, receives the gray value data of the T column and the C column sent by the image processing module, takes the T / C gray ratio as the characteristic value, and inputs the characteristic value into the fitting equation of the corresponding item to calculate the concentration value of the object to be detected; the data processing module stores the detection parameters corresponding to each detection item and the fitting equation after the T / C gray ratio after image processing is fitted with the known target detection object concentration curve;

[0012] The human-computer interaction module is connected to the data analysis module and is used for inputting detection items and outputting detection results.

[0013] The quantitative analysis system provided by the present invention relates to an immunochromatographic test strip. The immunochromatographic test strip is a test strip prepared by labeling the corresponding antigen and antibody with colloidal gold or other colorimetric markers (such as microspheres, colloidal selenium, etc.). The test strip is coated with a detection line and a quality control line with known antigen and antibody concentrations. The present invention analyzes the color difference between the detection line and the quality control line on the immunochromatographic test strip based on image processing to achieve quantitative detection.

[0014] The present invention realizes quantitative detection through image analysis, therefore, the consistency of images between batches is a prerequisite for ensuring the accuracy of the detection results. The present invention has a matching closed image acquisition device, the image acquisition conditions remain consistent, and the error caused by image acquisition is reduced.

[0015] Furthermore, the image acquisition device includes a shell with a hollow structure, the shell is provided with a card slot that is limitedly assembled with the test strip shell, the card slot is connected to the internal cavity of the shell, and the shell is provided with a light source and a camera arranged opposite to the card slot; the camera is connected to the image processing module through an image transmission element.

[0016] Specifically, the sample to be tested is dripped onto a lateral flow immunochromatographic test strip that is pre-labeled and coated with the corresponding antigen and antibody. After the reaction is colored, the test strip is inserted into the detection card slot of the image acquisition device. The card slot limits the test strip to ensure that the detection line and quality control line on the test strip are placed within the image capture area of ​​the camera. The immunochromatographic test strip has a wrapping shell to avoid sample contamination problems. After the test strip shell is inserted into the card slot, it forms a closed space with the internal cavity of the shell, providing relatively consistent conditions for image acquisition and avoiding interference from environmental factors.

[0017] The shell size of the immunochromatographic test strips commonly used in the market and the color development positions of the detection line and the quality control line are generally the same. Therefore, the above device is suitable for image acquisition of most immunochromatographic test strips currently on the market.

[0018] The fixed light source in the housing provides light source for image acquisition, and the light source adopts white light.

[0019] The camera is based on CMOS sensor technology, with a resolution of 1080p, a 4-megapixel camera chip, an aperture of F2.4, and a capture area image size of 1280×720.

[0020] The image transmission element realizes the communication connection between the image acquisition device and the data processing device, and the communication method can adopt wireless communication methods such as WiFi, Bluetooth, radio frequency, etc. Preferably, the image transmission element is a Bluetooth sensor.

[0021] The light source, camera and image transmission element are all controlled by the switch element to turn on or off. Each component is connected to its own indicator light through the circuit. When the component is turned on, the indicator light is on, and when it is turned off, the indicator light is off. When the image transmission element is successfully connected to the image processing module, the indicator light changes color.

[0022] The image acquisition device is also provided with a power supply module for providing power to various components in the device.

[0023] Further, the image processing module is a microprocessor with a built-in image processing algorithm. In the present invention, image processing is implemented by calling a function in the Python OpenCV library. First, the original image is cropped to remove useless background, and a detection window image and an image copy containing a detection line and a quality control line are obtained. Then, the detection window image is converted into an HSV mode by color space conversion. Compared with the RGB space, the HSV space can very intuitively express the brightness, hue, and vividness of the color, which is convenient for color comparison; the image is binarized to separate the color development area of ​​the detection line and the quality control line from the background, and then the image copy is grayed and masked, and the color development area distinguished by the corresponding binarization processing is output, and the grayscale image of the color development area is obtained, and the image contour of the color development area is found and drawn, and the centroid of the detection line and the quality control line is calculated and drawn; finally, the grayscale value of the centroid is calculated.

[0024] Specifically, the mode conversion uses CV_RGB2GRAY, CV_RGB2HSY, CV_BGR2HLS and other forms; the image binarization processing uses the color picker function inRange() function; the image contour is found using the findContours() function; the center of mass is calculated using the moments() function and the center of mass coordinate calculation formula; the gray value is calculated using the numpy.mean() function.

[0025] Furthermore, the centroid is drawn using the circle() function, and the image outline is drawn using the rectangle() function. When debugging the program, drawing the image outline can determine whether the color display area is captured, which is convenient for adjusting the color range.

[0026] In the present invention, the image processing module finds the centroids of the detection line and the quality control line, and uses the grayscale value of the centroid to represent the grayscale value of the color development area. This method ensures the accuracy of the quantitative results while greatly saving the image processing operation time.

[0027] In order to improve the accuracy of the quantitative analysis results, noise reduction, edge enhancement and other processing are performed during the image processing process to avoid the influence of interference factors. Preferably, before the detection window image mode is converted, the image is Gaussian blurred; after the image mode is converted, the image is eroded; after the image is binarized, it is expanded, or after the image is eroded, a convolution filter is performed to make the image edge sharper and more obvious.

[0028] Specifically, the image processing algorithm includes: image denoising, image cropping, Gaussian blur, mode conversion (from RGB to HSV), image erosion, image convolution filtering, color extraction, expansion, image masking, finding and drawing image contours, calculating and drawing the centroid of the detection line (T column) and the quality control line (C column), and calculating the grayscale values ​​of the T column and the C column.

[0029] The image processing module transmits the gray value data to the data analysis module. The data analysis module stores the standard fitting equation for different detection items, which is a curve fitting of the T / C gray ratio and the known target detection object concentration. The data analysis module receives the data sent by the image processing module, takes the T / C gray ratio as the characteristic value, and inputs the characteristic value into the fitting equation of the corresponding item to calculate and output the concentration value of the object to be detected.

[0030] The fitting equation is obtained by fitting actual data of a large sample size. For different detection items, corresponding fitting equations need to be selected for data calculation. Specifically, the user can select the corresponding detection item through the human-computer interaction interface, and the data analysis module starts the fitting equation of the detection item for calculation, outputs the detection result, and then the human-computer interaction module displays the output.

[0031] The present invention uses the T / C grayscale ratio as a characteristic value, which can reflect three situations that occur in the test strip detection. 1. Both the test line and the quality control line are colored: the concentration of the target detection object is calculated according to the above steps; 2. The quality control line is colored and the test line is not colored: after image analysis, the grayscale value of the test line is calculated to be 0, and the characteristic value T / C is 0, that is, the output display is 0; 3. The test line is colored and the quality control line is not colored: after image analysis, the grayscale value of the quality control line is calculated to be 0, and the denominator of the characteristic value T / C is 0, then the data analysis module output displays "uploaded image abnormality".

[0032] Furthermore, the data processing device also includes a storage module for storing historical detection records. The storage module is connected to the human-computer interaction module, and the user can view the historical detection data through the human-computer interaction interface. Specifically, the user can retrieve historical data according to year, month, and day, and the results can be displayed in the form of a curve chart.

[0033] The image acquisition device and data processing device of the present invention can be integrated into one device or can be two separate devices. For ease of use, a mobile phone APP or WeChat applet is developed based on the above-mentioned image processing algorithm and data processing mode to achieve real-time transmission, analysis and storage of data through mobile phones and other communication devices.

[0034] The present invention also provides a detection method of an immunochromatographic test strip quantitative analysis system based on image processing, comprising the following steps:

[0035] (1) Turn on the image acquisition device and the data processing device, and connect them to each other; select the corresponding immunochromatographic detection item through the human-computer interaction module;

[0036] (2) starting the image acquisition device to acquire an image of the test paper strip;

[0037] (3) The image processing module of the data processing device receives and processes the image sent by the image acquisition device, wherein the image processing includes: firstly, denoising and cropping the image to obtain a detection window image and an image copy, and then converting the detection window image from the RGB mode to the HSV mode through mode conversion, and using a color picker function to set the pixel values ​​of the detection line and the quality control line area to white, and the remaining area to black, so as to obtain a black-and-white image; then, the image copy is grayed out and masked, and an image corresponding to the white part in the black-and-white image is output, and the image contours of the detection line T column and the quality control line C column are found, and the centroid of the two is calculated; finally, the grayscale value of the centroid is calculated;

[0038] (4) The data analysis module receives the grayscale values ​​of the T-column and C-column sent by the image processing module, takes the T / C grayscale ratio as the characteristic value, and inputs the characteristic value into the fitting equation of the corresponding project to calculate the concentration value of the object to be tested, and then transmits the calculation result to the human-computer interaction module for display output.

[0039] Furthermore, in step (3), the image processing method includes:

[0040] a. Use the filtering algorithm to reduce the noise of the test strip image, then intercept the target analysis area including the test line and the quality control line, remove the useless background, and obtain the test window image and the image copy;

[0041] b. Convert the detection window image from RGB mode to HSV mode, and then perform corrosion segmentation to separate independent image elements to obtain a preprocessed image;

[0042] c. Use the inRange() function to pick colors from the preprocessed image, set the pixel values ​​in the detection line and quality control line areas to white, and set the remaining areas to black to obtain a black and white image, and then dilate;

[0043] d. Convert the image copy in step a into a grayscale image, then perform image masking to output the image of the white part of the grayscale image corresponding to the black-and-white image to obtain the source image, then use the findContours() function to find the image contour of the source image, and then use the moments() function to determine the center of mass and calculate the coordinate position;

[0044] e. Use the if-else nested function to lock the centroid coordinate point, and use the numpy.mean() function to calculate the mean of the centroid grayscale value as the grayscale value of the T column or C column.

[0045] Preferably, in step a, the filtering algorithm is a median filtering algorithm.

[0046] The function and parameters used for image cropping: cropped = imgHsvExtrator[y0:y1,x0:x1], the cropping coordinates are [y0:y1,x0:x1], and the upper left corner of the original image is the origin of the coordinates. imgHsvExtrator = cv2.imread(), first use the imread method to read the image to be cropped.

[0047] Then take a copy of the cropped image: srcImg = cropped.copy().

[0048] The detection window image is used for image processing in steps b and c to determine the color development areas of the detection line and the quality control line; the image copy is used for processing in steps d and e to calculate the grayscale value of the color development area.

[0049] Preferably, in step b, before the mode conversion, Gaussian blur is used to perform noise reduction processing on the detection window image. Function and parameters used: gs_frame = cv2.GaussianBlur (cropped, (5,5), 0).

[0050] Mode conversion uses CV_RGB2GRAY, CV_RGB2HSY, CV_BGR2HLS, etc. Select the mode conversion method according to different color media. For example, the immunochromatographic test strip uses colloidal gold as the color marker, and the conversion from RGB to HSV mode uses the function: hsv = cv2.cvtColor (gs_frame, cv2.COLOR_BGR2HSV); for example, if polymer microspheres are used as color markers and appear black, then first convert from RGB to GARY grayscale image, then to RGB three channels, and then convert to HSV mode.

[0051] After the mode conversion, the image is eroded to separate independent image elements to prevent the formation of linkage areas (also part of the noise reduction process), and also play a role in strengthening the edges. Preferably, it is iterated twice, and the function used is: erode_hsv = cv2.erode (hsv, None, iterations = 2).

[0052] Preferably, the HSV pattern image is eroded and then combined with convolution filtering to make the image edge sharper and more obvious.

[0053] In step c, the cv2.inRange() function is used to perform the color picker color picking operation, and the pixel values ​​within the two thresholds are set to white (255), and the pixel values ​​outside the threshold interval are set to black (0). The function used is: masks = cv2.inRange (erode_hsv, l_b, u_b). The image value is lower than the l_b value and higher than the u_b value. The image value becomes 0 black, and the value between l_b and u_b becomes 255 white. Different color media have different color picking ranges. For example, colloidal gold takes the purple-red range, l_b = np.array ([127, 30, 0]); u_b = np.array ([180, 255, 148]), then the pixel values ​​between 127-180; 30-255; 0-148 are set to white. The microsphere takes the black range.

[0054] In order to eliminate the situation where the T-pillar is broken, the black and white image after color extraction is expanded.

[0055] In step d, the image mask is for the copy srcImg cropped in step a, and only the part of the output image pixel whose element at the corresponding position of mask is not 0 (i.e. white) is output, otherwise all channel components of the pixel at this position are set to 0 (black); the function used is: res = cv2.bitwise_and(srcImg, srcImg, mask = masks).

[0056] Find the image contour for the source image obtained after the image mask, the function used is:

[0057] cnts=list(cv2.findContours(masks.copy(),cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)[-2])

[0058] The cv2.findContours() function first returns a list. Each element in the list is a contour in the image, and each element is traversed one by one to find the centroid.

[0059] The function used to find the center of mass is: M_center = cv2.moments(cnts[i]). The moments function provides a dictionary of all calculated moment values, which can be used to extract useful data - the center of mass.

[0060] The formula for calculating the horizontal coordinate of the center of mass is: cx = int(M_center["m10"] / M_center["m00"]); the formula for calculating the vertical coordinate of the center of mass is: cy = int(M_center["m01"] / M_center["m00"]).

[0061] Preferably, the center of mass is drawn using the circle drawing function, the function used is: cv2.circle(res,(cx,cy),7,128,-1), the parameters are the source image, the center coordinates of the circle, the radius of the circle, the color of the circle boundary line, and the thickness of the circle boundary line in pixels, where the settings of radius, color, and line thickness can be adjusted according to specific circumstances.

[0062] Frame the image outline corresponding to the centroid. The function used is: cv2.rectangle(res,(x,y),(x+w,y+h),(255,0,0),2,cv2.LINE_AA). The parameters are the source image, the starting coordinates of the rectangular outline, the ending coordinates of the rectangular outline, the line color, the thickness of the line in pixels, and anti-aliasing smoothing. The line drawing parameter settings can be adjusted according to the specific situation.

[0063] In step e, after the centroid is determined, the centroid coordinate point is locked in the grayscale image using the if-else nested function, and the mean of the grayscale values ​​of the four adjacent pixels above, below, left and right of the centroid is obtained as the grayscale value through the numpy.mean(a, axis, dtype, out, keepdims) function.

[0064] In step (4), the data analysis module receives the instruction from the human-computer interaction module to call the fitting curve model of the corresponding project, and takes the T / C grayscale ratio of the data sent by the image processing module as the feature value, inputs it into the fitting curve model, and runs the output result.

[0065] In the present invention, the fitting equation in the data analysis module is updated or new detection items are added through software upgrade.

[0066] The present invention has the following beneficial effects:

[0067] (1) The immunochromatographic test strip quantitative analysis system provided by the present invention is equipped with an image acquisition device, which uses a built-in camera in the instrument to take pictures, thereby reducing errors caused by taking pictures.

[0068] (2) The data processing device of the present invention has a built-in image processing algorithm, which captures the detection line and the quality control line by binarizing the image and finding the center of mass of the image contour. The method is simple and has high accuracy. Using the detection method of the present invention, it only takes about 10 seconds from image input to detection result output. The present invention directly uses a mask to capture the detection line and the quality control line of the detection window. Compared with the image processing method based on template matching of the current detection instruments on the market, the analysis time is greatly shortened.

[0069] (3) Based on the image processing algorithm provided by the present invention, a mobile phone APP or WeChat applet is developed, which can realize real-time transmission, analysis and storage of data as well as remote management of data through portable communication devices such as mobile phones and tablet computers. It can replace some of the existing bulky detection equipment, making it more portable in actual applications, and providing customers with digital medical services without leaving their homes.

[0070] (4) The present invention is universal and applicable to the quantitative result determination of immunochromatographic test strips of different specifications and detection types. Semi-quantitative detection can also be achieved using qualitative test strips. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 Schematic diagram of the image processing flow of the present invention.

[0072] Figure 2 This is the image captured by the image capture device in Example 1.

[0073] Figure 3 This is the image obtained after cropping the image in Example 1.

[0074] Figure 4 This is the image after Gaussian blur in Example 1.

[0075] Figure 5 This is the image converted into HSV mode in Example 1.

[0076] Figure 6 This is the image after color extraction and expansion in Example 1.

[0077] Figure 7 This is a grayscale image of the cropped copy of the image in Example 1.

[0078] Figure 8 This is the image after drawing the centroid and image contour in Example 1. DETAILED DESCRIPTION

[0079] The present invention will be further described below in conjunction with specific embodiments. The following embodiments are only used to illustrate the present invention and are not intended to limit the scope of application of the present invention. Without departing from the spirit and essence of the present invention, modifications or replacements made to the inventive method, steps or conditions all belong to the scope of the present invention.

[0080] Example 1

[0081] This embodiment provides an immunochromatographic test strip quantitative analysis system, which relates to an immunochromatographic test strip, which is a test strip prepared by labeling corresponding antigens and antibodies with colloidal gold or microspheres, colloidal selenium and other particles, and the test strip has a plastic shell that wraps the test paper. This embodiment is described by taking a colloidal gold test strip as an example.

[0082] The immunochromatographic test strip quantitative analysis system in this embodiment is composed of an image acquisition device and a data processing device, wherein the image acquisition device is used to acquire the image of the test strip; the data processing device analyzes and processes the acquired image to provide a quantitative analysis result.

[0083] Specifically, the image acquisition device includes a shell with a hollow structure, the shell is provided with a card slot that is limitedly assembled with the test strip shell, the card slot is connected to the internal cavity of the shell, and the shell is provided with a light source and a camera facing the card slot, wherein the light source adopts white light to provide light source for taking pictures, the camera adopts model CMOS OV2640, the resolution is 1080p, the camera chip is 4 million pixels, and the aperture is F2.4. The image size of the capture area is 1280×720. The camera is connected to the data processing device through a Bluetooth sensor.

[0084] After the test strip shell is inserted into the card slot, it forms a closed space with the internal cavity of the shell, providing relatively consistent conditions for image acquisition. The card slot limits the test strip to ensure that the detection line and quality control line on the test strip are placed within the image capture area of ​​the camera. The camera sends the image to the data processing device via Bluetooth transmission.

[0085] The shell is provided with a switch element to control the opening or closing of the light source, camera and image transmission element. Each component is connected to its own indicator light through a circuit. When the component is turned on, the indicator light lights up, and when it is turned off, the indicator light goes out. When the Bluetooth sensor communicates with the image processing module successfully, the indicator light changes color.

[0086] The housing is also provided with a power supply module for providing power to various components in the device. The power supply is provided by a rechargeable battery.

[0087] The data processing device includes an image processing module, a data analysis module, a human-computer interaction module and a storage module. The image processing module is used to receive the image sent by the image acquisition device and process it through the built-in image processing algorithm to calculate the grayscale value of the test line T and the quality control line C; the data analysis module stores a grayscale ratio and concentration conversion relationship table, the data analysis module is connected to the image processing module, receives the grayscale value data sent by it, takes the T / C grayscale ratio as the characteristic value, and inputs the characteristic value into the fitting equation of the corresponding project to calculate the concentration value of the object to be tested; the human-computer interaction module is connected to the data analysis module, the user selects the test project through the human-computer interaction interface, the data analysis module calls the calculation program of the corresponding test project to perform data analysis, and the calculation result is displayed and output by the human-computer interaction interface. The historical test records are stored in the storage module, the storage module is connected to the human-computer interaction module, and the user views the historical test data through the human-computer interaction interface.

[0088] In this embodiment, the method for quantitatively detecting a transverse immunochromatographic test strip includes: dripping a sample to be tested onto a colloidal gold immunochromatographic test strip pre-labeled with a corresponding antigen-antibody, then inserting the test strip into a detection slot of an image acquisition device, starting a camera to collect image information of a test line and a quality control line on the test strip, and transmitting the image to a data processing device. The data processing device obtains the concentration of the positive marker contained in the sample to be tested and displays and stores the result value based on the color chromaticity of the test band (T line) and the quality control band (C line) on the nitrocellulose membrane of the test strip and the background chromaticity according to the calculation program inside the instrument or the gray value and concentration conversion relationship table.

[0089] In order to realize the established functions of the detection system, it is necessary to take corresponding processing measures according to the characteristics of the collected test strip image, then select appropriate characteristic values ​​for calculation, and finally fit the working curve according to the known concentration and the measured characteristic values. The specific method is as follows:

[0090] 1. The specific process of image processing (calling functions in the Python OpenCV library) is:

[0091] 1. Image Noise Reduction:

[0092] Due to interference from imaging devices and circuit non-idealities, noise is still inevitably present in the image, that is, it is impossible to completely avoid the presence of noise in the collected image, so some image filtering methods must be used to solve the noise in the image.

[0093] The filtering algorithm adopted is the median filtering algorithm. In the neighborhood of the currently processed pixel, the grayscale values ​​of all pixels are compared, arranged in order, and then the median value is taken to replace the grayscale value of the original pixel.

[0094] The function used is: MedianFilter(img,k=3,padding=None).

[0095] 2. Image cropping:

[0096] like Figure 2 As shown in the figure, the original image captured by the camera not only contains valid information such as the test line and the quality control line, but also contains invalid information such as the test window background and the test strip shell. Among them, the test strip shell occupies the largest proportion in the image and has a clear boundary with the test window containing the test line and the quality control line. Therefore, in order to reduce the interference of invalid information on the final result, it is necessary to segment the original test strip image and extract the test window containing valid information. After obtaining an accurate test window image, the test window image is processed accordingly to obtain the information of the test line and the quality control line, thereby realizing the quantification of the test results.

[0097] like Figure 3 As shown, the image is cropped to extract the target analysis area, laying the foundation for the subsequent color block extraction and reducing interference. Functions and parameters used:

[0098] cropped=imgHsvExtrator[300:400,380:940]

[0099] Note (#): The cropping coordinates are [y0:y1,x0:x1], and the upper left corner of the original image is the origin of the coordinates.

[0100] imgHsvExtrator=cv2.imread("block1.jpg")

[0101] #First use the imread method to read the image to be cropped.

[0102] Then take a copy of the cropped image: srcImg = cropped.copy(). The copy is converted to grayscale and used as the image mask later.

[0103] 3. Gaussian blur:

[0104] Also known as Gaussian smoothing, it is part of the noise reduction process. The image generated by this blurring technique has a visual effect like looking at the image through frosted glass, which is significantly different from the bokeh effect of the lens and the effect in ordinary lighting shadows. Gaussian smoothing is often used in the pre-processing stage of computer vision algorithms to reduce image noise and reduce the level of detail.

[0105] Functions and parameters used:

[0106] gs_frame=cv2.GaussianBlur(cropped,(5,5),0)

[0107] #Three parameters: original image, width and height of Gaussian kernel (odd numbers are recommended), standard deviation of x and y axes.

[0108] The image after Gaussian blur is as follows Figure 4 shown.

[0109] 4. Mode conversion:

[0110] The colloidal gold immunochromatographic test strip image is converted from RGB to HSV mode, such as Figure 5 shown.

[0111] Conversion purpose: The conversion requirements between the two come from the needs of hardware implementation and display effect adjustment. The RGB mode satisfies the simple and efficient implementation in the specific processing process, and the HSV mode is adjusted according to the recognition characteristics of the human eye, which makes it easier to achieve the display effect adjustment expected by the human eye.

[0112] Functions used:

[0113] hsv=cv2.cvtColor(gs_frame,cv2.COLOR_BGR2HSV)

[0114] #BGR to HSV conversion modes include CV_RGB2GRAY, CV_RGB2HSY, CV_BGR2HLS, etc.

[0115] 5. Corrosion segmentation:

[0116] Erosion isolates independent image elements to prevent the formation of linked areas (also part of the noise reduction process) and strengthen the edges.

[0117] Functions and parameters used:

[0118] erode_hsv=cv2.erode(hsv,None,iterations=2)

[0119] #Three parameters: original image, convolution kernel, number of iterations = 2; corrosion was performed twice.

[0120] 6. Pick colors

[0121] Use the cv2.inRange() function to perform the color picker color picking operation: it can realize the binarization function, and more importantly, it can operate on multiple channels at the same time - mainly setting the pixel values ​​within the two thresholds to white (255), and the pixel values ​​not in the threshold interval to black (0).

[0122] Functions and parameters used:

[0123] masks=cv2.inRange(erode_hsv,l_b,u_b)

[0124] l_b = np.array([127,30,0])

[0125] u_b = np.array([180,255,148])

[0126] #In the image, values ​​lower than l_b and higher than u_b become 0 black, while values ​​between l_b and u_b become 255 white;

[0127] #inrange turns the image into black and white. This embodiment extracts purple and turns it into white between 127-180; 30-255; 0-148.

[0128] 7. Expansion

[0129] Functions and parameters used:

[0130] cv2.dilate(masks,None,iterations=11)

[0131] Expand 11 times: Eliminate the situation where the T-pillar is broken, otherwise the next centroid will become two centers. Figure 6 shown.

[0132] 8. Image Mask:

[0133] The cropped copy is srcimg, and the copy is converted to grayscale as follows Figure 7 As shown, the image mask outputs image pixels only when the mask corresponding position element is not 0 (i.e. white), otherwise all channel components of the pixel at that position are set to 0 (black);

[0134] Functions used:

[0135] res=cv2.bitwise_and(srcImg,srcImg,mask=masks)

[0136] 9. Find image contours:

[0137] The source image is a copy of the mask (preferably a binary image); only the outermost contour is detected, and the inner contour contained in the outer contour is ignored; only the inflection point information of the contour is saved, and the information points on the straight line segment between the inflection points are not retained; the contour of each target can be easily obtained by directly using the findContours() function of Opencv, but after visualization, the order is disordered and needs to be sorted;

[0138] Functions used:

[0139] Cnts=list(cv2.findContours(masks.copy(),cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)[-2])

[0140] 10. Find the center of mass:

[0141] The cv2.findContours() function first returns a list. Each element in the list is a contour in the image. It traverses each element one by one to find the centroid - using the moments() function.

[0142] Functions used:

[0143] M_center=cv2.moments(cnts[i])

[0144] The #moments function provides a dictionary of all calculated moment values, from which useful data can be extracted - the centroid.

[0145] cx=int(M_center["m10"] / M_center["m00"])

[0146] #Calculation formula for the horizontal coordinate of the center of mass;

[0147] cy=int(M_center["m01"] / M_center["m00"])

[0148] #Calculation formula for the centroid ordinate.

[0149] 11. The circle() function draws the centroid, and the rectangle() function frames the found contour:

[0150] Functions and parameters used:

[0151] cv2.circle(res,(cx,cy),7,128,-1)

[0152] #Draw the center point and use the circle function (that is, draw the found center of mass).

[0153] cv2.rectangle(res,(x,y),(x+w,y+h),(255,0,0),2,cv2.LINE_AA)

[0154] #Source image; coordinates of the upper left point of the rectangular outline; coordinates of the lower right point of the rectangular outline; RGB color corresponding to the line (blue); 2 is the width of the drawn line; anti-aliasing smoothing.

[0155] like Figure 8 shown.

[0156] 12. Use the if-else nested function to lock the centroid coordinate point, and use the numpy.mean(a, axis, dtype, out, keepdims) function to calculate the mean of the grayscale values ​​of the four adjacent pixels above, below, left and right of the centroid as the grayscale values ​​of the T column and the C column respectively.

[0157] In this embodiment, the grayscale ratio of T / C is selected as the eigenvalue. Three situations need to be discussed: ① T and C are both present: the grayscale ratio T / C can be calculated according to the above steps; ② C is present but T is not: the eigenvalue T / C is 0, that is, the output is displayed as 0; ③ T is present but C is not: the denominator of the eigenvalue T / C is 0, that is, the output shows that the uploaded image is abnormal. The eigenvalue selected is T / C, and each of the above three situations is well handled.

[0158] 2. Fitting equation

[0159] Combine the T / C grayscale ratio with the concentration value of the standard solution, draw a correlation curve between the two, and obtain the relationship equation and correlation coefficient value.

[0160] In this embodiment, based on the above-mentioned image processing algorithm and data processing mode, a corresponding mobile phone APP is developed, and the whole process is completed based on the mobile phone APP end. The mobile phone is connected to the image acquisition device through communication to realize real-time transmission of data, and the processing software on the smart phone analyzes, stores and displays the data. The fitting equation in the data analysis module can be updated or new detection items can be added through software upgrade.

[0161] In this embodiment, a device identification code is printed on the image acquisition device, and the smart phone connects to the device by scanning the identification code.

[0162] Application Example 1

[0163] 1. Quantitative detection of early pregnancy (HCG) test strips (colloidal gold method) using the immunochromatographic test strip quantitative analysis system of Example 1 is performed, and the steps are as follows:

[0164] (1) Turn on the image acquisition device, start the mobile phone APP, enter "My-Device Management-Add Instrument" in the human-computer interaction interface, scan the identification code on the image acquisition device, turn on the mobile phone Bluetooth, and connect the mobile phone to the image acquisition device;

[0165] (2) Drop the urine sample to be tested onto the early pregnancy (HCG) test strip, and then insert the test strip into the card slot of the image acquisition device.

[0166] (3) Start the image acquisition device to take a photo, enter the mobile phone APP module detection page, and select "Early Pregnancy Test".

[0167] (4) Click “Start Detection” and wait for the test results.

[0168] (5) The built-in algorithm calculates and analyzes the test result value, and displays the concentration value and the corresponding expert advice within the range on the result page.

[0169] (6) Enter the "My-Test Record" on the mobile app, select the corresponding test item, and you can view the test curve of concentration changes over time and observe the change trend.

[0170] 2. Results Analysis

[0171] This embodiment uses David test strips commonly used on the market to test 100 urine samples with known HCG concentrations, and then uses the above system for quantitative analysis. The results are displayed in about 8 to 9 seconds, and the accuracy of the results is above 85.62%.

[0172] Application Example 2

[0173] 1. Quantitative detection of ovulation (LH) test strip (colloidal gold method) using the immunochromatographic test strip quantitative analysis system of Example 1 is performed as follows:

[0174] (1) Turn on the image acquisition device, start the mobile phone APP, enter "My-Device Management-Add Instrument", scan the identification code on the image acquisition device, turn on the mobile phone Bluetooth, and connect the mobile phone to the image acquisition device;

[0175] (2) Add the urine sample to be tested onto the ovulation (LH) test strip, and then insert the test strip into the card slot of the image acquisition device.

[0176] (3) Start the image acquisition device to take a photo, enter the mobile phone APP module detection page, and select "Ovulation Detection".

[0177] (4) Click “Start Detection” and wait for the test results.

[0178] (5) The built-in algorithm calculates and analyzes the test result value, and displays the concentration value and the corresponding expert advice within the range on the result page.

[0179] (6) Enter the mobile APP "My-Test Record", select the corresponding test item, and you can view the test curve of concentration change over time and observe the change trend.

[0180] 2. Results Analysis

[0181] This embodiment uses the commonly used Jinxiuer test strips on the market to test 100 urine samples with known LH concentrations, and then uses the above system for quantitative analysis. The results are displayed in about 8 to 9 seconds, and the accuracy of the results is above 81.25%.

[0182] Finally, it should be noted that the above specific implementation methods are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. An immunochromatographic test strip quantitative analysis system based on image processing, characterized in that: It includes an image acquisition device and a data processing device, wherein the image acquisition device is used to acquire the image of the test paper strip; the data processing device is communicatively connected with the image acquisition device, and includes an image processing module, a data analysis module and a human-computer interaction module; The image processing module is used to receive and process the image sent by the image acquisition device, and the image processing includes: firstly, denoising and cropping the original image to obtain a detection window image containing the detection line and the quality control line and an image copy; then, the detection window image is converted from the RGB mode to the HSV mode, and the image is binarized to distinguish the color development areas of the detection line and the quality control line to obtain a binary image; then, the image copy is converted to grayscale and an image mask is performed, and an image corresponding to the color development area of ​​the binary image is output, and the image contours of the detection line T column and the quality control line C column are found, and the centroid is calculated; finally, the grayscale value of the centroid is calculated; The data analysis module is connected to the image processing module, receives the gray value data of the T column and the C column sent by the image processing module, takes the T / C gray ratio as the characteristic value, and inputs the characteristic value into the fitting equation of the corresponding item to calculate the concentration value of the object to be detected; the data processing device stores the detection parameters corresponding to each detection item and the fitting equation after the T / C gray ratio after image processing is fitted with the known target detection object concentration curve; The human-computer interaction module is connected to the data analysis module and is used for inputting detection items and outputting detection results.

2. The immunochromatographic test strip quantitative analysis system based on image processing according to claim 1, characterized in that: The image acquisition device includes a shell with a hollow structure, the shell is provided with a card slot that is limitedly assembled with the test strip shell, the card slot is connected to the internal cavity of the shell, and a light source and a camera arranged opposite to the card slot are provided inside the shell; the camera is connected to the image processing module through an image transmission element.

3. The immunochromatographic test strip quantitative analysis system based on image processing according to claim 2, characterized in that: The light source is white light; the camera is a camera based on CMOS sensor technology; and the image transmission element is a Bluetooth sensor.

4. The immunochromatographic test strip quantitative analysis system based on image processing according to claim 1, characterized in that: The image processing module is a microprocessor with a built-in image processing algorithm, wherein the image binarization process uses the inRange() function; the image contour is found using the findContours() function; the centroid is found using the moments() function and the centroid coordinate calculation formula; and the gray value is calculated using the numpy.mean() function.

5. The immunochromatographic test strip quantitative analysis system based on image processing according to claim 1, characterized in that: Before the detection window image mode conversion, the image is Gaussian blurred; after the image mode conversion, the image is eroded; and after the image is binarized, it is expanded.

6. A detection method of the immunochromatographic test strip quantitative analysis system based on image processing according to any one of claims 1 to 5, characterized in that: The following steps are involved: (1) Turn on the image acquisition device and the data processing device, and connect them to each other; select the corresponding immunochromatographic detection item through the human-computer interaction module; (2) starting the image acquisition device to acquire an image of the test paper strip; (3) The image processing module of the data processing device receives and processes the image sent by the image acquisition device, wherein the image processing includes: firstly, denoising and cropping the image to obtain a detection window image and an image copy, and then converting the detection window image from the RGB mode to the HSV mode through mode conversion, and using a color picker function to set the pixel values ​​of the detection line and the quality control line area to white, and the remaining area to black, so as to obtain a black-and-white image; then, the image copy is grayed out and masked, and an image corresponding to the white part in the black-and-white image is output, and the image contours of the detection line T column and the quality control line C column are found, and the centroid of the two is calculated; finally, the grayscale value of the centroid is calculated; (4) The data analysis module receives the grayscale values ​​of the T-column and C-column sent by the image processing module, takes the T / C grayscale ratio as the characteristic value, and inputs the characteristic value into the fitting equation of the corresponding project to calculate the concentration value of the object to be tested, and then transmits the calculation result to the human-computer interaction module for display output.

7. The detection method according to claim 6, characterized in that In step (3), the image processing method comprises: a. Use the filtering algorithm to reduce the noise of the test strip image, then intercept the target analysis area including the test line and the quality control line, remove the useless background, and obtain the test window image and the image copy; b. Convert the detection window image from RGB mode to HSV mode, and then perform corrosion segmentation to separate independent image elements to obtain a preprocessed image; c. Use the inRange() function to pick colors from the preprocessed image, set the pixel values ​​in the detection line and quality control line areas to white, and set the remaining areas to black to obtain a black and white image; d. Convert the image copy in step a into a grayscale image, then perform image masking to output the image of the white part of the grayscale image corresponding to the black-and-white image, obtain the source image, then use the findContours() function to find the image contour, and then use the moments() function to determine the center of mass and calculate the coordinate position; e. Use the if-else nested function to lock the centroid coordinate point, and use the numpy.mean() function to calculate the mean of the centroid grayscale value as the grayscale value of the T column or C column.

8. The detection method according to claim 7, characterized in that In step a, the filtering algorithm is a median filtering algorithm.

9. The detection method according to claim 7, characterized in that: In step b, before mode conversion, Gaussian blur is used to perform noise reduction on the detection window image; after mode conversion, the image erosion segmentation is iterated twice.

10. The detection method according to claim 7, characterized in that: In step d, after determining the centroid position, use the circle() function to draw the centroid and use rectangle() to frame the corresponding outline.

Images