An immunochromatographic image analysis method and storage medium

Abstract

The present application relates to the technical field of colloidal gold immunochromatography, and discloses an immunochromatography image analysis method and a storage medium. A target detection module is constructed and trained, and is used to detect an image of an immunochromatography strip to be detected to obtain a region where a T line is located. A cutting module is constructed, and is used to cut and process a local region of the immunochromatography strip image to be detected, where the T line is located, to eliminate noise at the image boundary, and to reduce images irrelevant to detection and the influence of useless information on network training and operation. Finally, an output module is constructed, and is used to extract features of the cut immunochromatography strip image, encode the extracted features, and decode the features into one-hot coding of concentration. The one-hot coding is calculated to obtain a concentration represented by the T line in the immunochromatography strip.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of colloidal gold immunochromatography technology, in particular to an immunochromatography image analysis method and a storage medium. BACKGROUND

[0002] The immunochromatography method realizes the rapid detection of the concentration of the target by combining the antigen-antibody binding immunological reaction and the chromatography technology. At present, it is widely used in the screening and detection of food, nucleic acid and the like due to the advantages of high accuracy, fast detection speed, simple and convenient detection and instant on-site detection. However, the method needs to use a biological detection instrument to quantitatively analyze the concentration result. The expensive, bulky and complex operation of the biological detection instrument makes the fluorescent immunochromatography method unable to be widely applied to the concentration detection. The existing immunochromatography image analysis method mainly compares the RGB of the provided whole image without considering the influence of image noise, which may lead to inaccurate extraction result. Moreover, the large image results in large calculation amount, which increases the detection cost. SUMMARY

[0003] Therefore, the present application aims to overcome the problems of the prior art, such as the expensive, bulky and complex operation of the biological detection instrument, which makes it unable to be widely applied to the concentration detection, and the large calculation amount and high cost caused by the size of the image.

[0004] To solve the above technical problems, the present application provides an immunochromatography image analysis method, comprising:

[0005] S1: obtaining an immunochromatography strip image to be detected;

[0006] S2: using a trained target detection module constructed to detect the immunochromatography strip image to be detected, and positioning a local area where a T line of the immunochromatography strip image is located in the immunochromatography strip image;

[0007] S3: using a constructed cropping module to crop the local area where the T line of the immunochromatography strip image to be detected is located to obtain a cropped image with a length-width ratio of w x h, wherein w > h, and scaling the width of the cropped image to y pixels according to the length-width ratio of w x h, wherein y > 300, at this time the length of the cropped image is y / h x w, and the cropped image with the scaled width and height is center-cropped to retain a region with a pixel size of a x a in the center of the image, wherein a < y, and the region with the pixel size of a x a is reduced to a region with a pixel size of b x b, wherein b < a, to obtain a cropped strip image;

[0008] S4: using the constructed output module to perform feature extraction on the cropped paper strip image, encoding and decoding the extracted features into one-hot encoding of concentration, and calculating the output one-hot encoding to obtain the concentration represented by the T line in the immunochromatography paper strip.

[0009] Further, the target detection module comprises a target detection algorithm, and the target detection algorithm is a YOLOv5 target detection algorithm network.

[0010] Further, the training set of the YOLOv5 target detection algorithm network during training is an immunochromatography paper strip image labeled by artificial marking, and a data enhancement mechanism is used to process the immunochromatography paper strip image.

[0011] Further, the data enhancement mechanism comprises: random rotation, random cropping, and up-down or left-right flipping, the rotation angle range of the random rotation is set to -20°-20°, the cropping ratio of the random cropping should be the same as the cropping module, that is, the width of the cropped image is first scaled to y pixels, where y>300, at this time the length of the cropped image is y / h x w, the cropped image with the scaled width and height is center cropped, and the center pixel size of the region is a x a, where a<y, the region with the pixel size a x a obtained is reduced to a region with the pixel size b x b, where b<a.

[0012] Further, the calculation of the output one-hot encoding to obtain the concentration represented by the T line in the immunochromatography paper strip is specifically:

[0013]

[0014] where C i represents the concentration value corresponding to the i-th bit of the one-hot encoding, P i represents the probability that the concentration of the to-be-detected substance is C i .

[0015] Further, the output module comprises:

[0016] a feature extractor for extracting image features;

[0017] an encoder for encoding the features into a single value;

[0018] a decoder for decoding the single value output by the encoder into one-hot encoding of concentration.

[0019] Further, the encoder in the output module adopts a full connection design method, and the full connection design method is to first map the feature vector extracted by the convolution kernel into a single value and then map it into one-hot encoding.

[0020] Further, the loss function of the output module adopts a CCE loss function.

[0021] Further, the target detection module and the output module both use an Adam optimizer.

[0022] A storage medium, characterized in that: the storage medium has a computer program stored thereon, the computer program is executed by a processor to implement the steps of the immune chromatography image analysis method of any one of the above.

[0023] The immune chromatography image analysis method and the storage medium provided by the application are based on the image detection method of the convolutional neural network, and are designed around the characteristics that the T line RGB value of the immune chromatography test strip image and the concentration of the measured object exist a relationship. The T line region is extracted through a target detection module, and a cropping module is used to process the extracted picture, so that the noise of the image boundary is eliminated, the image irrelevant part is reduced, the influence of useless information on network training and operation is reduced, and finally the concentration information is output by using a convolution network. The method reduces the influence of useless information of the picture on network training and operation, so that the network also has high performance in the case of less data, and does not need to use biological detection instruments. The image acquisition device and the analysis device used by the application have low cost and simple operation, and the concentration of the measured object in the immune test strip is analyzed conveniently and efficiently. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to make the content of the application more easily understood, the application will be further described in detail below according to specific embodiments of the application and in combination with the drawings, in which

[0025] Figure 1 is a flowchart of an immune chromatography image analysis method of an embodiment of the application;

[0026] Figure 2 is a schematic diagram of an image collection device of an embodiment of the application;

[0027] Figure 3 is a decomposition diagram of an immune chromatography image detection method of an embodiment of the application;

[0028] Figure 4 is a schematic diagram of a cropping module of an embodiment of the application;

[0029] Figure 5 is a structure diagram of a convolutional neural network of an output module of the application;

[0030] Figure 6 is a parameter optimization diagram of a convolutional neural network of an output module in an embodiment of the application;

[0031] Figure 7is a loss of the output module convolutional neural network training graph changing with the number of training times in the embodiment of the present application;

[0032] Figure 8 is a recall and accuracy of the target detection algorithm changing with the training cycle graph in the embodiment of the present application;

[0033] Figure 9 is a positioning error changing with the training cycle graph of the target detection algorithm training set and test set in the embodiment of the present application;

[0034] Figure 10 is the analysis result before and after labeling in the embodiment of the present application. DETAILED DESCRIPTION

[0035] The present application will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present application and implement it, but the embodiments are not as a limitation on the present application.

[0036] Embodiment one:

[0037] Referring to Figure 1 The flow chart of the embodiment one of the present application is shown in the figure, which includes:

[0038] S1: obtaining an image of an immune chromatography strip to be detected;

[0039] S2: using the trained target detection module constructed to detect the image of the immune chromatography strip to be detected, and positioning the local area where the T line of the image of the immune chromatography strip is located in the immune chromatography strip;

[0040] S3: using the constructed cropping module to crop the local area where the T line of the image of the immune chromatography strip to be detected is located to obtain a cropped image with a length-width ratio of w x h, wherein w>h, and the width of the cropped image is scaled to y pixels according to the length-width ratio of w x h, wherein y≥300, at this time the length of the cropped image is y / h x w, the cropped image with the scaled width and height is center cropped, and the area with a pixel size of a x a in the center of the picture is retained, wherein a<y, the area with the pixel size of a x a obtained is reduced to an area with a pixel size of b x b, wherein b<a, and a cropped strip image is obtained;

[0041] S4: using the constructed output module to extract features from the cropped strip image, and encoding and decoding the extracted features into one-hot encoding of concentration, and calculating the output one-hot encoding to obtain the concentration expectation represented by the T line in the immune chromatography strip.

[0042] The immunochromatography image analysis method provided by the application is based on an image detection method of a convolutional neural network, is designed around the characteristics that the T line RGB value of an immunochromatography test strip image and the concentration of a to-be-detected substance exist in a relationship, does not need a biological detection instrument for detection, and utilizes a constructed cropping module to crop the picture, eliminates the noise of the image boundary, reduces the image of the part irrelevant to detection, reduces the influence of useless information on network training and operation, enables the network to also have high performance in the case of less data, and realizes convenient and efficient analysis of the concentration of the to-be-detected substance in the immunochromatography test strip.

[0043] Embodiment two

[0044] The image collection device of the embodiment two of the application is shown in Figure 2

[0045] The image collection device can be composed of an ultraviolet light source with a wavelength of 254 nm, a light isolation box and an OPPO mobile phone with a model of PFJM10.

[0046] The immunochromatography image detection method of the embodiment two of the application is shown in Figure 3

[0047] After the immunochromatography test strip image is collected by the image collection device, the image is output to a target detection module, the target detection module includes a YOLOv5 target detection algorithm, and the target detection algorithm is used to receive the image collected from the image collection device and locate the local area where the T line of the immunochromatography image is located.

[0048] The cropping module constructed by the application is shown in Figure 4 The cropping module crops the local area where the T line of the immunochromatography test strip image is located in the to-be-detected immunochromatography test strip image to obtain a cropped image with a length-width ratio of w x h, the width of the obtained cropped image is scaled to 340 pixels according to the length-width ratio of w x h, at this time, the length of the cropped image is 340 / h x w, the cropped image with the changed width and height is center-cropped, a region with a pixel size of 300 x 300 in the center of the picture is reserved, the obtained region with a pixel size of 300 x 300 is reduced to a region with a pixel size of 50 x 50, and a cropped test strip image is obtained.

[0049] The cropping module constructed by the application eliminates the noise of the image boundary, reduces the image of the part irrelevant to detection, and reduces the influence of useless information on network training and operation.

[0050] The cropped test strip image is input into the constructed output module as shown in Figure 5 ​​As shown, the output module is composed of a feature extractor, an encoder and a decoder. The feature extractor is composed of a convolutional layer and a pooling layer, which is used to extract image features. The encoder is composed of a fully connected layer and a Dropout layer, which is used to encode the features into a single value. The Dropout layer has a deactivation probability of 95%. The decoder is composed of a fully connected layer, which is used to decode the single value into a one-hot encoding of the concentration. When training the network, the training data is the picture of the YOLOv5 module manually labeled region, and data augmentation is used to enhance the data amount. Data augmentation includes random rotation, random cropping and up-down or left-right flipping. The random rotation angle range is set to -20°-20°. The cropping ratio of random cropping should be the same as the cropping module, that is, first scale the picture length-width proportionally to the short side pixel length to 340, then perform center cropping with a pixel size of 300x300, and finally scale to 50x50 as input. Finally, the one-hot encoding output by the output module is converted into six concentration corresponding probabilities through the softmax function, and the final concentration is obtained by calculating the concentration expectation, as follows:

[0051]

[0052] wherein Ci represents the concentration value corresponding to the i-th bit of the one-hot encoding, and Pi represents the probability that the concentration of the to-be-detected substance is Ci.

[0053] The encoder in the output module of the embodiment adopts a full connection design method, which is a special full connection design method that maps the feature vector extracted by the convolution kernel into a single value and then into a one-hot encoding, and the loss function adopts a CCE loss function. The full connection design method is to map the feature vector extracted by the convolution kernel into a single value and then into a one-hot encoding.

[0054] The target detection module and the output module of the embodiment both use an Adam optimizer.

[0055] In the embodiment, as shown in Figure 6 The influence of the Dropout layer deactivation probability and whether to encode into a single value on the network performance is explored, and the network performance is represented by the cross-validation method. Figure 6 (a) shows the relationship between the Dropout layer probability and the number of encoded values of the encoding-decoding network and the accuracy; Figure 6 (b) shows that when p is selected as 0.95, the accuracy reaches the maximum value, which also shows that the Dropout layer significantly improves the network performance. For the number of encoded values, it can be seen that when the value is 1, the network performance is best; Figure 7 shows the change process of the model loss with the number of training times when all data is used as the training set to obtain the final model parameters; Figure 8The recall and accuracy of the YOLOv5 module over the training period are shown. Figure 9 The localization loss over the training period on the training set and the test set is shown. The results show that the output module has the advantages of high accuracy, and the target detection module has the advantages of high recall, high accuracy, and high localization precision, and the two modules can well complete the target task.

[0056] In this embodiment, the method is used to analyze the test immunochromatography test strip picture and label the concentration and confidence on the picture. Figure 10 (a) is a test picture, Figure 10 (b) is a processed picture. It can be seen that the method can accurately identify the target position of the T line and correctly analyze the target concentration, and can analyze multiple targets in the picture at one time, improving the analysis efficiency.

[0057] The image detection method based on the convolutional neural network of the application is designed around the characteristics of the relationship between the T line RGB value of the immunochromatography test strip image and the concentration of the measured object. The T line region is extracted by the target detection module, the extracted picture is processed by the cropping module, and finally the concentration information is output by the convolutional network. This method reduces the influence of useless information in the picture on network training and operation, so that the network can also have high performance in the case of less data, and realizes convenient and efficient analysis of the concentration of the measured object in the immunochromatography test strip.

[0058] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0059] The present application is described with reference to flowcharts and block diagrams of the method and computer program product according to the embodiments of the present application. It should be understood that each flow in the flowchart can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a means for implementing the functions specified in the flowchart Figure 1 The apparatus specified in one flow or multiple flows.

[0060] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flow Figure 1 specified in the flow or flows.

[0061] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 specified in the flow or flows.

Claims

1. An immunochromatographic image analysis method characterized by, The method comprises the following steps: S1: acquiring an image of an immune chromatography strip to be detected; S2: detecting the image of the immune chromatography strip to be detected by using a trained target detection module, and locating a local area where a T line of the immune chromatography strip image is located in the image of the immune chromatography strip; S3: using the constructed cutting module, cutting the local region where the T line of the positioned immune chromatography strip image in the image of the immune chromatography strip to be detected to obtain a cutting image with an aspect ratio of , wherein , the obtained cutting image is scaled according to an aspect ratio of , the width of the cutting image is scaled to pixels, wherein , the length of the cutting image is , the cutting image with the scaled width and height is center-cut to retain a region with a pixel size of , wherein , the obtained region with a pixel size of is scaled to a region with a pixel size of , wherein , a cutting paper strip image is obtained. S4: extracting features of the cropped strip image by using an output module, encoding the extracted features, and decoding the encoded features into one-hot encoding of the concentration, calculating the output one-hot encoding to obtain the concentration represented by the T line in the immune chromatography strip; the output module comprises: a feature extractor composed of a convolutional layer and a pooling layer, used for extracting image features; an encoder composed of a fully connected layer and a Dropout layer, used for encoding the features into a single value, wherein the Dropout layer has a deactivation probability of 95%; a decoder composed of a fully connected layer, used for decoding the single value output by the encoder into one-hot encoding of the concentration; and a softmax function is used to convert the one-hot encoding output by the output module into probabilities corresponding to six concentrations, and the final concentration is obtained by calculating the expected concentration.

2. The immunochromatographic image analysis method according to claim 1, characterized by: The target detection module comprises a target detection algorithm, and the target detection algorithm is a YOLOv5 target detection algorithm network.

3. The method of image analysis of an immunochromatographic test according to claim 2, characterized in that: The training set of the YOLOv5 target detection algorithm network during training is an immune chromatography strip image labeled by a person, and the immune chromatography strip image is processed by using a data enhancement mechanism.

4. The immuno-chromatographic image analysis method according to claim 3, characterized in that: The data enhancement mechanism includes: random rotation, random cropping and up-down or left-right flipping, the rotation angle range of random rotation is set to -20°-20°, the cropping ratio of random cropping should be the same as the cropping module, that is, first scale the width of the cropped image to pixels, wherein at this time the length of the cropped image is , the cropped image with the scaled width and height is center cropped, and the area with a picture center pixel size of is reserved, wherein the obtained area with a pixel size of is reduced to an area with a pixel size of , wherein .

5. The method of claim 1, wherein: The calculation of the output one-hot encoding to obtain the concentration represented by the T line in the immune chromatography strip is specifically: ； where C i represents the concentration value corresponding to the representation of the one-hot encoding of the i-th bit, P i represents the probability that the concentration of the substance to be detected is C i .

6. The method of claim 1, wherein: The encoder in the output module adopts a full connection design method, which is to map the feature vector extracted by the convolution kernel into a single value and then into one-hot encoding.

7. The method of image analysis of an immunochromatographic test according to claim 6, characterized in that: The loss function of the output module adopts a CCE loss function.

8. The method of claim 1, wherein: The target detection module and the output module both use an Adam optimizer.

9. A storage medium characterized by: The storage medium stores a computer program, and the computer program is executed by the processor to implement the immune chromatography image analysis method according to any one of claims 1 to 8.

Images