A color recognition method for immunochromatographic test paper

Abstract

The present invention discloses a method for color recognition of immunochromatographic test paper: a group of raw milk test information table images are collected, and five kinds of double colloidal gold test papers on each test information table are intercepted as the data set to be tested; the C2f module of the YOLOv8 backbone network is replaced with an irregular feature extraction module, and a position information attention module is added to the tail of the backbone network to construct a test paper color recognition model; the test paper color recognition model is trained based on an improved WIoU loss function and a custom sample set containing different color development conditions; the data set to be tested is input into the trained test paper color recognition model for recognition, and multiple prediction frames are obtained; the regional color recognition result is determined by comparing the projection size of the accumulated pixel values ​​on the same ordinate of multiple prediction frames. The present invention applies target detection technology to the task of color recognition of immunochromatographic test paper, and provides new ideas and solutions for batch immunochromatographic test paper color recognition.

Description

Technical Field

[0001] The present invention relates to the field of deep learning, and relates to but is not limited to an immunochromatographic test paper color development and recognition method. Background Art

[0002] Immunochromatographic test strips are widely used in the field of food safety because they are fast, economical and easy to use. In the interpretation of the color development results of the chromatographic test strips, due to the phenomenon of weak reagent reaction (usually refers to the phenomenon that in a chemical reaction, the reaction effect of the reagent is not obvious or the reaction rate is slow), it is easy to cause the color development area to be missed, resulting in a low accuracy rate in the recognition of the color development results of the chromatographic test strips, which makes it difficult to meet the detection needs of the samples.

[0003] Immunochromatographic test strips are time-sensitive. Traditional immunochromatographic test strips rely mainly on manual interpretation. However, manual testing is time-consuming and not suitable for rapid testing of large-scale samples. In addition, differences in experience and vision of different testers can lead to inconsistent results, which in turn affects the accuracy of the test. Therefore, there is an urgent need for an automated recognition method for immunochromatographic test strips. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method for colorimetric identification of an immunochromatographic test paper, which at least solves the problem of low accuracy of colorimetric identification of an immunochromatographic test paper due to weak reaction of reagents.

[0005] The technical solution of the embodiment of the present invention is achieved as follows:

[0006] The embodiment of the present invention provides a method for color development and recognition of immunochromatographic test paper, the method comprising:

[0007] A group of raw milk test information table images are collected, and five types of double colloidal gold test strips on each test information table are intercepted as the data set to be tested; the C2f module of the YOLOv8 backbone network is replaced with an irregular feature extraction module, and a position information attention module is added to the tail of the backbone network to construct a test strip color recognition model; wherein the irregular feature extraction module is used to extract the features of the detection line in the test strip image through variable convolution; the position information attention module enhances the model's recognition ability of the position features of the color development area of ​​the chromatography test strip by introducing a multi-head self-attention mechanism; the test strip color recognition model is trained based on an improved WIoU loss function and a custom sample set containing different color development conditions; wherein the WIoU loss function adjusts the weights of positive and negative samples by dynamically adjusting the focusing coefficient; the data set to be tested is input into the trained test strip color recognition model for recognition to obtain multiple prediction frames; and the regional color recognition result is determined by comparing the projection sizes of the accumulated pixel values ​​on the same ordinate of the multiple prediction frames.

[0008] Furthermore, the calculation formula of the variability convolution in the irregular feature extraction module is as follows:

[0009] ;

[0010] In the formula, is the output pixel value corresponding to each position in the feature map, is the weight in the convolution kernel, represents the center position of the convolution kernel, for Spread out to form a set of positions in a 3×3 rectangular area. is any position in the position set, It is the offset between the output feature map and the original feature map. is the sampling point weight.

[0011] Furthermore, the position information attention module uses a multi-head self-attention mechanism to process the output of the improved spatial pyramid pooling SPPF module ,

[0012] in Indicates that the SPPF module outputs n The features on the channels are N channel features After N self-attention modules are processed, N new feature maps are obtained. Finally, the N new feature maps are merged to output the feature vector Then it is passed to the fully connected layer for target classification and position regression.

[0013] Furthermore, the calculation process of the multi-head self-attention mechanism is as follows:

[0014] ;

[0015] ;

[0016] ;

[0017] ;

[0018] ;

[0019] ;

[0020] In the formula, is the query vector; Express Calculate the 2D convolution weights for Q values; is the key vector; Express Calculate the two-dimensional convolution weights of K value; is a value vector; Express Calculate the two-dimensional convolution weight of V value; Calculate the function for the self-attention mechanism; softmax ( ) is the activation function; for Dimensions; For the n The output of a self-attention head; For connection operation.

[0021] Furthermore, the improved WIoU loss function is defined by the following formula:

[0022] ;

[0023] ;

[0024] ;

[0025] ;

[0026] in, represents the improved WIoU loss function, IoU is the intersection-over-union ratio between the prediction box and the target box, ( x, y ) is the center coordinate of the prediction box, ( x gt , y gt ) is the target box center coordinate, w、h are the width and height of the minimum enclosed area that contains both the predicted box and the true box, r is the nonmonotonic clustering coefficient, α and δ is a hyperparameter, β is the outlier estimate of the prior box, L IoU represents the loss function based on intersection-over-union ratio, for L IoU The average value of express L IoU separation value.

[0027] Furthermore, the regional color recognition result is determined by comparing the projection sizes of the accumulated pixel values ​​on the same vertical coordinate of the multiple prediction boxes, including: calculating the sum of the pixel projections on the vertical coordinates in each prediction box; determining the minimum vertical coordinate projection of each prediction box and comparing them; when the minimum vertical coordinate projection of the prediction box in the top comparison area is smaller than the minimum vertical coordinate projection of the prediction box in other areas, determining that the test paper detection result is positive; when the minimum vertical coordinate projection of the prediction box in the top comparison area is larger than the minimum vertical coordinate projection of the prediction box in other areas, determining that the test paper detection result is negative.

[0028] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0029] In the embodiment of the present invention, a data set is constructed on the immunochromatographic test strip color recognition task, and the target detection technology is applied to the test strip recognition field. In view of the shape characteristics of the reagent weak reaction phenomenon, the C2f module of the YOLOv8 backbone network is replaced with an irregular feature extraction module, and in view of the position characteristics of the color development area, a position information attention module is designed and added to the tail of the backbone network. At the same time, the loss function is improved, and the coordinate projection method is used to identify the test strip color development results. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0031] Figure 1 Schematic diagram of the results of immunochromatographic test strip color recognition in different scenarios using the conventional image processing method provided in an embodiment of the present invention; wherein (1) the original image, (2) grayscale, (3) the result of the average threshold method, (4) the result of the Gaussian threshold method, and (5) the result of the global threshold method;

[0032] Figure 2 A schematic diagram of the results of color recognition of immunochromatographic test strips based on the YOLOv8 model provided in an embodiment of the present invention;

[0033] Figure 3 A schematic diagram of the process of the immunochromatographic test paper color recognition method provided in an embodiment of the present invention;

[0034] Figure 4 A schematic diagram of the network architecture of a test paper color recognition model based on YOLOv8 provided in an embodiment of the present invention;

[0035] Figure 5A schematic diagram of convolution effects of different convolution modes provided in an embodiment of the present invention; wherein part a corresponds to ordinary convolution, and part b corresponds to variable convolution;

[0036] Figure 6 A schematic diagram of size parameters of a prediction box and a target box of an object to be detected provided by an embodiment of the present invention;

[0037] Figure 7 A curve showing a change in the sum of pixel projections of a prediction frame on a test paper image in a longitudinal coordinate provided by an embodiment of the present invention;

[0038] Figure 8 Schematic diagram of a normal color test paper image and a reagent weak reaction color image provided in an embodiment of the present invention; wherein part a corresponds to a normal phenomenon, and part b corresponds to a reagent weak reaction phenomenon;

[0039] Fig. 9 A schematic diagram showing a comparison of characteristic graphs before and after the improvement of the C2f module provided in an embodiment of the present invention;

[0040] Fig.10 A schematic diagram of the loss convergence effect after adding the MHSA module and the improved WIoU loss function to YOLOv8 provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0042] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0043] It should be pointed out that the terms "first\second\third" involved in the embodiments of the present invention are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present invention described here can be implemented in an order other than that illustrated or described here.

[0044] Those skilled in the art will appreciate that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the field to which the embodiments of the present invention belong. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0045] Smartphone-based immunochromatographic test strip colorimetric recognition is an emerging technology that provides a wide range of application value in instant testing. Smartphone-based immunochromatographic test strip recognition is divided into the following methods: using optical devices to read data and using smartphones to transmit results, and using smartphone cameras to take photos and retain them, and then using computer vision technology to obtain results. The former uses optical sensor devices to read fixed positions on the test strips, which has high accuracy, but cannot be used for batch recognition and requires strict prior conditions. In order to better balance the speed and accuracy of chromatographic test strip colorimetric recognition, researchers mostly use image analysis to determine the chromatographic test strip colorimetric results.

[0046] In one related technology, for drug driving saliva test images, the images are preprocessed to eliminate complex backgrounds and then principal component analysis is performed, and then a support vector machine is used to classify the saliva test images. In another related technology, a regional growing algorithm combined with fast peak detection is used to quantitatively analyze the test lines and quality control lines of colloidal gold immunochromatographic test strip images. In other related technologies, image processing technology is used to extract and analyze the pigments on the immunochromatographic test strips, the test results are quantitatively analyzed, and the performance differences of different machine learning algorithms are compared. However, there are some challenges when traditional computer vision technology is applied to detect reaction areas, especially when the reagent reaction is not strong enough, the color of the color-developed area is lighter and similar to the background color, it is difficult to effectively distinguish the color-developed area from the background, resulting in missed detection.

[0047] In addition, affected by camera parameters and ambient light conditions, traditional computer vision technology requires the development of specific algorithm programs for different application scenarios. Even for the same algorithm program, different threshold parameters need to be set in different scenarios to ensure the stability of the detection effect. This process often relies on manual adjustment of parameters, but in complex and changeable practical applications, it is impractical to adjust parameters one by one for each situation. Figure 1 As shown, the original image can be grayscaled, or average threshold, Gaussian threshold, or global threshold processing can be performed according to the application scenario, which limits the traditional image processing methods in large-scale applications of immunochromatographic test strip color recognition.

[0048] Object detection technology provides a new approach to solving these problems. Taking the YOLO model as an example, by annotating the collected images and using them for network training, the model can detect objects in various complex scenes through a large number of parameters obtained through training. The detection results are as follows: Figure 2 As shown. Although the accuracy of the YOLO model still has room for improvement, it has shown great potential and feasibility in detecting color areas. The detection of color areas of chromatographic test strips under the phenomenon of weak reagent reaction belongs to the task of non-significant target detection. This type of task mainly optimizes the backbone network in the target detection model by combining the characteristics of its own scene, and enhances the ability to extract non-significant target features by improving the network structure. In a related technology, the detection area of ​​the microfluidic paper-based analytical device (μPAD) is identified based on the YOLOv3 target detection algorithm, which realizes fast, convenient and close to end-to-end detection, and effectively reduces the problem of missed detection caused by the weak reaction phenomenon of μPAD. In another related technology, a feature conversion module embedded with an attention mechanism is proposed for non-significant industrial defect detection to enhance the extraction of the maximum scale feature map. In another related technology, a gray value curve enhancement module and a model specifically used for weld defect detection are proposed for the low contrast of weld images, so that the model can better distinguish between foreground and background areas.

[0049] In the task of non-significant target detection, it is necessary to design a model that best meets the target characteristics according to the task scenario. Based on the above considerations, the present invention constructs a data set on the immunochromatographic test strip color recognition task and applies the target detection technology to the field of test strip recognition. In view of the shape characteristics of the weak reaction phenomenon of the reagent, the C2f module of the YOLOv8 backbone network is replaced with an irregular feature extraction module. In view of the position characteristics of the color development area, a position information attention module is designed and added to the tail of the backbone network. At the same time, the loss function is improved, and the coordinate projection method is used to identify the test strip color development results.

[0050] Figure 3 A schematic diagram of a flow chart of a method for color recognition of an immunochromatographic test paper provided in an embodiment of the present invention, such as Figure 3 As shown, the method comprises at least the following steps:

[0051] Step S110, collecting a group of raw milk test information table images, and intercepting five types of double colloidal gold test papers on each test information table as the data set to be tested.

[0052] Here, we collected images of the raw milk testing information table uploaded by the ranch to be tested within a fixed period of time, and selected five double colloidal gold test papers on the testing information table, including aflatoxin M1, acetaminophen, lincosamides, meloxicam, and clavulanic acid, and cut them into new images for the experiment.

[0053] Step S120, replace the C2f module of the YOLOv8 backbone network with an irregular feature extraction module, and add a position information attention module to the end of the backbone network to construct a test paper color recognition model.

[0054] Here, the irregular feature extraction module is used to extract the features of the detection line in the test paper image through variable convolution; the position information attention module enhances the model's recognition ability of the position features of the color development area of ​​the tomographic test paper by introducing a multi-head self-attention mechanism.

[0055] Due to the weak reaction of the reagent, the features of the detection line in the image are not obvious and present an irregular shape. The C2f module does not fully learn the features of the detection line in the image due to the limitation of the convolution method. Therefore, the embodiment of the present invention replaces the convolution method in the C2f module with a variable convolution. Due to its unique convolution method, irregular features can be extracted well.

[0056] Since the relative position of the area to be detected in different types of test strip images is relatively fixed, the current model only focuses on the area to be detected, resulting in that its positional features relative to the image are not fully utilized. This neglect of position information is also a factor that affects model performance. In order to solve the above problem, the embodiment of the present invention introduces a position information attention module (Multi-Head Self-Attention, MHSA) to enable the model to consider the relationship between different positions, capture richer feature representations, and fully understand image features using contextual information.

[0057] In the MHSA module, the input sequence is linearly transformed into query (Q), key (K), and value (V) matrices, which are then fed into multiple parallel self-attention layers. Each self-attention layer computes the output of a set of heads, which are subsequently concatenated together and transformed through another linear layer to produce the final multi-head self-attention output.

[0058] Since the linear self-attention mechanism in MHSA is too expensive for image calculation, the linear transformation of Q, K, and V in the self-attention mechanism is modified to perform two-dimensional convolution on them separately. This is because two-dimensional convolution usually has higher parameter efficiency than linear transformation. It can improve model performance while using fewer parameters and reduce model complexity. In addition, two-dimensional convolution has a natural advantage in image data processing. It can better capture the spatial and local features in the image, allowing the model to better utilize spatial and channel information, further enhancing its contextual understanding ability and improving the model's ability to detect unclear regional features under weak reagent reaction conditions.

[0059] Step S130, training the test paper color recognition model based on the improved WIoU loss function and a custom sample set containing different color conditions.

[0060] Here, the WIoU (i.e. Wise-IoU) loss function dynamically adjusts the focus coefficient and the weights of positive and negative samples. For example, according to the characteristics of the task, it reduces the penalty for low-quality labeled samples and increases the attention to medium-quality samples, so as to better guide the model to learn the characteristics of the colored area and enhance the generalization ability of the model.

[0061] Before model training, we first collect a large number of images of immunochromatographic test strips to ensure that the images contain a variety of different color development conditions, including weak reaction phenomena. We preprocess the images, such as resizing, normalizing brightness and contrast, to improve the generalization ability of the model. We accurately annotate the color development areas in the images, including features such as the location, shape, and color of the color development areas, to construct a custom sample set. We then use this custom sample set to train the model and evaluate the performance of the model on the validation set.

[0062] Step S140, inputting the to-be-tested data set into the trained test paper color recognition model for recognition to obtain a plurality of prediction frames.

[0063] Here, the trained test paper color recognition model is used to recognize the test data set, i.e., the actual test paper color image, to obtain multiple prediction boxes (Bounding Boxes), thereby realizing the positioning of the area to be detected based on the target detection technology.

[0064] Step S150, determining the regional color recognition result by comparing the projection sizes of the accumulated pixel values ​​on the same vertical coordinate of the multiple prediction boxes.

[0065] Here, the coordinate projection method is used to solve the problem of quantitatively representing the color depth of the area to be tested. Due to the characteristics of the test paper's color development, the area to be tested often shows a certain extension in the vertical direction. Therefore, by comparing the projection sizes of different areas on the ordinate, that is, accumulating each row of pixels in the prediction box in the y direction to obtain a projection value, the different degrees of color development results can be inferred.

[0066] In the embodiment of the present invention, a data set is constructed on the immunochromatographic test strip color recognition task, and the target detection technology is applied to the test strip recognition field. In view of the shape characteristics of the reagent weak reaction phenomenon, the C2f module of the YOLOv8 backbone network is replaced with an irregular feature extraction module, and in view of the position characteristics of the color development area, a position information attention module is designed and added to the tail of the backbone network. At the same time, the loss function is improved, and the coordinate projection method is used to identify the test strip color development results.

[0067] In some possible embodiments, the calculation formula of the variability convolution in the irregular feature extraction module is as follows:

[0068] Formula (1);

[0069] In the formula, is the output pixel value corresponding to each position in the feature map, is the weight in the convolution kernel, represents the center position of the convolution kernel, for Spread out to form a set of positions in a 3×3 rectangular area. is any position in the position set, is the offset between the output feature map and the original feature map, is the sampling point weight.

[0070] It should be noted that the 3×3 convolution kernel of the ordinary convolution slides from the input feature map with a fixed step size and performs a convolution operation on each position. Each output pixel value is by adding the weights in the convolution kernel Corresponding position in the input feature map The value of is calculated as shown in formula (2): Formula (2);

[0071] in represents the center position of the convolution kernel, yes Spread out to form a 3×3 rectangular area. .

[0072] like Figure 5 As shown in Figure 1, the variable convolution does not change the way the convolution is calculated. It still needs to sample 9 positions from the convolution kernel. The difference is that these 9 positions are the original convolution kernel plus It allows the convolution kernel to change to an irregular shape and adds sampling point weights To learn , and define a 3×3 convolution layer. This convolution layer is calculated with the input feature map. The output feature map is the size of the original feature map, but the number of channels is twice the original, which is used to represent the horizontal and vertical offsets respectively. That is, it is calculated by the offset between the output feature map and the original feature map. is the learned weight. The advantage of this is that for some unreasonable sampling points, their weights can be learned to be 0, thereby avoiding affecting the results. These parameters are learned through back propagation.

[0073] In the above embodiment, according to the shape characteristics of the weak reaction phenomenon, an irregular feature extraction module is designed to replace the C2f module in the YOLOv8 backbone network. This module should be able to capture the subtle color changes on the test paper.

[0074] In some possible embodiments, the position information attention module uses a multi-head self-attention mechanism to process the output of the improved spatial pyramid pooling SPPF module. ,in Indicates that the SPPF module outputs n The features on the channels are N channel features After N self-attention modules are processed, N new feature maps are obtained. Finally, the N new feature maps are merged to output the feature vector Then it is passed to the fully connected layer for target classification and position regression.

[0075] In the above embodiment, a position information attention module is designed to connect to the tail of the backbone network according to the position features of the color display area. This module can enhance the model's ability to recognize the position of the color display area.

[0076] In some possible embodiments, the calculation process of the multi-head self-attention mechanism is as follows:

[0077] Formula (3);

[0078] Formula (4);

[0079] Formula (5);

[0080] Formula (7);

[0081] Formula (8);

[0082] Formula (9);

[0083] In the formula, is the query vector; Express Calculate the 2D convolution weights for Q values; is the key vector; Express Calculate the two-dimensional convolution weights of K value; is a value vector; Express Calculate the 2D convolution weights of the V value; Calculate the function for the self-attention mechanism; softmax ( ) is the activation function; for Dimensions; For the n The output of a self-attention head; For connection operation.

[0084] In some possible embodiments, the improved WIoU loss function is defined by the following formula:

[0085] Formula (10);

[0086] Formula (11);

[0087] Formula (12);

[0088] Formula (13);

[0089] in, represents the improved WIoU loss function, IoU is the intersection-and-union ratio between the predicted box and the target box (i.e., the real box), such as Figure 6 As shown, ( x, y ) is the center coordinate of the prediction box (Anchor Boxes), ( x gt , y gt ) is the target box center coordinate, w、h are the width and height of the minimum enclosed area that contains both the predicted box and the true box, r is the nonmonotonic clustering coefficient, α and δ is a hyperparameter, β is the outlier estimate of the prior box, L IoU represents the loss function based on intersection-over-union ratio, for L IoU The average value of express L IoU separation value.

[0090] It should be noted that YOLOv8 uses CIoU (Complete-IoU) to complete the calculation of the regression loss of the anchor box of the target to be detected. The specific formula of CIoU is as follows:

[0091] Formula (14);

[0092] Among them v Describes aspect ratio consistency:

[0093] Formula (15);

[0094] in, l It represents the diagonal distance of the minimum enclosing area that can contain both the prediction box and the target box. is the intermediate variable, v Indicates aspect ratio consistency ，w g , h g are the width and height of the target box, respectively. w p , h p are the width and height of the prediction box respectively.

[0095] For scenes with non-significant small targets such as the test strip to be detected area, the data annotation process will inevitably include some low-quality annotations. The distance and aspect ratio of the prediction box and target box used in calculating the loss function will increase the penalty for these low-quality annotations and reduce the generalization of the model. A good loss function needs to be able to weaken the penalty of these factors in order to bring better model performance to the model. Based on this idea, WIoU (i.e. wise iouv2) is used to decouple the two categories of distance and aspect ratio. WIoU introduces a dynamic non-monotonic focusing mechanism, which can dynamically adjust the weights of positive and negative samples through the IoU values ​​of the prediction box and the ground truth box. Specifically, for prediction boxes with lower IoU, WIoU will give them higher weights, so that the model pays more attention to these samples during training.

[0096] In some possible embodiments, the regional color recognition result is determined by comparing the projection sizes of the accumulated pixel values ​​on the same vertical coordinate of the multiple prediction boxes, including: calculating the sum of the pixel projections on the vertical coordinates in each prediction box; determining the minimum vertical coordinate projection of each prediction box and comparing them; when the minimum vertical coordinate projection of the prediction box in the top comparison area is smaller than the minimum vertical coordinate projection of the prediction box in other areas, determining that the test paper detection result is positive; when the minimum vertical coordinate projection of the prediction box in the top comparison area is larger than the minimum vertical coordinate projection of the prediction box in other areas, determining that the test paper detection result is negative.

[0097] Here, given a test paper image, there are N prediction boxes in the test paper ,assumed is the predicted box of the top comparison area. Assume that the coordinates of the upper left corner of the rectangle corresponding to one of the predicted boxes are , the coordinates of the lower right corner are , the pixel value in the rectangle can be expressed as V(x i ,yj ),in , the projection set of the prediction box R1 on each vertical coordinate is expressed as:

[0098] ;

[0099] Among them, assuming that the minimum value in the projection set is for ,

[0100] Similarly, N prediction boxes Get the minimum value of the vertical coordinate projection of N rectangles ,Will Compared with other area values, if If the test result is positive, If the test result is negative, Figure 7 shown.

[0101] The above-mentioned immunochromatographic test paper color development and recognition method is described below in conjunction with a specific embodiment. However, it should be noted that this specific embodiment is only for better illustrating the present invention and does not constitute an improper limitation to the present invention.

[0102] The embodiment of the present invention proposes an immunochromatographic test paper color recognition method based on YOLOv8 and coordinate projection method, which is implemented through the following process:

[0103] S1, build a custom sample set.

[0104] Since there is no public test strip dataset, in order to prove the effectiveness of the improved algorithm, the embodiment of the present invention uses a self-built custom sample set to experimentally verify the improved algorithm, and uses the LabelImg tool for annotation. The sample set uses 5000 test strip images cropped by target detection technology, where the ratio of training set to validation set is 4:1. There are a large number of weak reaction instances of reagents in the sample set, such as Figure 8 As shown, examples of weak reagent reactions are marked with triangles. The custom sample set constructed by the present invention is a good data set that can ensure the diversity and practicality of model training. In some embodiments, the sample data is preprocessed, such as resizing, normalizing brightness and contrast, etc., to improve the generalization ability of the model. In some embodiments, the color-developed areas in the image are accurately annotated, including features such as the position, shape and color of the color-developed areas.

[0105] S2, build a test strip color recognition model based on the YOLOv8 network.

[0106] Here, according to the shape characteristics of the weak reaction phenomenon, an irregular feature extraction module is designed to replace the C2f module in the YOLOv8 backbone network. This module can better capture the subtle color changes on the test paper. At the same time, according to the position characteristics of the color area, a position information attention module is designed, which can enhance the model's ability to recognize the position of the color area.

[0107] The test strip color recognition model is as follows Figure 4 As shown, it includes a backbone network, a neck network and a head network. The irregular feature extraction module in the backbone network receives the input test paper image, extracts the features of the detection lines in the test paper image through variable convolution, and then inputs them into the position information attention module to capture richer position feature representations and enhance the model's perception of the position information of the test paper color display area.

[0108] S3, model training and validation.

[0109] Here, the model is trained using the constructed training set, and the performance of the model is evaluated on the validation set. The experimental environment configuration and experimental parameter settings for deep learning are shown in Table 1 and Table 2 respectively:

[0110] Table 1 Environmental configuration

[0111]

[0112] Table 2 Experimental parameters

[0113]

[0114] In order to objectively evaluate the improved YOLOv8 algorithm, the embodiment of the present invention uses evaluation indicators such as precision, recall, mean average precision (mAP), model size (using Giga Floating-point Operations Per Second (GFLOPs) to describe the computational size of the model) to measure the algorithm performance, as shown in detail:

[0115] Formula (16);

[0116] Formula (17);

[0117] Formula (18);

[0118] Formula (19);

[0119] Wherein, N represents the number of categories; TP is the number of correctly predicted positive samples, FN is the number of incorrectly predicted negative samples, and FP is the number of incorrectly predicted categories and positive samples; Precision represents the accuracy rate, which reflects the model's misdetection situation, and the value range is [0,1]. The closer to 1, the less actual background there is in the detection line detected by the model. AP represents the average precision, which reflects the balance of the model between precision and recall. The value range is [0,1]. The closer to 1, the better the balance between precision and recall, and the better the performance. The Precision-Recall curve is drawn using precision and recall as the horizontal and vertical coordinates, and the area under the curve is the AP value. mAP represents the mean average precision of all categories, which is calculated by adding the AP values ​​of different categories to find the average. In the embodiment of the present invention, there is only one type of object to be detected, the test line, so mAP is AP, and the value range is also [0,1]. The closer to 1, the better the model performance.

[0120] In order to verify the superiority and accuracy of the method of the embodiment of the present invention, the DCN_C2f module, the MHSA module, and the WIoU loss function, multiple groups of ablation experiments were designed on the self-built test paper dataset. YOLOv8n was used as the benchmark model, and the evaluation indicators of the network model were tested by gradually replacing the modules. The ablation experiment results are shown in Table 3:

[0121] Table 3 Ablation experiment

[0122]

[0123] Comparing the evaluation indicators of YOLOv8 and YOLOv8+DCN_C2f, it can be seen that after replacing DCN_C2f, the mAP is improved by three percentage points compared with the original YOLOv8 model, and the feature maps of the DCN_C2f and C2f modules are as follows: Fig. 9 As shown, it can be seen that the characteristic contour of the quality control line is clearer than before the improvement, indicating that in the test line detection of the test paper, DCN_C2f can more accurately extract the edge feature information of the detection line by offsetting the convolution kernel, and effectively improve the model's detection ability for the weak reaction test line area of ​​the test paper.

[0124] like Fig.10As shown in the figure, comparing the evaluation indicators of YOLOv8 and YOLOv8+MHSA, it can be seen that after adding the MHSA module, the mAP of the original YOLOv8 model is increased by 1 percentage point, indicating that it has a certain extraction ability for quality control line detection of weak reaction phenomena. Comparing the evaluation indicators and loss function diagrams of YOLOv8 and YOLOv8+WIoU, it can be seen that after changing to the WIoU loss function, the model has better convergence ability for boundary loss, and the mAP is increased by 1 percentage point, indicating that the WIoU loss function has good convergence ability, which enables the model to more accurately predict the position of the bounding box, thereby improving the overall accuracy of target detection.

[0125] In summary, the DM-YOLOv8 network structure proposed in the embodiment of the present invention combines the advantages of DCNv2, MHSA and WIoU in the test paper quality control line target detection, so that its precision, recall rate and mAP indicators on the validation set are higher, which are 91.0%, 91.2% and 91.1% respectively, proving the effectiveness of adding modules.

[0126] In order to verify the superiority of the test paper color recognition model (hereinafter referred to as DM-YOLOv8) constructed in the embodiment of the present invention, other algorithms of the same magnitude with excellent target detection performance, YOLOv5n, YOLOv6n, YOLOv7n, Gold-YOLOn, and YOLOv8n, are selected for comparative experiments. When the experimental data set, experimental hardware and software equipment, and hyperparameters remain the same, the accuracy, recall rate, and mAP@50 related experimental results of the above methods are statistically shown in Table 4:

[0127] Table 4 Performance comparison experimental results of different models

[0128]

[0129] It can be seen from the data in Table 4 that the complexity of the improved model is not greatly improved, but the effect is much better than YOLOv8n, and it is far ahead of other algorithms. In summary, the DM-YOLOv8 network structure proposed in the embodiment of the present invention combines the advantages of DCNv2, MHSA, and WIoU in the target detection in the test paper to be detected area, so that YOLOv8 has better mAP@50 and mAP@50:95 in the detection of the test paper to be detected area, which are 98.7% and 59.5%. Without significantly increasing the model parameters, the accuracy of the model is maximized, and the performance indicators are optimized.

[0130] In order to evaluate the color recognition performance of the test paper, the embodiment of the present invention selects the correct rate and error rate ,in T Represents the number of test strips with correct identification results. FThe number of test strips representing incorrect recognition results is calculated as follows:

[0131] Formula (20);

[0132] Formula (21);

[0133] The improved model is combined with the coordinate projection method for color recognition of chromatography test strips for comparative experiments, and the experimental results are shown in Table 5. The experimental results show that the quality of the target detection model also affects the effect of color recognition of chromatography test strips. The effect of DM-YOLOv8 on color recognition of chromatography test strips is better than that before the improvement.

[0134] Table 5 Comparison of recognition effects of different models

[0135]

[0136] The embodiment of the present invention collected data from more than 50 ranches of raw milk testing information tables, constructed an immunochromatographic test strip color recognition data set through target detection technology cutting, and improved the YOLOv8 target detection algorithm based on the characteristics of the weak reaction of the reagent. The results show that the mAP50 of the improved target detection algorithm DM-YOLOv8 can reach 92.2%, and the recognition accuracy can reach 95%. Applying target detection technology to the task of immunochromatographic test strip color recognition provides a new idea and solution for batch immunochromatographic test strip color recognition.

[0137] In an embodiment of the present invention, an irregular feature extraction module is designed based on the C2f module to enhance the model's ability to extract irregular features of the color development area under the phenomenon of weak reagent reaction, and a position information attention module is added to the tail of the backbone network to enhance the model's perception of the position information of the test paper color development area. Then, the loss function is replaced with WIoU to enhance the model's fitting ability for non-significant small targets, and finally, the coordinate projection method is combined to identify the test paper color development results.

[0138] It should be noted that in the embodiment of the present invention, if the above-mentioned immunochromatographic test paper color recognition method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions for enabling an electronic device to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.

[0139] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0140] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0141] In the several embodiments provided by the present invention, it should be understood that the disclosed methods can be implemented in other ways. The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments. The features disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0142] The above is only an embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An immunochromatographic test paper color recognition method, characterized in that: include: Collect a group of raw milk test information table images, and intercept five types of double colloidal gold test papers on each test information table as the data set to be tested; The C2f module of the YOLOv8 backbone network is replaced with an irregular feature extraction module, and a position information attention module is added to the tail of the backbone network to construct a test strip color recognition model; wherein the irregular feature extraction module is used to extract the features of the detection line in the test strip image through variable convolution; the position information attention module enhances the model's recognition ability of the position features of the color development area of ​​the tomographic test strip by introducing a multi-head self-attention mechanism; The test paper color recognition model is trained based on an improved WIoU loss function and a custom sample set containing different color conditions; wherein the WIoU loss function adjusts the weights of positive and negative samples by dynamically adjusting the focusing coefficient; Inputting the to-be-tested data set into the trained test paper color recognition model for recognition to obtain a plurality of prediction frames; Calculate the sum of pixel projections corresponding to all horizontal coordinates on each vertical coordinate in each prediction frame and use it as a vertical coordinate projection value list of the corresponding prediction frame; Determine and compare the minimum ordinate projection values ​​in the ordinate projection value lists of the multiple prediction boxes; When the minimum value of the vertical coordinate projection of the prediction box of the top comparison area is greater than the minimum value of the vertical coordinate projection of the prediction box of other areas, the test paper test result is determined to be positive; When the minimum value of the vertical coordinate projection of the prediction box of the top comparison area is less than the minimum value of the vertical coordinate projection of the prediction box of other areas, it is determined that the test paper test result is negative.

2. The method according to claim 1, characterized in that The calculation formula of the variability convolution in the irregular feature extraction module is as follows: ； In the formula, is the output pixel value corresponding to each position in the feature map, is the weight in the convolution kernel, represents the center position of the convolution kernel, for Spread out to form a set of positions in a 3×3 rectangular area. is any position in the position set, is the offset between the output feature map and the original feature map, is the sampling point weight.

3. The method according to claim 1, characterized in that The position information attention module uses a multi-head self-attention mechanism to process the output of the improved spatial pyramid pooling SPPF module ,in Indicates that the SPPF module outputs n The features on the channels are N channel features After N self-attention modules are processed, N new feature maps are obtained. Finally, the N new feature maps are merged to output the feature vector Then it is passed to the fully connected layer for target classification and position regression.

4. The method according to claim 3, characterized in that The calculation process of the multi-head self-attention mechanism is as follows: ； ； ； ； ； ； In the formula, is the query vector; Express Calculate the 2D convolution weights for Q values; is the key vector; Express Calculate the two-dimensional convolution weights of K value; is a value vector; Express Calculate the two-dimensional convolution weight of V value; Calculate the function for the self-attention mechanism; softmax ( ) is the activation function; for Dimensions; For the n The output of a self-attention head; For connection operation.

5. The method according to any one of claims 1 to 4, characterized in that: The improved WIoU loss function is defined by the following formula: ； ； ； ； in, represents the improved WIoU loss function, IoU is the intersection-over-union ratio between the prediction box and the target box, ( x, y ) is the center coordinate of the prediction box, ( x gt , y gt ) is the target box center coordinate, w、h are the width and height of the minimum enclosed area that contains both the predicted box and the true box, r is the nonmonotonic clustering coefficient, α and δ is a hyperparameter, β is the outlier estimate of the prior box, L IoU represents the loss function based on intersection-over-union ratio, for L IoU The average value of express L IoU separation value.

Images