Fluorescence detection test paper visualization system and method based on deep learning

Abstract

The invention relates to the technical field of water quality detection, and provides a fluorescence detection test paper visualization system and method based on deep learning, and the method comprises the steps: detecting a fluorescence detection test paper image obtained through shooting through a target detection network, and obtaining a test paper main body region, a colorimetric block region and a test paper key angular point coordinate; performing homographic perspective transformation on the main body area of the test paper according to the key angular point coordinates of the test paper, generating a test paper standard view after perspective correction, performing color space standardization correction, generating a mixed deep learning model, and outputting a fluorescence intensity concentration predicted value of the target detection area; generating an interpretable thermodynamic diagram based on the intermediate features of the mixed deep learning model; according to the method, shooting deviation is eliminated through multi-stage image correction, the fluorescence intensity is accurately quantified by fusing CNN and Transform models, the result credibility is improved in combination with a thermodynamic diagram, and the detection automation is comprehensively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of water quality detection, in particular to a fluorescent detection test paper visualization system and method based on deep learning. BACKGROUND

[0003] The fluorescent detection test paper has become one of the important tools for rapid detection outside the laboratory due to its simple operation, low cost, and portability. The fluorescent detection test paper detection mainly relies on loading specific fluorescent probes on the test paper. When the test paper contacts the water sample containing target ions, the visible change of fluorescence intensity or color is caused by the selective combination with metal ions. The user can qualitatively or semi-quantitatively judge the concentration of the measured substance by colorimetry with the naked eye or a portable colorimetric card.

[0004] The existing fluorescent detection test paper is widely used in on-site rapid detection of heavy metal ions, drug residues, etc. It usually relies on the user to compare the color or fluorescence intensity of the test paper with the pre-prepared standard color card to determine the concentration range of the measured substance. However, this detection method highly depends on the subjective judgment and experience level of the operator. Different personnel in different lighting conditions, shooting angles, and background environments often have large differences in the perception of color or fluorescence intensity, resulting in poor repeatability and consistency of the detection results. Experimental results in the prior art show that the concentration error of different observers in judging the color of the same test paper can reach ± 50%, especially in the low concentration range [2] .

[0005] In order to improve the objectivity of the naked eye colorimetry, some technical solutions in the industry try to introduce a mobile phone camera to capture the test paper image and compare it with a pre-set standard color reference image to assist in judgment, which to some extent reduces the influence of the operator's subjective factors and improves the consistency. However, due to the complex and variable on-site environment, the shooting process is easily affected by many factors such as environmental light intensity, light source color temperature, shooting angle, etc. The obtained image often appears tilted, distorted or color distorted, resulting in large fluctuations in the detection results. Experimental results show that the RGB value of the same object under different lighting conditions can fluctuate by more than 20%, which seriously affects the quantitative accuracy [3] In addition, the distribution of fluorescent signals on the test paper usually has gradient and local characteristics. Observation with a confocal microscope shows that the fluorescent probes are non-uniformly diffused on the cellulose paper, forming a gradient distribution of "bright center-weak edge", which is related to the ion concentration [4], traditional colorimetric cards or simple image comparison cannot fully extract and quantify such spatial distribution information, and traditional fluorescence detection mostly relies on simple threshold or single-point colorimetry, lacking the ability to deeply learn the spatial distribution characteristics of fluorescence signals on the test paper surface, thus lacking sensitivity to weak fluorescence signals in low concentration and complex background, limited quantitative precision, and difficult to visualize the detection basis, making it difficult for users and regulators to trace the credibility and accuracy of the detection process. At the same time, existing detection results are often isolated records, lacking binding with test paper numbers and real-time linkage with cloud databases, which is not conducive to batch management, data traceability and large-scale environmental data analysis in the later stage.

[0006] Therefore, it is necessary to design a fluorescence detection test paper visualization system and method based on deep learning to solve the technical problems of the prior art in the aspects of automation, quantitative ability, detection consistency, and result reliability.

[0007] The cited documents in the background art are:

[0008] [1] Wang, X. et al. (2021). Recent advances in optical sensors for copper ion detection in environmental water samples. TrAC Trends in Analytical Chemistry, 135, 116148.

[0009] [2] Zhang, H. et al. (2019). Evaluation of visual interpretation variability in lateral flow assays. Analytical Chemistry, 91(15), 9623-9630.

[0010] [3] Bianco, S. et al. (2015). How far can you go with uncalibrated smartphone colorimetry? IEEE Sensors Journal, 15(11), 6448-6456.

[0011] [4] Xu, W. et al. (2022). Spatially resolved analysis of paper-based fluorescent biosensors using deep learning. Lab on a Chip, 22(8), 1520-1531. SUMMARY

[0012] In view of this, the present application proposes a fluorescent test paper visualization system and method based on deep learning, aiming to solve the technical problems of the prior art in the aspects of automation degree, quantitative ability, detection consistency and result reliability.

[0013] In one aspect, the present application proposes a fluorescent test paper visualization system based on deep learning, comprising:

[0014] A first calibration module detects the fluorescent test paper image obtained by shooting through a target detection network to obtain a test paper main body region, a color block region and a test paper key corner coordinate, wherein the target detection network is a deep learning model specially designed to identify specific object targets in images or videos and accurately calibrate positions;

[0015] A second calibration module performs homographic perspective transformation on the test paper main body region according to the test paper key corner coordinate to generate a perspective-corrected test paper standard view;

[0016] A third calibration module calculates a color mapping matrix based on the actual color value of the color block region and the preset standard color value, performs color space standardization correction on the perspective-corrected image, and outputs a color-corrected test paper image;

[0017] An identification module inputs the color-corrected test paper image into a hybrid deep learning model composed of a convolutional neural network and a Transformer to extract local texture features and spatial long dependence features of the fluorescent region, and outputs a fluorescence intensity concentration prediction value of the target detection area;

[0018] A judgment module generates an explainability heat map based on the intermediate features of the hybrid deep learning model for positioning the fluorescence signal contribution area.

[0019] Preferably, the first calibration module comprises:

[0020] YOLOv8 network fine-tuning and lightweight deployment TensorRT acceleration;

[0021] The original shooting image containing the fluorescent test paper is input, and the initial features of the picture include the model of the mobile phone, the background, the light, and the shooting angle;

[0022] The detected test paper body region, colorimetric block region and corner point coordinates are used for subsequent perspective correction and color correction.

[0023] Preferably, the first calibration module further comprises:

[0024] A diversified dataset is constructed and YOLOv8 is fine-tuned, a custom detection head is added: test paper body, colorimetric block, test paper four-corner key points, converted to ONNX, compiled to INT8 / FP16 using TensorRT, mobile terminal deployment high frame rate inference; collect test paper images taken in multiple environments, establish a training set containing multiple angles, different illuminations and different backgrounds;

[0025] Among them, on the basis of the open source YOLOv8 model, the custom detection head is added, including:

[0026] The Head1 detection head is used for test paper body detection, the task type is target detection, and the boundary box BBox and confidence of the test paper body are outputted, which are used for positioning the overall region of the test paper in the image;

[0027] The Head2 detection head is used for colorimetric block detection, the task type is target detection, and the boundary box BBox and class probability of the colorimetric block are outputted, which are used for identifying the position and type of each colorimetric block;

[0028] The Head3 detection head is used for test paper four-corner key point detection, the task type is key point detection, and the coordinates (x, y) and visibility confidence of the four corners of the test paper are outputted;

[0029] Transfer learning is used to load COCO pre-training weights to improve convergence efficiency;

[0030] After the model training, the ONNX format is exported, and the INT8 or FP16 acceleration model is compiled in the mobile terminal inference engine TensorRT / OpenVINO;

[0031] The inference engine is deployed on the APP terminal or embedded terminal to realize 30fps real-time detection.

[0032] Preferably, the second calibration module comprises:

[0033] Perspective correction is realized by using homography matrix calculation, and the colorimetric block mean value is extracted, and the least square fitting color space mapping matrix is fitted;

[0034] Among them, the four-point homography transformation is used to obtain the transformation matrix H, including:

[0035] H=A -1 B;

[0036] Wherein A is the detected corner point coordinate matrix, B is the standard rectangular coordinate matrix generated by the standard test paper size;

[0037] Apply to the original image, from the tilt / distortion state, to a fixed size, angle standard rectangular view, so as to facilitate color area extraction, color identification and subsequent analysis.

[0038] Preferably, the third calibration module is also used to eliminate color deviation caused by equipment and light, including extracting the RGB / Lab mean value of the color block in the image, establishing a mapping relationship with the preset standard value, performing color matrix transformation on the pixels of the perspective correction image, and outputting the standardized color;

[0039] Wherein, the color block construction mapping table is constructed, including:

[0040] {batch_id:{level1:lab1,level2:lab2,...}};

[0041] Wherein, batch_id represents the number of a group of samples; level 1, level 2,... represent multiple label levels contained in the batch data; lab1, lab2,... represent feature values at the level;

[0042] The standard value includes using standard light source D65 to shoot the standard test paper at the factory, and obtaining the Lab or sRGB value of each color block;

[0043] The least square fitting is used to solve the color correction matrix M;

[0044] min M ‖X·M-Y‖ 2 ;

[0045] Wherein, M is the mapping matrix, X is the sampling value matrix, Y is the standard value matrix; ‖X·M-Y‖ 2 Indicates the sum of squares of errors; min M Indicates the M that minimizes the error.

[0046] Preferably, the recognition module inputs the color corrected test paper image into a hybrid deep learning model composed of convolutional neural network and Transformer, extracts local texture features and spatial long dependence features of the fluorescent region, and outputs the fluorescence intensity concentration prediction value of the target detection area, including:

[0047] The preprocessed fluorescent image is divided into several uniform small blocks, denoted as Patch, including:

[0048] The test paper region is divided into N sub-regions with size HxW, forming an image block sequence with shape (N, H, W, 3), where 3 is due to the fusion of three channels for fluorescence intensity calculation; each image block is mapped to a fixed-dimensional feature vector space through a Patch Embedding layer, thereby converting the two-dimensional image block into a one-dimensional visual word vector for easy processing by the Transformer;

[0049] Through the multi-head self-attention mechanism, the embedded vectors are sent to the Transformer encoder. The visual word vector Z is as follows:

[0050]

[0051] where Q is Query, representing the content that the current color block image block needs to pay attention to; K is Key, indicating the information type contained in other image blocks, including edges, textures, and fluorescence; V is Value, the actual feature information extracted from the image block, including original RGB values and gradient features obtained by linear transformation of the Patch Embedding vector; d k is the dimension of the Key vector, used to scale the dot product to prevent gradient vanishing;

[0052] Stack multiple Encoder Blocks. Multiple Transformer Encoder Blocks are stacked together to extract higher-level semantic features layer by layer. Each layer can further integrate context information, so that the output feature vector not only contains local brightness information, but also integrates the spatial distribution trend and structural rules of the entire test paper. The output global feature vector sequence is further processed, i.e., labeled or global average pooling, to obtain a comprehensive global feature representation, i.e., to obtain cross-regional long dependency features. At the same time, local texture features extracted by CNN are also retained;

[0053] The global feature of the Transformer and the local texture feature of the CNN are spliced and input into a multi-layer perceptron regression head, and finally the quantitative concentration value of the fluorescence signal and its confidence interval are output.

[0054] Preferably, the judgment module generates an explainability heat map based on intermediate features of the hybrid deep learning model, for locating a fluorescence signal contribution area, the intermediate features including features in a Transformer self-attention weight map and feature maps of a last CNN convolution layer relied on by a Grad-CAM method; a fusion type attention visualization atlas is generated from the explainability heat map by superimposing the Transformer self-attention weight map and the Grad-CAM heat map, so as to intuitively show a key fluorescence area focused by the model when making a concentration judgment, and to enhance explainability and credibility of a result, including:

[0055] According to a model structure, a feature layer most representative for decision is selected as an analysis target, including: a final output layer of a Transformer encoder, for obtaining a global attention distribution or feature maps A k for capturing a local spatial sensitive area;

[0056] wherein the Grad-CAM is calculated by the following manner, for a selected target convolution layer feature map A k ∈R H×W×C wherein R is a data type representing the feature map, H is a height, W is a width, and C is a channel number, for each channel k (k = 1, 2,..., C), in addition, (i, j) represents a spatial position, and the following operation is performed to calculate a gradient of a final prediction output y fluorescence concentration value with respect to each channel feature map in the layer, including:

[0057]

[0058] wherein is a partial derivative, A k is a feature map of the kth channel, and k is an importance weight of the kth channel.

[0059] The channel feature maps are weighted and summed, and a spatial heat map is generated through a ReLU activation function:

[0060] L Grad-CAM = ReLU(∑ k α k A k );

[0061] wherein L Grad-CAM is a heat map generated by the Grad-CAM; and ReLU is an activation function.

[0062] L Grad-CAM is pixel-aligned with an input image, and a resolution is adjusted through bilinear interpolation or up-sampling, and then a pseudo-color map or a gray-scale map is generated, which is superimposed on the original image to mark a high weight area.

[0063] Preferably, the judgment module further comprises extracting the average attention weight matrix of the last layer or multi-head attention from the Transformer model, which describes the degree of correlation between each image patch, including:

[0064]

[0065] wherein Attention i is the attention weight matrix of the i-th attention head in the visual word vector Z; h is the number of attention heads, since the size of the attention weight matrix is usually the same as the length of the input sequence, in order to visualize it to the spatial resolution of the original image, an upsampling operation is needed, including bilinear interpolation, nearest neighbor interpolation, after upsampling, an attention heat map L Attention .

[0066] Preferably, the judgment module further comprises generating a fusion attention heat map:

[0067] L Fusion =λ·L Grad-CAM +(1-λ)·L Attention ;

[0068] wherein λ is an adjustable parameter for balancing the influence of local sensitivity and global attention.

[0069] On the other hand, the present application also provides a fluorescent test paper visualization method based on deep learning, comprising:

[0070] Step S1: detecting the fluorescent test paper image obtained by shooting through a target detection network to obtain the test paper main body region, color block region and test paper key corner coordinates, wherein the target detection network is a deep learning model specially designed for identifying specific object targets in images or videos and accurately marking positions;

[0071] Step S2: performing homographic perspective transformation on the test paper main body region according to the test paper key corner coordinates to generate a perspective-corrected test paper standard view;

[0072] Step S3: calculating a color mapping matrix based on the actual color value of the color block region and the preset standard color value, performing color space standardization correction on the perspective-corrected image, and outputting a color-corrected test paper image;

[0073] Step S4: input the color-corrected test paper image into a hybrid deep learning model composed of a convolutional neural network and a Transformer, extract local texture features and spatial long dependence features of the fluorescent region, and output the fluorescence intensity concentration prediction value of the target detection area;

[0074] Step S5: generate an explainable heat map based on the intermediate features of the hybrid deep learning model for locating the fluorescent signal contribution area.

[0075] Compared with the prior art, the beneficial effects of the present application are that a complete fluorescent test paper visualization intelligent analysis system is constructed by deeply integrating computer vision and deep learning technology. Firstly, the improved YOLOv8 multi-task detection network is used to accurately locate the test paper main body, color block and key corner points, realize robust perception under complex shooting conditions (such as different mobile phone models, uneven light, angle tilt), and then perform perspective correction through homography transformation to eliminate geometric deformation and generate a standard orthogonal view. Further, based on the actual color value and the preset standard value of the color block area, the least squares method is used to fit the color mapping matrix to correct the image in the color space, effectively overcome the color deviation caused by equipment difference and environmental light, and improve the cross-scene consistency. In the recognition stage, a deep learning model combining CNN and Transformer is used, CNN extracts local texture features of the fluorescent region, and Transformer models long-distance spatial dependence relationship to realize high-precision quantitative prediction of the fluorescence signal intensity; and Grad-CAM and attention mechanism are used to generate an explainable heat map to intuitively show the model decision basis, significantly enhancing the credibility and explainability of the results. Finally, the system binds the detection results with the unique identification of the test paper, such as a two-dimensional code, and uploads it to the cloud database to realize centralized storage, batch management and whole-process traceability of the detection data. The present application comprehensively solves the bottleneck problems of traditional fluorescent test paper detection in low automation, weak quantitative ability, strong environmental sensitivity and unreliable results. BRIEF DESCRIPTION OF DRAWINGS

[0076] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not intended to limit the scope of the application. Moreover, the same reference numerals are used throughout the various drawings to designate identical elements. In the drawings:

[0077] Figure 1 a functional block diagram of the deep learning-based fluorescent test paper visualization system provided by the embodiments of the present application;

[0078] Figure 2The implementation logic of the fusion attention heat map of the generated fluorescence signal key area of the fluorescence test paper visualization system based on deep learning provided by the embodiment of the present application is provided;

[0079] Figure 3 The detection program schematic diagram of the fluorescence test paper visualization system based on deep learning provided by the embodiment of the present application is provided.

[0080] Figure 4 The flowchart of the fluorescence test paper visualization method based on deep learning provided by the embodiment of the present application is provided. DETAILED DESCRIPTION

[0081] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0082] Reference Figure 1 As shown in the drawings, the embodiment proposes a fluorescence test paper visualization system based on deep learning, which comprises:

[0083] The first calibration module detects the fluorescence test paper image obtained by shooting through a target detection network to obtain the test paper main body area, the color block area and the test paper key corner coordinates, wherein the target detection network is a deep learning model specially designed to identify specific object targets in images or videos and accurately calibrate positions;

[0084] The second calibration module performs homographic perspective transformation on the test paper main body area according to the test paper key corner coordinates to generate a perspective-corrected test paper standard view;

[0085] The third calibration module calculates a color mapping matrix based on the actual color value of the color block area and the preset standard color value, performs color space standardization correction on the perspective-corrected image, and outputs the color-corrected test paper image;

[0086] The recognition module inputs the color-corrected test paper image into a hybrid deep learning model composed of a convolutional neural network and a Transformer to extract local texture features and spatial long dependence features of the fluorescence area, and outputs the fluorescence intensity concentration prediction value of the target detection area;

[0087] The judgment module generates an explainable heat map based on the intermediate features of the hybrid deep learning model for positioning the fluorescence signal contribution area.

[0088] In addition, the embodiment also proposes to bind the detection result with the unique identification information of the test paper, and upload it to the cloud database to realize the storage, traceability and batch management of the detection result.

[0089] Referring to Figure 2 As shown in the figure, the original test paper image is the input data, i.e. the test paper image to be analyzed, which is the basis of the entire analysis process; the strong fluorescence region refers to the region highlighted by Grad-CAM (Gradient-weighted Class Activation Mapping). Grad-CAM is a technique for visualizing the decision-making process of deep learning models, which can point out the image regions that the model pays most attention to when making predictions. In this context, it helps to identify the regions in the image with strong fluorescence signals, which are related to specific chemical reactions; the weak fluorescence region refers to the region associated by the self-attention mechanism. The self-attention mechanism allows the model to capture the dependencies between different positions in the input sequence, which can be understood as the interaction between different pixels or regions in image processing. Here, it helps to identify regions that have weak fluorescence signals but still contribute to the overall reaction; the test paper edge refers to the region identified by gradient correlation. Gradient correlation refers to determining the importance of certain features or structures in an image based on gradient information. In this context, it is used to detect the edge of the test paper, which is very important for accurate segmentation and positioning of the reaction area on the test paper; the fusion atlas refers to the final generated reaction core region atlas, which is obtained by fusing the results of the above three methods. This fusion can be achieved by weighted superposition, element-by-element multiplication, etc. The purpose is to consider the local sensitivity (Grad-CAM), global attention distribution (self-attention mechanism) and structural information (gradient correlation) comprehensively, so as to more comprehensively and accurately identify and interpret the key reaction regions in the test paper image.

[0090] Referring to Figure 3 As shown in the figure, the embodiment uses MIPs (molecularly imprinted polymers) as the carrier or recognition element of the fluorescent probe, selectively captures the target ion Cu 2+ , and uses the visualization system of the embodiment to complete the detection of MIIMs. Since MIIMs is a visualization system based on blue response signal and red internal standard signal fluorescence color, the B / R color channel mode is selected. The figure is the RGB value of the fluorescence color of MIIMs and the ratio in B / R mode read by the detection software through the input of the corresponding fluorescence picture under the concentration of 20-100nM Cu2+.

[0091] It can be understood that the visualization of the fluorescent detection test paper is an important indicator of its application. The MIIMs fluorescent detection test paper does not emit fluorescence under daylight lamps, and the film body color is light brown. Under 365 nm ultraviolet excitation, both fluorescent detection strips show a visible fluorescent color response to different concentrations of Cu2+. Although IIMs produces a fluorescent color change, compared with the MIIMs fluorescent detection test paper, the fluorescent detection effect of IIMs is not obvious enough and the color is not bright enough due to the insufficient number of surface recognition sites and background fluorescence interference without the magnetic guidance process to coat part of Fe3O4@SiO2@IIPs in the PVDF film. It affects the subsequent smartphone-assisted detection. The MIIMs fluorescent detection test paper exhibits a fluorescent response with bright and uniform fluorescent color and more intuitive color change. Comparison of the surface SEM of the two fluorescent detection strips shows that the imprint layer uniformly loaded on the base film based on the surface imprinting technology effectively improves the visualization performance of the fluorescent detection strip, which can be used for on-site visual detection of Cu2+.

[0092] In the embodiments of the present application, the recovery rates of MIIMs for Cu2+ added in deionized water and tap water are 98.5%-102% and 101%-104%, respectively, and the RSDs are 1.3%-2.3% and 1.2%-2.2%, respectively. In the direct detection of electroplating wastewater, the detection concentration of MIIMs is similar to that of ICP-MS. The comparison results show that the MIIMs based on smartphone assistance can accurately and quickly detect Cu2+ in actual samples on site, and the detection results are close to those of ICP-MS. The results of the detection of Cu2+ concentration in actual samples by the MIIMs visualization method and the ICP-MS method are compared in the following table:

[0093]

[0094] In some embodiments of the present application, the first calibration module comprises:

[0095] YOLOv8 network fine-tuning and lightweight deployment TensorRT acceleration;

[0096] The input includes the original shooting image containing the fluorescent detection test paper. The initial features of the picture include the model of the mobile phone, the background, the light, and the shooting angle.

[0097] The detected test paper main body area, colorimetric block area, and corner point coordinates are used for subsequent perspective correction and color correction.

[0098] Specifically, the dataset construction includes the following data collection strategies: using different brand mobile phones such as iPhone, Huawei, Xiaomi, etc. to take pictures under various lighting conditions; environmental diversity includes natural light including sunny / cloudy, indoor light including LED / incandescent light, background including desktop, hand, paper, textured background, shooting angle including overhead, side tilt, rotation ± 45°; the number of samples is ≥5000 original images, and each is labeled with multiple targets.

[0099] In some embodiments of the present application, the first calibration module further comprises:

[0100] Build a diversified dataset and fine-tune YOLOv8, add custom detection heads: test paper body, color block, test paper corner key points, convert to ONNX, compile to INT8 / FP16 using TensorRT, mobile deployment high frame rate inference; collect test paper images taken in multiple environments to establish a training set containing multiple angles, different lighting, and different backgrounds;

[0101] Among them, on the basis of the open source YOLOv8 model, add custom detection heads, including:

[0102] Make Head1 detection head to detect the test paper body, and its task type is target detection, output the bounding box BBox and confidence of the test paper body, which is used to locate the overall area of the test paper in the image;

[0103] Make Head2 detection head to detect the color block, and its task type is target detection, output the bounding box BBox and class probability of the color block, which is used to identify the position and type of each color block;

[0104] Make Head3 detection head to detect the test paper corner key points, and its task type is key point detection, output the coordinates (x, y) and visibility confidence of the four corner points of the test paper;

[0105] Use transfer learning to load COCO pre-trained weights to improve convergence efficiency;

[0106] After the model is trained, export it in ONNX format, and compile it into an INT8 or FP16 acceleration model in the mobile inference engine TensorRT / OpenVINO;

[0107] Deploy the inference engine on the APP side or embedded terminal to realize 30fps real-time detection.

[0108] Specifically, INT8 quantization requires a calibration set:

[0109] Prepare 100-300 unlabeled images for calibration;

[0110] Use the default Entropy Calibrator TensorRT;

[0111] Mobile deployment includes:

[0112] Android integration with TensorRT via JNI or using DeepJavaLib

[0113] iOS: Metal Performance Shaders+CoreML

[0114] Embedded systems: Jetson Nano / TX2 runs TensorRT inference at 30+ FPS.

[0115] In some embodiments of this application, the second calibration module includes:

[0116] Perspective correction is achieved by calculating the homography matrix, and the mean of the color patches is extracted. The color space mapping matrix is ​​then fitted using least squares.

[0117] The transformation matrix H obtained by using four-point homography transformation includes:

[0118] H = A -1 B;

[0119] Where A is the detected corner coordinate matrix, and B is the standard rectangular coordinate matrix, which is a coordinate system generated by the standard test strip size;

[0120] The original image is transformed from a tilted / distorted state into a rectangular view with a fixed size and standard angles to facilitate colorimetric region extraction, color recognition, and subsequent analysis.

[0121] Specifically, this embodiment provides the following example of the algorithm for implementing perspective correction:

[0122] import cv2

[0123] import numpy as np

[0124] def perspective_correction(image,corners,target_size=(300,100)):

[0125] #corners:[(x1,y1),(x2,y2),(x3,y3),(x4,y4)] are arranged clockwise.

[0126] src = np.float32(corners)

[0127] dst = np.float32([[0, 0], [target_size[0], 0], [target_size[0], target_size[1]], [0, target_size[1]]])

[0128] H = cv2.getPerspectiveTransform(src, dst)

[0129] corrected = cv2.warpPerspective(image, H, target_size)

[0130] return corrected, H

[0131] In some embodiments of the present application, the third calibration module is further configured to eliminate color deviation caused by the device and light, including extracting the RGB / Lab average value of the color block in the image, establishing a mapping relationship with the preset standard value, performing color matrix transformation on the pixels of the perspective corrected image, and outputting the standardized color;

[0132] Wherein, the standard value database Y includes standard values including using standard light source D65 to shoot standard test paper at the time of factory shipment, obtaining the Lab or sRGB value of each color block, and constructing a color block mapping table:

[0133] {batch_id:{level1:lab1,level2:lab2,...}};

[0134] Wherein, batch_id represents the number of a group of samples; level 1, level 2,... represent multiple label levels contained in the batch data; lab1, lab2,... represent feature values at the level;

[0135] The least square fitting is used to solve the color correction matrix M;

[0136] min M ‖X·M-Y‖ 2 ;

[0137] Wherein, M is a mapping matrix, X is a sampling value matrix, Y is a standard value matrix; ‖X·M-Y‖ 2 represents the sum of squares of errors; min M represents the M that minimizes the error.

[0138] It is understood that the standard light source D65 is a kind of lighting body simulating daylight, which is widely used in color management, printing, photography and display technology fields. It aims to imitate the natural light conditions under the noon sunlight in the northern hemisphere, and its color temperature is about 6500K, which means it has a cooler white tone with more blue components. The spectral distribution of D65 light source is calculated based on a specific mathematical model, aiming to accurately reflect the characteristics of natural daylight as much as possible, including its radiation intensity at different wavelengths.

[0139] In practical applications, using D65 as a standard light source helps to ensure color consistency and accuracy, whether in the color evaluation of physical samples or in the processing of digital images. By performing color matching and calibration under such standardized conditions, the influence of environmental light changes on color perception can be reduced, ensuring consistent color results at different times and places.

[0140] Specifically, the embodiment gives the following examples for the implementation algorithm of extracting the mean RGB of the color block:

[0141] block_regions = detect_color_blocks(corrected_img) # YOLO output or fixed ROI

[0142] X = []

[0143] for roi in block_regions:

[0144] mean_rgb = cv2.mean(roi)[:3]

[0145] X.append(mean_rgb)

[0146] X = np.array(X) # shape: (N, 3)

[0147] In some embodiments of the present application, the recognition module inputs the color-corrected test paper image into a hybrid deep learning model composed of a convolutional neural network and a Transformer, extracts local texture features and spatial long dependence features of the fluorescent region, and outputs the fluorescence intensity concentration prediction value of the target detection area, including:

[0148] The preprocessed fluorescent image is divided into several uniform small blocks, denoted as Patch, including:

[0149] The test paper region is divided into N sub-regions with a size of HxW, forming an image block sequence with a shape of (N, H, W, 3), where 3 is due to the fusion of three channels for fluorescence intensity calculation; each image block is mapped to a fixed-dimensional feature vector space through a Patch Embedding layer, thereby converting the two-dimensional image block into a one-dimensional visual word vector for easy processing by the Transformer;

[0150] Through the multi-head self-attention mechanism, the embedded vectors are sent to the Transformer encoder, and the visual word vector Z is as follows:

[0151]

[0152] Where Q is Query, representing the content that the current color block image block needs to pay attention to; K is Key, indicating the information type contained in other image blocks, including edges, textures, and fluorescence; V is Value, the actual feature information extracted from the image block, including original RGB values, gradient features, and linear transformations of the Patch Embedding vectors; d k is the dimension of the Key vector, used to scale the dot product to prevent gradient disappearance, where:

[0153]

[0154] And the dimension of Q is [N·h, d_k], the query requirement of all image blocks, such as paying attention to fluorescence intensity; the dimension of K is [N·h, d_k], the feature identification of all image blocks, such as strong fluorescence texture; the dimension of V is [N·h, d_v], the actual feature value of all image blocks, such as RGB gradient, where N is the scalar image block quantity, i.e., the number of test paper blocks; h is the number of attention heads.

[0155] Stack multiple Encoder Blocks, multiple Transformer Encoder Blocks are stacked together to extract higher-level semantic features layer by layer, and each layer can further integrate context information, so that the output feature vector not only contains local brightness information, but also integrates the spatial distribution trend and structural rules of the entire test paper. The output global feature vector sequence is further processed, i.e., labeled or global average pooling, to obtain a comprehensive global feature representation, i.e., to obtain cross-regional long dependency features, while retaining the local texture features extracted by CNN;

[0156] The global feature of the Transformer and the local texture feature of the CNN are spliced and input into a multi-layer perceptron regression head, and finally the quantitative concentration value of the fluorescence signal and its confidence interval are output.

[0157] Specifically, the stacked multi-layer Encoder Block refers to connecting multiple Transformer Encoder Blocks with the same structure and progressive functions in sequence to form a deep neural network structure for extracting and abstracting high-level semantic features of the input image layer by layer. The Transformer Encoder Block is the basic component unit of the Transformer model, which includes two core components, including a multi-head self-attention mechanism (Multi-Head Self-Attention) and a feed-forward neural network (Feed-Forward Network, FFN); in this application, the multi-head self-attention mechanism (Multi-Head Self-Attention) is used to capture the global dependency between different parts of the input sequence, and the input is the feature sequence after image patching (Patch), which can model the correlation between different regions of the test paper (such as gradient change, symmetry, etc.). The feed-forward neural network (Feed-Forward Network, FFN) performs nonlinear transformation on the features of each position to enhance the expression ability of the model. Each sub-layer is equipped with residual connection (Residual Connection) and layer normalization (Layer Normalization) to ensure stable training of the deep network.

[0158] It can be understood that the stacked multi-layer Encoder Block refers to connecting multiple Transformer Encoder Blocks containing multi-head self-attention mechanisms and feed-forward neural networks in sequence to form a deep feature extraction network for modeling the long-distance spatial dependency between regions in the fluorescent test paper image layer by layer, realizing semantic abstraction from local brightness to global distribution pattern, and with the increase of the number of layers, the feature representation gradually evolves from local brightness to global structure. The present embodiment gives the following table as an example:

[0159]

[0160] It can be understood that z i (N) represents the feature vector generated by the Nth layer; the preliminary feature vector z i (1) and z i (2) These vectors mainly contain local brightness information; the intermediate feature vector z i (3) to z i (6) These vectors begin to integrate spatial information in a larger range; the high-level feature vector z i (7) to z i (12)These vectors contain the global spatial distribution rules. The final output global feature vector sequence Z = [z1, z2,..., zT] is obtained by processing the input image sequence X = [x1, x2,..., xT] through the multi-layer encoder. N ] where each z i is the final feature vector obtained after processing through the multi-layer encoder, which integrates the context information of all levels.

[0161] In some embodiments of the present application, the judgment module generates an explainability heat map based on the intermediate features of the hybrid deep learning model for positioning the fluorescence signal contribution area, the intermediate features including the features in the Transformer self-attention weight map and the feature map of the last CNN convolution layer relied on by the Grad-CAM method; by superimposing the Transformer self-attention weight map and the Grad-CAM heat map, a fusion attention visualization atlas is generated according to the explainability heat map to intuitively show the key fluorescence area that the model focuses on when making concentration judgment, thereby enhancing the explainability and credibility of the results, including:

[0162] According to the model structure, the feature layer most representative of the decision is selected as the analysis target, including: the final output layer of the Transformer encoder for obtaining the global attention distribution or the feature map A k for capturing local spatial sensitive regions;

[0163] where Grad-CAM is calculated by the following method, for the selected target convolution layer feature map A k ∈R H×W×C , where R is the data type representing the feature map, H is the height, W is the width, and C is the number of channels. For each channel k (k = 1, 2,..., C), in addition to (i, j) representing the spatial position, the following operations are performed to calculate the gradient of the final prediction output y fluorescence concentration value with respect to each channel feature map within the layer, including:

[0164]

[0165] where is the partial derivative, A k is the feature map of the kth channel, and α k represents the importance weight of the kth channel.

[0166] The channel feature maps are weighted and summed, and a spatial heat map is generated through a ReLU activation function:

[0167] L Grad-CAM = ReLU(∑ k α k A k );

[0168] where LGrad-CAM is referred to as the heat map generated by Grad-CAM; ReLU is referred to as an activation function.

[0169] L Grad-CAM is pixel-aligned with the input image, the resolution is adjusted through bilinear interpolation or up-sampling, and then a pseudo-color map or a gray-scale map is generated and superimposed on the original image to mark the high-weight region.

[0170] In the embodiment, the following examples are given for the implementation algorithm of Grad-CAM+Attention visualization:

[0171]

[0172]

[0173] In some embodiments of the present application, the judgment module further comprises extracting the average attention weight matrix of the last layer or multi-head attention from the Transformer model, which describes the degree of mutual association between each image patch, including:

[0174]

[0175] wherein Attention i is the attention weight matrix of the i-th attention head in the visual word vector Z; h is the number of attention heads, since the size of the attention weight matrix is usually the same as the length of the input sequence, in order to visualize it to the spatial resolution of the original image, an up-sampling operation is needed, including bilinear interpolation, nearest neighbor interpolation, and after up-sampling, an attention heat map L Attention with the same size as the original image is obtained.

[0176] Specifically, Attention i is calculated by the following formula:

[0177]

[0178] wherein i is the attention head index, ranging from 1 to h; Q i is the query matrix of the i-th head; is the transpose of the key matrix of the i-th head; d k is the key vector dimension, typically 64.

[0179] In some embodiments of the present application, the judgment module further comprises generating a fusion attention heat map:

[0180] L Fusion = λ·L Grad-CAM +(1-λ)·L Attention ;

[0181] where λ is a tunable parameter to balance the influence of local sensitivity and global attention.

[0182] It can be understood that in this way, we can combine local and global information to more comprehensively understand the decision-making process of the model and improve the explainability and credibility of the results

[0183] Referring to Figure 4 The embodiment also provides a fluorescence test paper visualization method based on deep learning, and the method comprises the following steps:

[0184] Step S1: detecting the fluorescence test paper image acquired by shooting through a target detection network to acquire a test paper main body region, a color block region and test paper key corner point coordinates, wherein the target detection network is a deep learning model specially designed for identifying specific object targets in images or videos and accurately marking positions;

[0185] Step S2: performing homographic perspective transformation on the test paper main body region according to the test paper key corner point coordinates to generate a perspective-corrected test paper standard view;

[0186] Step S3: calculating a color mapping matrix based on actual color values of the color block region and preset standard color values, performing color space standardization correction on the perspective-corrected image, and outputting a color-corrected test paper image;

[0187] Step S4: inputting the color-corrected test paper image into a hybrid deep learning model composed of a convolutional neural network and a Transformer to extract local texture features and spatial long dependence features of a fluorescence region, and outputting a fluorescence intensity concentration prediction value of a target detection region;

[0188] Step S5: generating an explainability heat map based on intermediate features of the hybrid deep learning model, which is used for positioning a fluorescence signal contribution region.

[0189] In addition, the embodiment also proposes binding the detection result with unique identification information of the test paper and uploading the detection result to a cloud database to realize storage, tracing and batch management of the detection result.

[0190] The beneficial effects of the embodiment are that a complete fluorescent test paper visualization intelligent analysis system is constructed by deeply integrating computer vision and deep learning technology. First, an improved YOLOv8 multi-task detection network is used to accurately locate the test paper main body, color block and key corner points, realize robust perception under complex shooting conditions (such as different mobile phone models, uneven light, angle tilt), and then correct the perspective through homography transformation, eliminate geometric deformation, and generate standard orthogonal view. Further, based on the actual color value and the preset standard value of the color block area, the least square method is used to fit the color mapping matrix, and the image is corrected in color space, which effectively overcomes the color deviation caused by equipment difference and environmental light, and improves the cross-scene consistency. In the identification stage, a deep learning model combining CNN and Transformer is used, CNN extracts local texture features of the fluorescent area, and Transformer models long-distance spatial dependence relationship, realizes high-precision quantitative prediction of fluorescent signal intensity, and generates an interpretable heat map through Grad-CAM and attention mechanism, which intuitively shows the model decision basis, significantly enhances the credibility and interpretability of the results. Finally, the system binds the detection results with the unique identification of the test paper, such as a two-dimensional code, and uploads it to the cloud database to realize centralized storage, batch management and whole-process traceability of the detection data. The application solves the bottleneck problems of traditional fluorescent test paper detection in low automation, weak quantitative ability, strong environmental sensitivity and unreliable results.

[0191] Those skilled in the art will appreciate that embodiments of the application can be provided as methods, systems, or computer program products. Accordingly, the application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can be embodied in the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical memory, etc.) having computer usable program code embodied thereon.

[0192] The application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks. Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks.

[0193] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flow Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or multiple blocks.

[0194] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or multiple blocks.

[0195] Finally, it should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, rather than limiting the same. Even though the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced equivalently, and any modification or replacement without departing from the spirit and scope of the present application should be covered within the protection scope of the claims of the present application.

Claims

1. A deep learning based fluorescent test strip visualization system, characterized in that, The method comprises the following steps: A first calibration module detects the fluorescence test paper image obtained by shooting through a target detection network, obtains the test paper main body area, color block area and test paper key corner coordinates; A second calibration module performs homographic perspective transformation on the test paper main body area according to the test paper key corner coordinates to generate a perspective-corrected test paper standard view; A third calibration module calculates a color mapping matrix based on the actual color value of the color block area and the preset standard color value, performs color space standardization correction on the perspective-corrected image, and outputs a color-corrected test paper image; An identification module inputs the color-corrected test paper image into a hybrid deep learning model composed of a convolutional neural network and a Transformer, extracts local texture features and spatial long dependence features of the fluorescence region, and outputs a fluorescence intensity concentration prediction value of the target detection area; A judgment module generates an explainability heat map based on the intermediate features of the hybrid deep learning model for positioning the fluorescence signal contribution area.

2. The deep learning-based visualization system for fluorescent test strips according to claim 1, wherein, The first calibration module comprises: YOLOv8 network fine-tuning and lightweight deployment TensorRT acceleration; Input the original shooting image containing the fluorescence test paper, and the initial features of the picture include the phone model, background, illumination and shooting angle; The detected test paper main body area, color block area and corner coordinates are used for subsequent perspective correction and color correction.

3. The deep learning-based visualization system for fluorescent test strips according to claim 2, wherein, The first calibration module further comprises: Construct a diversified dataset and fine-tune YOLOv8, add a custom detection head: test paper body, color block, and test paper four-corner key points, convert to ONNX, use TensorRT to compile to INT8 / FP16, and deploy high-frame-rate inference on mobile terminals; Collect test paper images taken in multiple environments to establish a training set containing multiple angles, different illuminations and different backgrounds, including: Among them, based on the open source YOLOv8 model, the custom detection head is added, including: Make the Head1 detection head detect the test paper body, and the task type is target detection, which outputs the bounding box BBox and confidence of the test paper body, and is used to locate the overall area of the test paper in the image; Make the Head2 detection head detect the color block, and the task type is target detection, which outputs the bounding box BBox and class probability of the color block, and is used to identify the position and type of each color block; Make the Head3 detection head detect the test paper four-corner key points, and the task type is key point detection, which outputs the coordinates (x, y) and visibility confidence of the test paper four-corner key points; Use transfer learning to load the COCO pre-trained weight to improve the convergence efficiency; After the model training is completed, the ONNX format is exported, and the INT8 or FP16 acceleration model is compiled in the mobile terminal inference engine TensorRT / OpenVINO; Deploy the inference engine on the APP terminal or embedded terminal to realize 30fps real-time detection.

4. The deep learning-based visualization system for fluorescent test strips according to claim 3, wherein, The second calibration module comprises: Perspective correction is realized by using homographic matrix calculation, and the color block mean value is extracted to least square fit the color space mapping matrix; Among them, four-point homographic transformation is used to obtain the transformation matrix H, including: H = A -1 B; Wherein A is the detected corner point coordinate matrix, B is the standard rectangular coordinate matrix generated by the standard test paper size; Apply to the original image, from the tilt / distortion state, to a fixed size, angle standard rectangular view, so as to facilitate color area extraction, color identification and subsequent analysis.

5. The deep learning-based visualization system for fluorescent test strips according to claim 4, wherein, The third calibration module is also used to eliminate color deviation caused by equipment and light, including extracting the RGB / Lab average value of the color block in the image, establishing a mapping relationship with the preset standard value, performing color matrix transformation on the pixels of the perspective correction image, and outputting the standardized color; Wherein, the colorimetric mapping table is constructed, including: {batch_id:{level 1:lab1,level2:lab2,...}}; Wherein, batch_id represents the number of a group of samples; level 1, level 2,... represent a plurality of label levels contained in the batch data; lab1, lab2,... represent the characteristic values at the level; The preset standard value includes using standard light source D65 to shoot the standard test paper at the factory, and obtaining the Lab or sRGB value of each color block; The least square fitting is used to solve the color correction matrix M; min M ‖X·M-Y‖ 2 ; where M is a mapping matrix, X is a matrix of sampling values, Y is a matrix of standard values; ‖X·M-Y‖ 2 denotes the sum of squares of errors; min M denotes M that minimizes the error.

6. The deep learning-based visualization system for fluorescent test strips according to claim 5, wherein, The recognition module inputs the color corrected test paper image into a hybrid deep learning model composed of convolutional neural network and Transformer, extracts local texture features and spatial long dependence features of the fluorescent region, and outputs the fluorescence intensity concentration prediction value of the target detection area, including: The preprocessed fluorescent image is divided into several uniform small blocks, denoted as Patch, including: The test paper area is divided into N sub-regions with a size of HxW, forming an image block sequence with a shape of (N, H, W, 3), where 3 is the 3 channels that need to be fused for fluorescence intensity calculation; each image block is mapped to a fixed-dimensional feature vector space through a Patch Embedding layer, so as to convert the two-dimensional image block into a one-dimensional visual word vector, facilitating the processing of Transformer; Through the multi-head self-attention mechanism, the embedded vectors are sent to the Transformer encoder to calculate the visual word vector Z, as follows: where Q is Query representing the content that the current image patch needs to pay attention to; K is Key indicating the information type contained in other image patches, including edge / texture / fluorescence; V is Value representing the actual feature information extracted from the image patch, including original RGB value, gradient feature, and the result of linear transformation on the Patch Embedding vector; d k is the dimension of the Key vector, used to scale the dot product to prevent gradient disappearance; Stack multiple layers of Encoder Block, multiple Transformer Encoder Block are stacked together, and higher-level semantic features are extracted layer by layer, and each layer can further integrate context information, so that the output feature vector not only contains local brightness information, but also fuses the spatial distribution trend and structure rule of the whole test paper, and the output global feature vector sequence is further processed, i.e. labeled or global average pooling, to obtain a comprehensive global feature representation, i.e. to obtain cross-regional long dependence features, and also retain the local texture features extracted by CNN; The global feature of Transformer and the local texture feature of CNN are spliced and input into a multi-layer perceptron regression head to output the quantitative concentration value of the fluorescent signal and its confidence interval.

7. The deep learning-based visualization system for fluorescent test strips according to claim 6, wherein, The judgment module generates an explainability heat map based on intermediate features of the hybrid deep learning model, for positioning a fluorescence signal contribution area, the intermediate features including features in a Transformer self-attention weight map and feature maps of a last CNN convolution layer relied on by a Grad-CAM method; a fusion type attention visualization atlas is generated from the explainability heat map by superimposing the Transformer self-attention weight map and the Grad-CAM heat map, so as to intuitively display a key fluorescence area focused on by the model when judging the concentration, and to enhance the explainability and reliability of the result, including: According to the model structure, the feature layer most representative of the decision is selected as the analysis target, including: the final output layer of the Transformer encoder for obtaining the global attention distribution or the feature map A of the last CNN convolution layer k for capturing a local spatially sensitive region; Wherein, Grad-CAM is calculated by the following way, for the selected target convolutional layer feature map A k ∈R H×W×C Wherein R is the data type representing the feature map, H is the height, W is the width, C is the number of channels, for each channel k (k = 1, 2, …, C), in addition (i, j) represents the spatial position, the following operation is performed, the gradient of the final prediction output y fluorescence concentration value with respect to each channel feature map in the layer is calculated, including: wherein denotes a partial derivative, A k feature map of the kth channel, a k denotes an importance weight of the kth channel; The channel feature maps are weighted and summed, and a spatial heat map is generated through a ReLU activation function: L Grad-CAM = ReLU(∑ k α k A k ); wherein L Grad-CAM denotes the heat map generated by Grad-CAM; ReLU denotes an activation function; L Grad-CAM Align with the input image pixel by pixel, adjust the resolution by bilinear interpolation or up-sampling, then generate a pseudo-color map or grayscale map, superimposed on the original image, indicating the high weight area.

8. The deep learning-based visualization system for fluorescent test strips according to claim 7, wherein, The judgment module further includes an average attention weight matrix of the last layer or multi-head attention extracted from the Transformer model, which describes the correlation degree between the image patches, including: wherein Attention i is the attention weight matrix of the i-th attention head in the visual word vector Z; h is the number of attention heads, since the size of the attention weight matrix is usually the same as the length of the input sequence, in order to visualize it to the spatial resolution of the original image, an upsampling operation is needed, including bilinear interpolation, nearest neighbor interpolation, after upsampling, an attention heat map L Attention with the same size as the original image is obtained.

9. The deep learning-based visualization system for fluorescent test strips according to claim 8, wherein, The judgment module further includes generating a fusion attention heat map: L Fusion = λ · L Grad-CAM + (1 - λ) · L Attention ; Where λ is an adjustable parameter for balancing the influence of local sensitivity and global attention.

10. A deep learning-based fluorescent test strip visualization method applied to the deep learning-based fluorescent test strip visualization system according to any one of claims 1-9, characterized in that, Including: Step S1: detecting a fluorescence test strip image obtained by shooting through a target detection network, to obtain a test strip main body area, a color block area and test strip key corner coordinates; Step S2: performing homographic perspective transformation on the test strip main body area according to the test strip key corner coordinates, to generate a perspective-corrected test strip standard view; Step S3: calculating a color mapping matrix based on actual color values of the color block area and preset standard color values, to perform color space standardization correction on the perspective-corrected image, and output a color-corrected test strip image; Step S4: inputting the color-corrected test strip image into a hybrid deep learning model composed of a convolutional neural network and a Transformer, to extract local texture features and spatial long dependence features of a fluorescence area, and output a fluorescence intensity concentration prediction value of a target detection area; Step S5: generating an explainability heat map based on intermediate features of the hybrid deep learning model, for positioning a fluorescence signal contribution area.

Images