A deep learning-based visualization system and method for fluorescence test strips

Abstract

This invention relates to the field of water quality testing technology, and provides a deep learning-based visualization system and method for fluorescent test strips. The method includes: detecting fluorescent test strip images acquired through photography using a target detection network to obtain the main body area of ​​the test strip, the colorimetric area, and the coordinates of key corner points; performing homography perspective transformation on the main body area of ​​the test strip based on the coordinates of the key corner points to generate a perspective-corrected standard view of the test strip; performing color space standardization correction and generating a hybrid deep learning model to output a predicted value of fluorescence intensity concentration in the target detection area; and generating an interpretable heatmap based on the intermediate features of the hybrid deep learning model. This invention eliminates shooting bias through multi-level image correction, accurately quantifies fluorescence intensity by fusing CNN and Transformer models, and improves the reliability of results by combining heatmaps, thus comprehensively enhancing the automation of the detection process.

Description

Technical Field

[0001] This invention relates to the field of water quality testing technology, and more specifically, to a deep learning-based visualization system and method for fluorescent test strips. Background Technology

[0002] Fluorescent test strips have become an important tool for rapid testing outside the laboratory due to their ease of use, low cost, and portability. Detection using fluorescent test strips relies on specific fluorescent probes loaded onto the strip. When the strip comes into contact with a water sample containing the target ion, it selectively binds to the metal ion, triggering a visible change in fluorescence intensity or color. Users can then qualitatively or semi-quantitatively determine the concentration of the analyte by visual comparison or using a portable colorimetric card.

[0003] Existing fluorescent test strips are widely used in rapid on-site detection scenarios such as heavy metal ions and drug residues. They typically rely on users visually comparing the color or fluorescence intensity of the test strip with a pre-made standard color chart to determine the concentration range of the analyte. However, this type of detection method is highly dependent on the operator's subjective judgment and experience. Different individuals often perceive color or fluorescence intensity differently under varying lighting conditions, shooting angles, and background environments, leading to poor repeatability and consistency of test results. Experiments in existing technologies show that the concentration error in different observers' judgments of the same test strip color can reach ±50%, especially noticeable in the low concentration range. [2] .

[0004] To improve the objectivity of visual colorimetry, some industry solutions have attempted to incorporate mobile phone cameras to capture images of test strips and compare them with preset standard color reference images to aid in judgment, thereby reducing the influence of operator subjectivity and improving consistency. However, due to the complex and variable on-site environment, the shooting process is easily affected by various factors such as ambient light intensity, light source color temperature, and shooting angle. The acquired images often exhibit tilt, distortion, or color inaccuracy, leading to significant fluctuations in test results. Experiments have shown that under different lighting conditions, the RGB value of the same object can fluctuate by more than 20%, severely affecting quantitative accuracy. [3] Furthermore, the distribution of fluorescent signals on test paper typically exhibits gradient and local characteristics. Confocal microscopy revealed that the fluorescent probe diffuses non-uniformly on cellulose paper, forming a gradient distribution that is "bright at the center and weak at the edges," which is correlated with ion concentration. [4]Traditional colorimetric cards or simple image comparisons cannot fully extract and quantify this type of spatial distribution information. Furthermore, traditional fluorescence detection mostly relies on simple thresholds or single-point colorimetry, lacking the ability to perform deep learning analysis on the spatial distribution characteristics of fluorescence signals on the test strip surface. Therefore, it suffers from insufficient sensitivity to weak fluorescence signals in low-concentration, complex backgrounds, limited quantitative accuracy, and difficulty in visually interpreting the detection results. Users and regulators find it difficult to trace the credibility and accuracy of the testing process. Simultaneously, existing test results are often recorded in isolation, lacking binding to test strip serial numbers and real-time linkage with cloud databases, which is detrimental to subsequent batch management, data traceability, and large-scale environmental data analysis.

[0005] Therefore, it is necessary to design a deep learning-based visualization system and method for fluorescent test strips to address the technical problems of existing technologies in terms of automation, quantitative capability, detection consistency, and result reliability.

[0006] The cited references in this background section are:

[0007] [1]Wang,X.etal.(2021).Recent advances in optical sensors for copperion detection in environmental water samples.TrAC Trends in AnalyticalChemistry,135,116148.

[0008] [2] Zhang, H. etal. (2019). Evaluation of visual interpretation variability in lateral flow assays. Analytical Chemistry, 91 (15), 9623–9630.

[0009] [3]Bianco, S. et al. (2015). How far can you go with uncalibrated smartphone colorimetry? IEEE Sensors Journal,15(11),6448–6456.

[0010] [4] Xu, W. etal. (2022). Spatially resolved analysis of paper-based fluorescent biosensors using deep learning. Lab on a Chip, 22(8), 1520–1531. Summary of the Invention

[0011] In view of this, the present invention proposes a deep learning-based visualization system and method for fluorescent test strips, aiming to solve the technical problems of insufficient automation, quantitative ability, detection consistency and result reliability in the existing technology.

[0012] In one aspect, this invention proposes a deep learning-based visualization system for fluorescence test strips, comprising:

[0013] The first calibration module uses a target detection network to detect the captured fluorescent test strip image and obtain the coordinates of the main area of ​​the test strip, the colorimetric area, and the key corner points of the test strip. The target detection network is a deep learning model specifically designed to identify specific objects in images or videos and accurately pinpoint their locations.

[0014] The second calibration module performs homography perspective transformation on the main area of ​​the test strip according to the coordinates of the key corner points of the test strip, and generates a standard view of the test strip after perspective correction.

[0015] The third calibration module calculates a color mapping matrix based on the actual color value of the colorimetric block area and the preset standard colorimetric value, performs color space standardization correction on the perspective-corrected image, and outputs a color-corrected test strip image.

[0016] The recognition module inputs the color-corrected test strip image into a hybrid deep learning model composed of a convolutional neural network and a Transformer, extracts the local texture features and spatial length dependence features of the fluorescent region, and outputs the predicted value of the fluorescence intensity concentration of the target detection area.

[0017] The judgment module generates an interpretable heatmap based on the intermediate features of the hybrid deep learning model, which is used to locate the contribution region of the fluorescence signal.

[0018] Preferably, the first calibration module includes:

[0019] YOLOv8 network fine-tuning and lightweight deployment of TensorRT acceleration;

[0020] Input the original image containing the fluorescent test strip. Initial image features include mobile phone model, background, lighting, and shooting angle.

[0021] The detected coordinates of the main area of ​​the test strip, the colorimetric area, and the corner points are used for subsequent perspective correction and color calibration.

[0022] Preferably, the first calibration module further includes:

[0023] We constructed diverse datasets and fine-tuned YOLOv8, adding custom detection heads: test strip body, colorimetric blocks, and key points at the four corners of the test strip. We converted them to ONNX, compiled them into INT8 / FP16 using TensorRT, and deployed high frame rate inference on mobile devices. We collected test strip images taken in various environments and built a training set containing multiple angles, different lighting conditions, and different backgrounds.

[0024] This involves adding the custom detection head to the open-source YOLOv8 model, including:

[0025] The Head1 detection head is used to detect the main body of the test strip. Its task type is target detection. It outputs the bounding box (BBox) and confidence score of the main body of the test strip, which are used to locate the overall area of ​​the test strip in the image.

[0026] Enable the Head2 detection head to perform color patch detection. Its task type is object detection. It outputs the bounding box (BBox) and class probability of the color patch to identify the position and type of each color patch.

[0027] The Head3 detection head is used to detect key points at the four corners of the test strip. The task type is key point detection, and the coordinates (x, y) and visibility confidence of the four corner points of the test strip are output.

[0028] Using transfer learning and loading COCO pre-trained weights improves convergence efficiency;

[0029] After the model is trained, export it in ONNX format and compile it into an INT8 or FP16 accelerated model in the mobile inference engine TensorRT / OpenVINO.

[0030] Deploy the inference engine on the APP or embedded terminal to achieve real-time detection at 30fps.

[0031] Preferably, the second calibration module includes:

[0032] Perspective correction is achieved by calculating the homography matrix, and the mean of the color patches is extracted. The color space mapping matrix is ​​then fitted using least squares.

[0033] The transformation matrix H obtained by using four-point homography transformation includes:

[0034] H = A -1 B;

[0035] Where A is the detected corner coordinate matrix, and B is the standard rectangular coordinate matrix, which is a coordinate system generated by the standard test strip size;

[0036] The original image is transformed from a tilted / distorted state into a rectangular view with a fixed size and standard angles to facilitate colorimetric region extraction, color recognition, and subsequent analysis.

[0037] Preferably, the third calibration module is also used to eliminate color deviations caused by equipment and lighting, including extracting the RGB / Lab mean of the colorimetric patch in the image, establishing a mapping relationship with the preset standard value, performing color matrix transformation on the pixels of the perspective-corrected image, and outputting standardized colors;

[0038] The construction of the colorimetric mapping table includes:

[0039] {batch_id:{level1:lab1,level2:lab2,...}};

[0040] Where batch_id represents the number of a group of samples; level 1, level 2, ... represent the multiple label levels contained in the batch data; lab1, lab2, ... represent the feature values ​​under the level;

[0041] The standard values ​​include the Lab or sRGB values ​​of each colorimetric patch obtained by photographing standard test strips with a standard light source (D65) at the time of manufacture.

[0042] The color correction matrix M is solved using least squares fitting.

[0043] min M ‖X·MY‖ 2 ;

[0044] Where M is the mapping matrix, X is the sampled value matrix, and Y is the standard value matrix; ||X·MY|| 2 Min represents the sum of squares of the errors; M Let M represent the value that minimizes the error.

[0045] Preferably, the recognition module inputs the color-corrected test strip image into a hybrid deep learning model composed of a convolutional neural network and a Transformer, extracts local texture features and spatial length dependence features of the fluorescent region, and outputs a predicted value of the fluorescence intensity concentration of the target detection area, including:

[0046] The preprocessed fluorescence image is divided into several uniform small patches, denoted as Patch, including:

[0047] The test strip area is divided into N sub-regions of size H×W, forming an image patch sequence of shape (N,H,W,3), where 3 is due to the need to fuse three channels for fluorescence intensity calculation; each image patch is mapped to a fixed-dimensional feature vector space through a Patch Embedding layer, thereby converting the two-dimensional image patch into a one-dimensional visual word vector, which is convenient for Transformer processing;

[0048] These embedded vectors are fed into the Transformer encoder using a multi-head self-attention mechanism. The formula for the visual word vector Z is as follows:

[0049]

[0050] Where Q is the Query, representing the content that needs to be focused on in the current color patch image patch; K is the Key, identifying the information type contained in other image patches, including edges / texture / fluorescence; V is the Value, the actual feature information extracted from the image patch, including the original RGB values ​​and gradient features, obtained by linearly transforming the Patch Embedding vector; d k It is the dimension of the key vector, used to scale the dot product and prevent gradient vanishing;

[0051] Multiple layers of Encoder Blocks are stacked together, and higher-level semantic features are extracted layer by layer. Each layer can further integrate contextual information, so that the output feature vector not only contains local brightness information, but also integrates the spatial distribution trend and structural pattern of the entire test strip. The output global feature vector sequence is further processed, i.e., labeled or global average pooling, to obtain a comprehensive global feature representation, i.e., to obtain long-term dependency features across regions. At the same time, the local texture features extracted by CNN are also preserved.

[0052] The global features of the Transformer are concatenated with the local texture features of the CNN and input into a multilayer perceptron regression head, which finally outputs the quantitative concentration value of the fluorescence signal and its confidence interval.

[0053] Preferably, the judgment module generates an interpretable heatmap based on the intermediate features of the hybrid deep learning model to locate the fluorescence signal contribution region. The intermediate features include features from the Transformer self-attention weight map and the feature map of the last CNN convolutional layer relied upon by the Grad-CAM method. By overlaying the Transformer self-attention weight map and the Grad-CAM heatmap, a fused attention visualization map is generated based on the interpretable heatmap, intuitively displaying the key fluorescence regions that the model focuses on when making concentration judgments, enhancing the interpretability and credibility of the results, including:

[0054] Based on the model structure, the feature layers most representative of the decision-making process are selected as the analysis targets, including: the final output layer of the Transformer encoder, used to obtain the global attention distribution, or the feature map A of the last CNN convolutional layer. k Used to capture local spatially sensitive areas;

[0055] Grad-CAM is calculated in the following way for the selected target convolutional layer feature map A. k ∈R H×W×C Where R represents the data type of the feature map, H is the height, W is the width, and C is the number of channels. For each channel k (k = 1, 2, ..., C), and (i, j) represents the spatial location, the following operations are performed to calculate the gradient of the final predicted output y fluorescence concentration value relative to the feature map of each channel within the layer, including:

[0056]

[0057] in This refers to the partial derivative, A k The feature map of the k-th channel, α k This represents the importance weight of the k-th channel;

[0058] The feature maps of each channel are weighted and summed, and a spatial heatmap is generated using the ReLU activation function:

[0059] L Grad-CAM =ReLU(∑ k α k A k );

[0060] Among them, L Grad-CAM This refers to the heatmap generated by Grad-CAM; ReLU refers to the activation function.

[0061] L Grad-CAM Align the image pixel-wise with the input image, adjust the resolution using bilinear interpolation or upsampling, and then generate a pseudo-color or grayscale image, which is then overlaid on the original image to mark high-weight regions.

[0062] Preferably, the judgment module further includes extracting the average attention weight matrix from the last layer or multi-head attention from the Transformer model, which describes the degree of interrelation between the various image patches, including:

[0063]

[0064] Among them, Attention i Let be the attention weight matrix of the i-th attention head in the visual word vector Z; h is the number of attention heads. Since the size of the attention weight matrix is ​​usually the same as the length of the input sequence, upsampling is required to visualize it at the spatial resolution of the original image. This upsampling includes bilinear interpolation and nearest neighbor interpolation. After upsampling, an attention heatmap L with the same size as the original image is obtained. Attention .

[0065] Preferably, the judgment module further includes generating a fusion attention heatmap:

[0066] L Fusion =λ·L Grad-CAM +(1-λ)·L Attention ;

[0067] λ is an adjustable parameter used to balance the effects of local sensitivity and global attention.

[0068] On the other hand, the present invention also provides a deep learning-based method for visualizing fluorescence test strips, comprising:

[0069] Step S1: The captured fluorescent test strip image is detected by the target detection network to obtain the main area of ​​the test strip, the colorimetric area, and the coordinates of the key corner points of the test strip. The target detection network is a deep learning model specifically designed to identify specific objects in images or videos and accurately pinpoint their locations.

[0070] Step S2: Perform homography perspective transformation on the main area of ​​the test strip according to the coordinates of the key corner points of the test strip to generate a standard view of the test strip after perspective correction;

[0071] Step S3: Based on the actual color value of the colorimetric patch area and the preset standard colorimetric value, calculate the color mapping matrix, perform color space standardization correction on the perspective-corrected image, and output the color-corrected test paper image;

[0072] Step S4: Input the color-corrected test strip image into a hybrid deep learning model composed of a convolutional neural network and a Transformer, extract the local texture features and spatial length dependence features of the fluorescent region, and output the predicted value of the fluorescence intensity concentration of the target detection area;

[0073] Step S5: Generate an interpretable heatmap based on the intermediate features of the hybrid deep learning model to locate the fluorescence signal contribution region.

[0074] Compared with existing technologies, the advantages of this invention lie in constructing a complete intelligent visualization analysis system for fluorescent test strips by deeply integrating computer vision and deep learning technologies. The system first utilizes an improved YOLOv8 multi-task detection network to accurately locate the test strip body, colorimetric blocks, and key corner points, achieving robust perception under complex shooting conditions (such as different mobile phone models, uneven lighting, and tilted angles). Then, perspective correction is performed through homography transformation to eliminate geometric distortion and generate a standard orthogonal view. Further, based on the actual color values ​​of the colorimetric block area and preset standard values, the least squares method is used to fit a color mapping matrix, and the image undergoes color space standardization correction, effectively overcoming color deviations caused by device differences and ambient lighting, and improving cross-scene consistency. In the recognition stage, a deep learning model integrating CNN and Transformer is used. CNN extracts local texture features of the fluorescent area, while Transformer models long-distance spatial dependencies, achieving high-precision quantitative prediction of fluorescence signal intensity. An interpretable heatmap is generated through Grad-CAM and an attention mechanism, intuitively displaying the model's decision-making basis and significantly enhancing the credibility and interpretability of the results. Finally, the system binds the test results to the unique identifier of the test strip, such as a QR code, and uploads them to a cloud database, enabling centralized storage, batch management, and full-process traceability of test data. This invention comprehensively solves the bottleneck problems of traditional fluorescent test strip detection, such as low automation, weak quantitative ability, strong environmental sensitivity, and unreliable results. Attached Figure Description

[0075] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0076] Figure 1 A functional block diagram of a deep learning-based fluorescence test strip visualization system provided in an embodiment of the present invention;

[0077] Figure 2The implementation logic of the fusion attention heatmap of key regions of the generated fluorescence signal in the deep learning-based fluorescence test strip visualization system provided in this embodiment of the invention;

[0078] Figure 3 A schematic diagram of the detection procedure of the deep learning-based fluorescence test strip visualization system provided in an embodiment of the present invention;

[0079] Figure 4 A flowchart illustrating a deep learning-based visualization method for fluorescence test strips provided in an embodiment of the present invention. Detailed Implementation

[0080] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0081] See Figure 1 As shown, this embodiment proposes a deep learning-based fluorescence test strip visualization system, including:

[0082] The first calibration module uses a target detection network to detect the captured fluorescent test strip image and obtain the coordinates of the main area of ​​the test strip, the colorimetric area, and the key corner points of the test strip. The target detection network is a deep learning model specifically designed to identify specific objects in images or videos and accurately pinpoint their locations.

[0083] The second calibration module performs homography perspective transformation on the main area of ​​the test strip based on the coordinates of the key corner points of the test strip, and generates a standard view of the test strip after perspective correction.

[0084] The third calibration module calculates the color mapping matrix based on the actual color value of the colorimetric patch area and the preset standard colorimetric value, performs color space standardization correction on the perspective-corrected image, and outputs the color-corrected test strip image.

[0085] The recognition module inputs the color-corrected test strip image into a hybrid deep learning model composed of a convolutional neural network and a Transformer, extracts the local texture features and spatial length dependence features of the fluorescent region, and outputs the predicted value of the fluorescence intensity concentration of the target detection area.

[0086] The judgment module generates interpretable heatmaps based on intermediate features from a hybrid deep learning model, which are used to locate the contribution areas of fluorescence signals.

[0087] In addition, this embodiment also proposes to bind the test results with the unique identification information of the test strip and upload them to the cloud database to realize the storage, traceability and batch management of the test results.

[0088] See Figure 2 As shown, the original test strip image is the input data, i.e., the test strip image to be analyzed, which forms the basis of the entire analysis process. Strongly fluorescent regions refer to areas highlighted using Grad-CAM (Gradient-weighted Class Activation Mapping). Grad-CAM is a technique for visualizing the decision-making process of deep learning models, indicating the image regions that the model focuses on most when making predictions. In this context, it helps identify regions in the image with strong fluorescent signals, which are associated with specific chemical reactions. Weakly fluorescent regions refer to regions associated through a self-attention mechanism. Self-attention allows the model to capture dependencies between different positions in the input sequence, which can be understood in image processing as interactions between different pixels or regions. Here, it helps identify regions with weak fluorescent signals that still contribute to the overall reaction. Test strip edges refer to regions identified through gradient correlation. Gradient correlation refers to determining the importance of certain features or structures in an image based on gradient information. In this context, it is used to detect the edges of the test strip, which is crucial for accurately segmenting and locating the reaction area on the test strip. Fusion mapping refers to the final reaction core area map being obtained by fusing the results of the above three methods. This fusion can be achieved through weighted superposition, element-wise multiplication, etc., aiming to comprehensively consider local sensitivity (Grad-CAM), global attention distribution (self-attention mechanism), and structural information (gradient correlation), thereby more comprehensively and accurately identifying and interpreting key reaction areas in the test strip image.

[0089] See Figure 3 As shown, this embodiment uses MIPs (molecularly imprinted polymers) as carriers or recognition elements for fluorescent probes to selectively capture the target ion Cu. 2+ The visualization system of this embodiment is used to detect MIIMs. Since MIIMs are a visualization system based on the fluorescence colors of the blue response signal and the red internal standard signal, the B / R color channel mode is selected. The figure shows the RGB values ​​of the MIIMs fluorescence colors read in the detection software and the ratio in B / R mode by inputting the fluorescence images corresponding to Cu2+ concentrations of 20-100 nM.

[0090] Understandably, the visualization of fluorescent test strips is a crucial indicator of their application. MIIMs fluorescent test strips show no fluorescence emission under fluorescent light, and the film itself is light brown. Under 365nm UV excitation, both fluorescent test strips exhibited visible fluorescence color responses to different concentrations of Cu2+. While IIMs produced fluorescence color changes, compared to MIIMs, the lack of a magnetic guidance process resulted in some Fe3O4@SiO2@IIPs being encapsulated within the PVDF film. Insufficient surface recognition sites and background fluorescence interference made the fluorescence detection effect of IIMs less obvious and the color less bright, impacting subsequent smartphone-assisted detection. MIIMs fluorescent test strips exhibited bright and uniform fluorescence color, with more intuitive color changes. A comparison of surface SEM images of the two fluorescent test strips shows that the imprinted layer uniformly loaded on the base film surface based on surface imprinting technology effectively improves the visualization performance of the fluorescent test strip, making it suitable for on-site visual detection of Cu2+.

[0091] In the embodiments of this application, the recoveries of spiked Cu2+ in deionized water and tap water by MIIMs were 98.%-102% and 101%-104%, respectively, with RSDs of 1.3%-2.3% and 1.2%-2.2%, respectively. In the direct detection of electroplating wastewater, the detection concentrations of MIIMs were similar to those detected by ICP-MS. The comparative results show that smartphone-assisted MIIMs can accurately and quickly detect Cu2+ in actual samples on-site, with detection results approaching those of ICP-MS. This embodiment compares the results of Cu2+ concentration detection in actual samples using the MIIMs visualization method and the ICP-MS method in the following table:

[0092]

[0093] In some embodiments of this application, the first calibration module includes:

[0094] YOLOv8 network fine-tuning and lightweight deployment of TensorRT acceleration;

[0095] Input the original image containing the fluorescent test strip. Initial image features include mobile phone model, background, lighting, and shooting angle.

[0096] The detected coordinates of the main area of ​​the test strip, the colorimetric area, and the corner points are used for subsequent perspective correction and color calibration.

[0097] Specifically, the dataset construction includes the following data collection strategies: shooting with different brands of mobile phones such as iPhone, Huawei, and Xiaomi under various lighting conditions; environmental diversity including natural light (sunny / cloudy days), indoor lighting (LED / incandescent lamps), backgrounds (desktop, hands, paper, textured backgrounds), shooting angles (overhead, side tilt, and rotation ±45°); and a sample size of ≥5000 original images, with multiple targets labeled in each image.

[0098] In some embodiments of this application, the first calibration module further includes:

[0099] We constructed diverse datasets and fine-tuned YOLOv8, adding custom detection heads: test strip body, colorimetric blocks, and key points at the four corners of the test strip. We converted them to ONNX, compiled them into INT8 / FP16 using TensorRT, and deployed high frame rate inference on mobile devices. We collected test strip images taken in various environments and built a training set containing multiple angles, different lighting conditions, and different backgrounds.

[0100] This involves adding a custom detection head to the open-source YOLOv8 model, including:

[0101] The Head1 detection head is used to detect the main body of the test strip. Its task type is target detection. It outputs the bounding box (BBox) and confidence score of the main body of the test strip, which are used to locate the overall area of ​​the test strip in the image.

[0102] Enable the Head2 detection head to perform color patch detection. Its task type is object detection. It outputs the bounding box (BBox) and class probability of the color patch to identify the position and type of each color patch.

[0103] The Head3 detection head is used to detect key points at the four corners of the test strip. The task type is key point detection, and the coordinates (x, y) and visibility confidence of the four corner points of the test strip are output.

[0104] Using transfer learning and loading COCO pre-trained weights improves convergence efficiency;

[0105] After the model is trained, export it in ONNX format and compile it into an INT8 or FP16 accelerated model in the mobile inference engine TensorRT / OpenVINO.

[0106] Deploy the inference engine on the APP or embedded terminal to achieve real-time detection at 30fps.

[0107] Specifically, INT8 quantization requires a calibration set:

[0108] Prepare 100-300 unlabeled images for calibration;

[0109] Use Entropy Calibrator TensorRT by default;

[0110] Mobile deployment includes:

[0111] Android integration with TensorRT via JNI or using DeepJavaLib

[0112] iOS: Metal Performance Shaders+CoreML

[0113] Embedded systems: Jetson Nano / TX2 runs TensorRT inference at 30+ FPS.

[0114] In some embodiments of this application, the second calibration module includes:

[0115] Perspective correction is achieved by calculating the homography matrix, and the mean of the color patches is extracted. The color space mapping matrix is ​​then fitted using least squares.

[0116] The transformation matrix H obtained by using four-point homography transformation includes:

[0117] H = A -1 B;

[0118] Where A is the detected corner coordinate matrix, and B is the standard rectangular coordinate matrix, which is a coordinate system generated by the standard test strip size;

[0119] The original image is transformed from a tilted / distorted state into a rectangular view with a fixed size and standard angles to facilitate colorimetric region extraction, color recognition, and subsequent analysis.

[0120] Specifically, this embodiment provides the following example of the algorithm for implementing perspective correction:

[0121] import cv2

[0122] import numpy as np

[0123] def perspective_correction(image,corners,target_size=(300,100)):

[0124] #corners:[(x1,y1),(x2,y2),(x3,y3),(x4,y4)] are arranged clockwise.

[0125] src = np.float32(corners)

[0126] dst=np.float32([[0,0],[target_size[0],0],[target_size[0],target_size[1]],[0,target_size[1]]])

[0127] H=cv2.getPerspectiveTransform(src,dst)

[0128] corrected=cv2.warpPerspective(image,H,target_size)

[0129] return corrected,H

[0130] In some embodiments of this application, the third calibration module is also used to eliminate color deviations caused by devices and lighting, including extracting the RGB / Lab mean of the colorimetric patch in the image, establishing a mapping relationship with a preset standard value, performing color matrix transformation on the pixels of the perspective-corrected image, and outputting standardized colors.

[0131] The establishment of a standard value database Y includes standard values ​​obtained by photographing standard test strips using a standard light source (D65) at the time of manufacture, acquiring the Lab or sRGB values ​​of each colorimetric patch, and constructing a colorimetric mapping table.

[0132] {batch_id:{level1:lab1,level2:lab2,...}};

[0133] Where batch_id represents the number of a group of samples; level 1, level 2, ... represent the multiple label levels contained in the batch data; lab1, lab2, ... represent the feature values ​​under the level;

[0134] The color correction matrix M is solved using least squares fitting.

[0135] min M ‖X·MY‖ 2 ;

[0136] Where M is the mapping matrix, X is the sampled value matrix, and Y is the standard value matrix; ||X·MY|| 2 Min represents the sum of squares of the errors; M Let M represent the value that minimizes the error.

[0137] Understandably, the D65 standard light source is an illuminator that simulates sunlight and is widely used in color management, printing, photography, and display technology. It is designed to mimic natural sunlight conditions at midday in the Northern Hemisphere, with a color temperature of approximately 6500K. This indicates that it has a cool white tone and contains a significant amount of blue. The spectral distribution of the D65 light source is calculated based on a specific mathematical model to accurately reflect the characteristics of natural sunlight, including its radiant intensity at different wavelengths.

[0138] In practical applications, using D65 as a standard light source helps ensure color consistency and accuracy, whether in the color evaluation of physical samples or in the processing of digital images. By performing color matching and calibration under such standardized conditions, the impact of ambient light variations on color perception can be reduced, ensuring consistent color results across different times and locations.

[0139] Specifically, this embodiment provides the following example of the algorithm for extracting the RGB mean value of color patches:

[0140] block_regions = detect_color_blocks(corrected_img) # YOLO output or fixed ROI

[0141] X = []

[0142] for roiin block_regions:

[0143] mean_rgb = cv2.mean(roi)[:3]

[0144] X.append(mean_rgb)

[0145] X = np.array(X) # Shape: (N, 3)

[0146] In some embodiments of this application, the recognition module inputs the color-corrected test strip image into a hybrid deep learning model composed of a convolutional neural network and a Transformer, extracts local texture features and spatial length dependence features of the fluorescent region, and outputs a predicted value of the fluorescence intensity concentration of the target detection area, including:

[0147] The preprocessed fluorescence image is divided into several uniform small patches, denoted as Patch, including:

[0148] The test strip area is divided into N sub-regions of size H×W, forming an image patch sequence of shape (N,H,W,3), where 3 is due to the need to fuse three channels for fluorescence intensity calculation; each image patch is mapped to a fixed-dimensional feature vector space through a Patch Embedding layer, thereby converting the two-dimensional image patch into a one-dimensional visual word vector, which is convenient for Transformer processing;

[0149] These embedded vectors are fed into the Transformer encoder using a multi-head self-attention mechanism. The formula for the visual word vector Z is as follows:

[0150]

[0151] Where Q is the Query, representing the content that needs to be focused on in the current color patch image patch; K is the Key, identifying the information type contained in other image patches, including edges / texture / fluorescence; V is the Value, the actual feature information extracted from the image patch, including the original RGB values ​​and gradient features, obtained by linearly transforming the Patch Embedding vector; d k This is the dimension of the key vector, used to scale the dot product and prevent gradient vanishing, where:

[0152]

[0153] Furthermore, the dimension of Q is the query requirements of all image patches in the range [N·h,d_k], such as needing to focus on fluorescence intensity; the dimension of K is the feature identifiers of all image patches in the range [N·h,d_k], such as strong fluorescence texture; and the dimension of V is the actual feature values ​​of all image patches in the range [N·h,d_v], such as RGB gradient, where N is the number of scalar image patches, i.e., the number of test strip blocks; and h is the number of attention heads.

[0154] Multiple layers of Encoder Blocks are stacked together, and higher-level semantic features are extracted layer by layer. Each layer can further integrate contextual information, so that the output feature vector not only contains local brightness information, but also integrates the spatial distribution trend and structural pattern of the entire test strip. The output global feature vector sequence is further processed, i.e., labeled or global average pooling, to obtain a comprehensive global feature representation, i.e., to obtain long-term dependency features across regions. At the same time, the local texture features extracted by CNN are also preserved.

[0155] The global features of the Transformer are concatenated with the local texture features of the CNN and input into a multilayer perceptron regression head, which finally outputs the quantitative concentration value of the fluorescence signal and its confidence interval.

[0156] Specifically, stacking multiple Encoder Blocks refers to sequentially connecting multiple Transformer Encoder Blocks with identical structures and progressive functions to form a deep neural network structure. This structure is used to extract and abstract high-level semantic features of the input image layer by layer. The Transformer Encoder Block is the basic building block of the Transformer model, containing two core components: a multi-head self-attention mechanism and a feed-forward network (FFN). In this application, the multi-head self-attention mechanism is used to capture the global dependencies between different parts of the input sequence. The input is the feature sequence after image patching. This mechanism can model the fluorescence intensity correlation between different regions of the test strip (such as gradient changes, symmetry, etc.). The feed-forward network (FFN) performs non-linear transformations on the features at each location, enhancing the model's expressive power. Each sub-layer is followed by a residual connection and layer normalization to ensure stable training of the deep network.

[0157] It is understandable that stacking multiple encoder blocks refers to sequentially connecting multiple Transformer encoder modules, including multi-head self-attention mechanisms and feedforward neural networks, to form a deep feature extraction network. This network is used to model the long-distance spatial dependencies between regions in a fluorescent test strip image layer by layer, achieving semantic abstraction from local brightness to global distribution patterns. As the number of layers increases, the feature representation gradually evolves from local brightness to global structure. The following table is provided as an example in this embodiment:

[0158]

[0159] Among them, it is understandable that z i (N) This represents the feature vector generated at the Nth layer; the initial feature vector z i (1) and z i (2) These vectors mainly contain local brightness information; the intermediate feature vector z i (3) To z i (6) These vectors begin to integrate a wider range of spatial information, resulting in high-level feature vectors z. i (7) To z i (12)These vectors contain the global spatial distribution patterns. The final output is a global feature vector sequence Z = [z1, z2, ..., z...]. N ]In each z i These are all final feature vectors obtained after processing by multiple layers of encoders, which integrate contextual information from all levels.

[0160] In some embodiments of this application, the judgment module generates an interpretable heatmap based on intermediate features of a hybrid deep learning model to locate the fluorescence signal contribution region. The intermediate features include features from the Transformer self-attention weight map and the feature map of the last CNN convolutional layer relied upon by the Grad-CAM method. By overlaying the Transformer self-attention weight map and the Grad-CAM heatmap, a fused attention visualization map is generated based on the interpretable heatmap, intuitively displaying the key fluorescence regions that the model focuses on when making concentration judgments, enhancing the interpretability and credibility of the results, including:

[0161] Based on the model structure, the feature layers most representative of the decision-making process are selected as the analysis targets, including: the final output layer of the Transformer encoder, used to obtain the global attention distribution, or the feature map A of the last CNN convolutional layer. k Used to capture local spatially sensitive areas;

[0162] Grad-CAM is calculated in the following way for the selected target convolutional layer feature map A. k ∈R H×W×C Where R represents the data type of the feature map, H is the height, W is the width, and C is the number of channels. For each channel k (k = 1, 2, ..., C), and (i, j) represents the spatial location, the following operations are performed to calculate the gradient of the final predicted output y fluorescence concentration value relative to the feature map of each channel within the layer:

[0163]

[0164] in This refers to the partial derivative, A k The feature map of the k-th channel, α k This represents the importance weight of the k-th channel;

[0165] The feature maps of each channel are weighted and summed, and a spatial heatmap is generated using the ReLU activation function:

[0166] L Grad-CAM =ReLU(∑ k α k A k );

[0167] Among them, LGrad-CAM This refers to the heatmap generated by Grad-CAM; ReLU refers to the activation function.

[0168] L Grad-CAM Align the image pixel-wise with the input image, adjust the resolution using bilinear interpolation or upsampling, and then generate a pseudo-color or grayscale image, which is then overlaid on the original image to mark high-weight regions.

[0169] In this embodiment, the following example is given for the implementation algorithm of Grad-CAM+Attention visualization:

[0170]

[0171]

[0172] In some embodiments of this application, the determination module further includes extracting the average attention weight matrix of the last layer or multi-head attention from the Transformer model. The matrix describes the degree of correlation between the individual image patches, including:

[0173]

[0174] Among them, Attention i Let be the attention weight matrix of the i-th attention head in the visual word vector Z; h is the number of attention heads. Since the size of the attention weight matrix is ​​usually the same as the length of the input sequence, upsampling is required to visualize it at the spatial resolution of the original image. This upsampling includes bilinear interpolation and nearest neighbor interpolation. After upsampling, an attention heatmap L with the same size as the original image is obtained. Attention .

[0175] Specifically, Attention i It is obtained by calculation using the following formula:

[0176]

[0177] Where i is the attention head index, ranging from 1 to h; Q i It is the query matrix of the i-th head; It is the transpose of the key matrix of the i-th head; d k The key vector dimension is typically 64.

[0178] In some embodiments of this application, the determination module further includes generating a fusion attention heatmap:

[0179] L Fusion =λ·L Grad-CAM +(1-λ)·L Attention ;

[0180] λ is an adjustable parameter used to balance the effects of local sensitivity and global attention.

[0181] Understandably, this approach allows us to combine local and global information to gain a more comprehensive understanding of the model's decision-making process and improve the interpretability and credibility of the results.

[0182] See Figure 4 As shown, this embodiment also provides a deep learning-based visualization method for fluorescence detection test strips, including:

[0183] Step S1: The captured fluorescent test strip image is detected by the target detection network to obtain the main area of ​​the test strip, the colorimetric area, and the coordinates of the key corner points of the test strip. The target detection network is a deep learning model specifically designed to identify specific objects in images or videos and accurately pinpoint their locations.

[0184] Step S2: Perform homography perspective transformation on the main area of ​​the test strip based on the coordinates of the key corner points of the test strip to generate a standard view of the test strip after perspective correction;

[0185] Step S3: Based on the actual color value of the colorimetric patch area and the preset standard colorimetric value, calculate the color mapping matrix, perform color space standardization correction on the perspective-corrected image, and output the color-corrected test paper image;

[0186] Step S4: Input the color-corrected test strip image into a hybrid deep learning model composed of a convolutional neural network and a Transformer, extract the local texture features and spatial length dependence features of the fluorescent region, and output the predicted value of the fluorescence intensity concentration of the target detection area.

[0187] Step S5: Generate an interpretable heatmap based on the intermediate features of the hybrid deep learning model to locate the fluorescence signal contribution region.

[0188] In addition, this embodiment also proposes to bind the test results with the unique identification information of the test strip and upload them to the cloud database to realize the storage, traceability and batch management of the test results.

[0189] The beneficial effect of this embodiment lies in the construction of a complete intelligent visualization analysis system for fluorescent test strips by deeply integrating computer vision and deep learning technologies. First, an improved YOLOv8 multi-task detection network is used to accurately locate the main body of the test strip, the colorimetric blocks, and key corner points, achieving robust perception under complex shooting conditions (such as different mobile phone models, uneven lighting, and tilted angles). Then, perspective correction is performed through homography transformation to eliminate geometric distortion and generate a standard orthogonal view. Further, based on the actual color values ​​of the colorimetric block areas and preset standard values, the least squares method is used to fit a color mapping matrix, and the image undergoes color space standardization correction, effectively overcoming color deviations caused by device differences and ambient lighting, and improving cross-scene consistency. In the recognition stage, a deep learning model integrating CNN and Transformer is used. CNN extracts local texture features of the fluorescent area, while Transformer models long-distance spatial dependencies, achieving high-precision quantitative prediction of fluorescence signal intensity. An interpretable heatmap is generated through Grad-CAM and an attention mechanism, intuitively displaying the model's decision-making basis and significantly enhancing the credibility and interpretability of the results. Finally, the system binds the test results to the unique identifier of the test strip, such as a QR code, and uploads them to a cloud database, enabling centralized storage, batch management, and full-process traceability of test data. This invention comprehensively solves the bottleneck problems of traditional fluorescent test strip detection, such as low automation, weak quantitative ability, strong environmental sensitivity, and unreliable results.

[0190] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0191] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0192] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0193] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A deep learning based fluorescent test strip visualization system, characterized in that, The method comprises the following steps: A first calibration module detects the fluorescence test paper image obtained by shooting through a target detection network, obtains the test paper main body area, colorimetric block area, and test paper key corner point coordinates; A second calibration module performs homographic perspective transformation on the test paper main body area according to the test paper key corner point coordinates to generate a perspective-corrected test paper standard view; A third calibration module calculates a color mapping matrix based on the actual color value of the colorimetric block area and the preset standard colorimetric value, performs color space standardization correction on the perspective-corrected image, and outputs a color-corrected test paper image; An identification module inputs the color-corrected test paper image into a hybrid deep learning model composed of a convolutional neural network and a Transformer, extracts local texture features and spatial long dependence features of the fluorescence region, and outputs a fluorescence intensity concentration prediction value of the target detection area; A judgment module generates an explainability heat map based on the intermediate features of the hybrid deep learning model, which is used to locate the fluorescence signal contribution area. The first calibration module further comprises: A diversified data set is constructed and YOLOv8 is fine-tuned, and a custom detection head is added: test paper body, colorimetric block, and test paper four-corner key points, which are converted into ONNX, compiled into INT8 / FP16 using TensorRT, and deployed on a mobile terminal for high-frame-rate inference; test paper images taken in multiple environments are collected to establish a training set containing multiple angles, different illuminations, and different backgrounds; the addition of the custom detection head includes: The Head1 detection head detects the test paper body, the task type is target detection, and the boundary box BBox and confidence of the test paper body are outputted to locate the overall area of the test paper in the image; The Head2 detection head detects the colorimetric block, the task type is target detection, and the boundary box BBox and class probability of the colorimetric block are outputted to identify the position and type of each colorimetric block; The Head3 detection head detects the test paper four-corner key points, the task type is key point detection, and the coordinates (x, y) and visibility confidence of the test paper four-corner key points are outputted; Transfer learning is used to load the COCO pre-trained weights to improve the convergence efficiency; After the model is trained, the ONNX format is exported, and the INT8 or FP16 acceleration model is compiled in the mobile terminal inference engine TensorRT / OpenVINO; The inference engine is deployed on the APP terminal or embedded terminal to realize 30fps real-time detection.

2. The deep learning-based visualization system for fluorescent test strips according to claim 1, wherein, The first calibration module comprises: YOLOv8 network fine-tuning and lightweight deployment TensorRT acceleration; The original shooting image containing the fluorescence test paper is inputted, and the initial features of the picture include the mobile phone model, background, illumination, and shooting angle; The detected test paper main body area, colorimetric block area, and corner point coordinates are used for subsequent perspective correction and color correction.

3. The deep learning-based visualization system for fluorescent test strips according to claim 2, wherein, The second calibration module comprises: Perspective correction is realized using homographic matrix calculation, and the colorimetric block mean value is extracted to least squares fit the color space mapping matrix; Wherein, using four-point homography transformation, the transformation matrix is obtained comprising: ； wherein A is the detected corner point coordinate matrix, and B is the standard rectangular coordinate matrix generated from the standard test paper size; Applying to the original image, converting from a tilted / distorted state into a fixed-size, angle-standard rectangular view, so as to facilitate color area extraction, color identification and subsequent analysis.

4. The deep learning-based visualization system for fluorescent test strips according to claim 3, wherein, The third calibration module is further configured to eliminate color deviation caused by equipment and illumination, including extracting RGB / Lab average values of the color blocks in the image, establishing a mapping relationship with preset standard values, performing color matrix transformation on pixels of the perspective-corrected image, and outputting standardized colors. The mapping table is constructed by: {batch_id:{level1:lab1,level2:lab2,...}}; Wherein, batch_id represents the number of a group of samples; level1, level2,... represent a plurality of label levels contained in the batch data; lab1, lab2,... represent the characteristic values at the level; The preset standard values include using a standard light source D65 to take a standard test paper at the time of factory shipment, and obtaining Lab or sRGB values of each color block; Solving a color correction matrix using least squares fitting ; ； wherein, is a mapping matrix, X is a matrix of sample values, and Y is a matrix of standard values; denotes a sum of squares of errors; denotes M that minimizes the error.

5. The deep learning-based visualization system for fluorescent test strips according to claim 4, wherein, The recognition module inputs the color-corrected test paper image into a hybrid deep learning model composed of a convolutional neural network and a Transformer, extracts local texture features and spatial long dependence features of the fluorescent region, and outputs a fluorescence intensity concentration prediction value of the target detection area, including: The preprocessed fluorescent image is divided into several uniform small blocks, denoted as Patch, including: The test paper region is divided into N sub-regions with a size of , and an image block sequence with a shape of is formed, wherein 3 is 3 channels to be fused for fluorescence intensity calculation; each image block is mapped to a fixed-dimensional feature vector space through a Patch Embedding layer, so as to convert the two-dimensional image block into a one-dimensional visual word vector, facilitating the processing of the Transformer; Through the multi-head self-attention mechanism, the embedded vectors are input into the Transformer encoder to calculate the visual word vector Z, and the formula is as follows: ； Wherein, Q is Query representing the content that the current color block image block needs to pay attention to; K is Key indicating the information type contained in other image blocks, including edge / texture / fluorescence; V is Value representing the actual feature information extracted from the image block, including original RGB value, gradient feature, and the feature information obtained by linear transformation on the Patch Embedding vector; is the dimension of the Key vector, used for scaling the dot product to prevent gradient disappearance; Stack multiple layers of Encoder Block, multiple Transformer Encoder Block are stacked together, and higher-level semantic features are extracted layer by layer, and each layer can further integrate context information, so that the output feature vector not only contains local brightness information, but also fuses the spatial distribution trend and structure rule of the entire test paper. The output global feature vector sequence is further processed, i.e. labeled or global average pooling, to obtain a comprehensive global feature representation, i.e. to obtain cross-regional long dependence features, and also retain the local texture features extracted by CNN; The global features of the Transformer and the local texture features of the CNN are spliced and input into a multi-layer perceptron regression head to output the quantitative concentration value of the fluorescent signal and its confidence interval.

6. The deep learning-based visualization system for fluorescent test strips according to claim 5, wherein, The judgment module generates an explainability heat map based on the intermediate features of the hybrid deep learning model, which is used to locate the fluorescent signal contribution area, and the intermediate features include features in the Transformer self-attention weight map and feature maps of the last CNN convolution layer relied on by the Grad-CAM method; by superimposing the Transformer self-attention weight map and the Grad-CAM heat map, a fusion attention visualization map is generated based on the explainability heat map, which intuitively displays the key fluorescent regions that the model focuses on when judging the concentration, enhances the explainability and reliability of the results, including: According to the model structure, the feature layer most representative of the decision is selected as the analysis target, including: the final output layer of the Transformer encoder for obtaining the global attention distribution or the feature map Ak of the last CNN convolution layer for capturing the local spatial sensitive area; where Grad-CAM is calculated by the following way, for the selected target convolutional layer feature map where R is the data type representing the feature map, H is the height, W is the width, and C is the number of channels, for each channel k (k = 1, 2,..., C), and in addition (i, j) represents the spatial position, the following operation is performed to calculate the gradient of the final prediction output yfluorescence concentration value with respect to each channel feature map within the layer, including: ； wherein denotes the partial derivative, the feature map of the kth channel, ak denotes the importance weight of the kth channel, The channel feature maps are weighted and summed, and a spatial heat map is generated through a ReLU activation function: ； wherein, Grad-CAM refers to a heat map generated by Grad-CAM; ReLU refers to an activation function. Will Align with the input image by pixel, adjust the resolution through bilinear interpolation or up-sampling, then generate a pseudo-color map or grayscale map, superimposed on the original map, indicating the high weight area.

7. The deep learning-based visualization system for fluorescent test strips according to claim 6, wherein, The judgment module further includes extracting the average attention weight matrix of the last layer or multi-head attention from the Transformer model, which describes the degree of mutual association between image patches, including: ； wherein, is the attention weight matrix of the i-th attention head in the visual word vector Z; h is the number of attention heads, since the size of the attention weight matrix is usually the same as the length of the input sequence, in order to visualize it to the spatial resolution of the original image, an upsampling operation is needed, including bilinear interpolation, nearest neighbor interpolation, after upsampling, an attention heat map with the same size as the original image is obtained .

8. The deep learning-based visualization system for fluorescent test strips according to claim 7, wherein, The judgment module further includes generating a fusion attention heat map: ； Where λ is an adjustable parameter for balancing the influence of local sensitivity and global attention.

9. A deep learning-based fluorescent test strip visualization method applied to the deep learning-based fluorescent test strip visualization system according to any one of claims 1-8, characterized in that, Including: Step S1: detecting the fluorescence test paper image obtained by shooting through the target detection network, obtaining the test paper main body area, color block area and test paper key corner coordinates; Step S2: performing homographic perspective transformation on the test paper main body area according to the test paper key corner coordinates to generate a perspective corrected test paper standard view; Step S3: based on the actual color value of the color block area and the preset standard color value, calculating the color mapping matrix, correcting the color space of the perspective corrected image, and outputting the color corrected test paper image; Step S4: inputting the color corrected test paper image into a hybrid deep learning model composed of a convolutional neural network and a Transformer to extract local texture features and spatial long dependence features of the fluorescence region, and outputting the fluorescence intensity concentration prediction value of the target detection area; Step S5: generating an explainable heat map based on the intermediate features of the hybrid deep learning model for positioning the fluorescence signal contribution area.

Images