Multiplexed immunofluorescence assay method for detecting multiple microbial pathogens

Abstract

The present application relates to the technical field of pathogen optical detection, and discloses a multiplex immunofluorescence analysis method for detecting various microbial pathogens. The method comprises acquiring multi-modal fluorescence data on the surface of a sample through a biomimetic visual perception system, and generating a pathogen surface distribution heat map by using a deep neural network to identify high-risk areas. According to the area division result, the system dynamically regulates the optical detection array to emit coded light signals to the sample interior, and reconstructs the internal three-dimensional structure of the pathogen by analyzing the fluorescence response. Through a space-time alignment mechanism, the surface and internal data are integrated, and a graph neural network is used for fusion analysis to generate a detection confidence score to drive adaptive decision-making. The present method realizes accurate and dynamic detection of pathogens from the surface to the interior, and improves the detection depth and result reliability of hidden targets in complex samples.

Description

Technical Field

[0001] This invention relates to the field of optical detection technology for pathogens, specifically a multiplex immunofluorescence analysis method for detecting various microbial pathogens. Background Technology

[0002] Currently, pathogen detection methods based on immunofluorescence technology are widely used. These methods typically rely on high-resolution microscopy or flow cytometry to image or scan samples, identifying and locating pathogens by analyzing the intensity and location of fluorescence signals. However, conventional techniques are mainly limited to capturing and analyzing two-dimensional fluorescence information on the sample surface or a fixed focal plane, and their detection modes are often static and passive. The system scans the entire sample area according to preset, uniform parameters, and cannot adaptively adjust according to the real-time characteristics of the sample.

[0003] Existing technologies have shortcomings. Due to the inherent light scattering characteristics and complexity of biological samples, imaging the internal structure or deep distribution of pathogens presents challenges. Homogeneous scanning strategies result in low detection efficiency, with significant resources wasted on low-risk or meaningless areas, while high-risk areas requiring focused attention may suffer from insufficient depth and loss of detail due to signal attenuation. Current analytical methods typically treat surface two-dimensional information separately from internal three-dimensional information, or simply overlay them, lacking an effective mechanism to accurately correlate and fuse data from different dimensions and time points. This data-level disconnect makes it difficult to dynamically understand the spatial distribution patterns, invasion processes, and interactions with the host of pathogens, limiting the accuracy, depth, and insight into biological processes. Summary of the Invention

[0004] The purpose of this invention is to provide a multiplex immunofluorescence assay method for detecting various microbial pathogens, thereby addressing the problems mentioned in the background section.

[0005] To achieve the above objectives, the present invention provides a multiplex immunofluorescence assay method for detecting various microbial pathogens, the method comprising:

[0006] A biomimetic visual perception system was used to dynamically capture multimodal fluorescence data of pathogens;

[0007] Multimodal fluorescence data is input into a deep neural network model to generate a pathogen surface distribution heatmap, and the pathogen surface distribution heatmap is divided into regions based on a preset risk threshold;

[0008] Based on the regional division results, an optical detection array is configured and coded light signals are emitted into the sample. After receiving the internal fluorescence response, the three-dimensional distribution of the pathogen inside the sample is calculated through a reconstruction algorithm.

[0009] Establish a spatiotemporal alignment between the pathogen surface distribution heatmap and the pathogen's internal three-dimensional distribution, and use a clock synchronization mechanism to align time information;

[0010] A graph neural network is used to fuse spatiotemporally aligned distribution data to generate a detection confidence score; a machine learning model is then applied to convert the detection confidence score into a detection instruction.

[0011] Monitor the changes in sample status after the execution of the detection command, calculate the deviation of the status change, generate the effectiveness index of the detection strategy, and dynamically adjust the detection parameters.

[0012] Preferably, the method of dynamically capturing multimodal fluorescence data of samples using a biomimetic visual perception system includes:

[0013] The pathogens include wheat dwarf smut, wheat leaf blight, wheat chrysophagus, onion pink root rot, sorghum root rot, grape stem blight, and cruciferous vegetable blackleg.

[0014] Multispectral imaging equipment, fluorescence microscope, and spectrometer are deployed to simultaneously acquire the fluorescence intensity distribution, spatial morphological characteristics, and spectral characteristics of samples. The focus adjustment mechanism of the biological visual system is simulated to divide the sample surface into high-resolution and low-resolution regions. The high-resolution region corresponds to the potential aggregation site of pathogens and adopts a dense sampling strategy, while the low-resolution region adopts a sparse sampling strategy. The feature changes in the acquired data are analyzed in real time, and the range of the high-resolution region is dynamically adjusted. When new feature changes are found, the high-resolution region is expanded, and when the feature disappears, the high-resolution region is shrunk.

[0015] Preferably, the step of inputting multimodal fluorescence data into a deep neural network model to generate a pathogen surface distribution heatmap includes:

[0016] Multimodal fluorescence data are standardized and preprocessed to merge into multi-channel data blocks; a convolutional neural network with an encoder-decoder structure is constructed, where the encoder extracts multi-scale features through convolution operations and the decoder recovers spatial dimensions through deconvolution operations; cross-layer connections are added between the encoder and decoder to transfer local features; and the output layer uses an activation function to generate a pathogen surface distribution heatmap.

[0017] Preferably, the step of dividing the pathogen surface distribution heatmap into regions based on a preset risk threshold includes:

[0018] The risk value of each pixel region in the pathogen surface distribution heatmap is calculated. The risk value is calculated based on a combination of fluorescence intensity, texture complexity, and gradient change. Low and high risk thresholds are set. Regions with risk values ​​below the low risk threshold are classified as low-risk regions, regions with risk values ​​between the low and high risk thresholds are classified as medium-risk regions, and regions with risk values ​​above the high risk threshold are classified as high-risk regions. Different colors are used to mark the classification results.

[0019] Preferably, the step of configuring an optical detection array according to the regional division results and emitting coded light signals into the sample, and then calculating the three-dimensional distribution of the pathogen inside the sample by reconstructing the internal fluorescence response includes: setting the distribution density of optical detection points according to the risk level, with the smallest spacing between detection points in high-risk areas, followed by medium-risk areas, and the largest spacing in low-risk areas; using a spatial light modulator to generate a sequence of coded light patterns and projecting it into the sample; collecting fluorescence signals and recording time-series data through a photoelectric sensor array; and using an iterative algorithm based on a light transmission model to solve for the distribution of internal fluorescence sources and generate three-dimensional grid data.

[0020] Preferably, the spatiotemporal alignment of establishing the pathogen surface distribution heatmap with the pathogen's internal three-dimensional distribution includes:

[0021] A world coordinate system is established by setting reference points on the sample surface; the internal and external parameters of the optical sensor are calibrated using calibration tools, and the surface distribution heat map of the pathogen is converted to the world coordinate system; time stamps are added to the three-dimensional distribution data inside the pathogen, and the clock signals of all sensors are synchronized using a network time protocol.

[0022] Preferably, the step of using a graph neural network to fuse spatiotemporally aligned distribution data to generate a detection confidence score includes:

[0023] Each risk region of the pathogen surface distribution heatmap is used as a surface node, and each voxel unit of the pathogen's internal three-dimensional distribution is used as an internal node to construct a heterogeneous graph structure. Feature vectors of surface nodes and internal nodes are extracted as node attributes. The graph neural network contains multiple layers of graph convolution operations, and each layer aggregates information from adjacent nodes to update the node state. Finally, the detection confidence score is output through the classification layer.

[0024] Preferably, the construction of the heterogeneous graph structure includes:

[0025] Calculate the spatial distance between surface nodes and internal nodes. If the distance is less than the connection threshold, establish an edge connection. At the same time, calculate the similarity of node feature vectors. If the similarity is higher than the similarity threshold, add an edge. The weight of the edge is weighted according to the distance and similarity.

[0026] Preferably, the application of the machine learning model to convert the detection confidence score into a detection instruction includes:

[0027] Collect historical detection confidence scores and corresponding operation instructions to train a support vector machine classifier; input the current detection confidence score into the support vector machine classifier and output the detection instruction type; the detection instruction type includes confirmation instruction, exclusion instruction and retest instruction.

[0028] Preferably, the process of monitoring sample state changes after the execution of the detection command, calculating state change deviations, generating detection strategy effectiveness indicators, and dynamically adjusting detection parameters includes:

[0029] Continuously acquire sample fluorescence image sequences and calculate the difference value between adjacent image frames; compare the difference value with a benchmark threshold to obtain the deviation ratio; calculate the detection strategy effectiveness index based on the deviation ratio; when the detection strategy effectiveness index is lower than the adjustment threshold, modify the optical detection parameters, including excitation light intensity and exposure time.

[0030] Compared with the prior art, the beneficial effects of the present invention are:

[0031] By dynamically configuring the optical detection array and emitting coded light signals based on the regional division results of surface distribution heatmaps, a shift from wide-area screening to precise detection of key areas has been achieved. This proactive and configurable detection method prioritizes limited detection resources to areas initially identified as high-risk, emitting optimized coded signals into these areas. Through modulation and demodulation techniques, the coded light signals effectively suppress background noise, improving penetration and signal-to-noise ratio in deep biological tissues. Subsequent reconstruction algorithms utilize the characteristics of the coded signals to more accurately resolve the three-dimensional spatial coordinates and morphological structure of pathogens within the sample. This transforms the detection process from a blind, full-area scan into a targeted, efficient, cross-scale detection, enhancing the ability to detect occult infections.

[0032] By establishing a spatiotemporal alignment mechanism between surface distribution and internal three-dimensional distribution, and employing a graph neural network to process the fused data, deep integration and intelligent analysis of multimodal data were achieved. A clock synchronization mechanism ensured the consistency of surface and internal data over time, laying the foundation for dynamic analysis. The graph neural network constructed a graph structure model of spatially discrete pathogen fluorescence signal points or regions, with nodes representing signal sources and edges representing spatial or potential functional associations. This model can effectively learn complex spatial topological relationships and non-Euclidean spatial features, capturing distribution patterns such as pathogen aggregation and diffusion. The detection confidence score generated based on this deep understanding of spatial relationships goes beyond simple fluorescence intensity or morphological parameters, incorporating richer spatial contextual information, thus making the final output detection command more accurate and reliable. Attached Figure Description

[0033] Figure 1 This is a schematic diagram illustrating the working principle of the multiplex immunofluorescence assay method for detecting various microbial pathogens described in this invention.

[0034] Figure 2 A flowchart of deep neural network processing for generating pathogen surface distribution heatmaps;

[0035] Figure 3 A flowchart for optical detection and reconstruction to calculate the three-dimensional distribution inside a pathogen;

[0036] Figure 4 This includes a heat map of pathogen surface distribution and a risk zone delineation map;

[0037] Figure 5 The distribution of confidence scores for pathogen detection and the classification threshold are plotted. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] Please see Figure 1This invention provides a multiplex immunofluorescence analysis method for detecting various microbial pathogens. The method includes: dynamically capturing multimodal fluorescence data of pathogens using a biomimetic visual perception system. This system simulates the perception mechanism of biological vision and can adaptively adjust the observation focus and resolution. The captured multimodal fluorescence data is then input into a deep neural network model, which is trained to generate a heatmap reflecting the probability of pathogen distribution on the sample surface. Based on a preset risk threshold, the surface distribution heatmap is divided into regions, identifying high-risk, medium-risk, and low-risk areas. Based on the region division results, the parameters of the optical detection array are configured to emit encoded light signals into the sample interior and receive response signals generated by internal fluorescent substances. These response signals are processed by a reconstruction algorithm to calculate the three-dimensional distribution of pathogens within the sample. A spatiotemporal alignment relationship is established between the pathogen surface distribution heatmap and the internal three-dimensional distribution to ensure consistency between surface and internal data in spatial coordinates and time. A graph neural network is used to perform fusion analysis on the spatiotemporally aligned distribution data to generate a comprehensive detection confidence score. A machine learning model is applied to convert this score into specific detection instructions, such as confirming the presence of pathogens, excluding suspected signals, or requiring retesting. The system monitors the state changes of the samples after the detection command is executed, calculates the deviation between the actual state change and the expectation, generates an effectiveness index of the detection strategy based on this, and dynamically adjusts the key parameters in subsequent detections based on the index to form a closed-loop optimization.

[0040] Example 1: See Figure 2 In its implementation, the biomimetic visual perception system targets pathogens including wheat dwarf smut, wheat leaf blight, wheat hull spot, onion pink root rot, sorghum root rot, grape stem blight, and cruciferous vegetable blackleg. The optical characteristics of these pathogens form the basis for the system's design and operation. A deployed multispectral imaging device captures the fluorescence intensity distribution of the sample at different wavelengths, a fluorescence microscope provides detailed spatial morphological features at high magnification, and a spectrometer analyzes the fine spectral characteristics of the fluorescence. These three devices achieve strict synchronization of acquisition actions at the hardware level through a synchronous signal trigger, ensuring the consistency of multimodal fluorescence data over time. The core of the biomimetic visual perception system lies in simulating the focus adjustment mechanism of biological visual systems. In its implementation, during the system initialization phase, based on a pre-stored knowledge base or a rapid panoramic scan, the sample surface is divided into high-resolution and low-resolution regions. The high-resolution region is designated as potential pathogen aggregation sites, such as known susceptible areas on crop leaves or areas showing early symptoms, while the low-resolution region covers the rest of the sample surface.

[0041] In practical implementation, for high-resolution areas, the system employs a dense sampling strategy. This strategy involves using higher pixel resolution for imaging spatially and capturing consecutive frames at shorter intervals temporally to obtain rich dynamic change information. For low-resolution areas, a sparse sampling strategy is used, which reduces spatial resolution and the acquisition frequency per unit time to balance system processing load and total data volume. The biomimetic visual perception system does not operate statically. In practical implementation, the system's built-in real-time analysis module continuously processes data streams from multispectral imaging devices, fluorescence microscopes, and spectrometers. The core function of the real-time analysis module is to identify spectral signal mutations or morphological anomalies associated with pathogen characteristics. When new feature changes are detected within the current scanning range, such as a signal matching the fluorescence characteristics of wheat dwarf smut fungus detected at the edge of a previously low-resolution area, the system immediately triggers a high-resolution area expansion operation. This expansion operation involves redefining the boundary of the region of interest and incorporating the new region into the high-resolution acquisition mode. Conversely, when the intensity of the characteristic signal continuously monitored in a certain high-resolution area decays to the background level or the characteristic morphology disappears, it indicates that the pathogen activity in the area has weakened or the previous judgment was incorrect. The system will perform a high-resolution area shrinkage operation, which will release the computing and acquisition resources of the area and downgrade it to a low-resolution area for monitoring.

[0042] In some embodiments, the division between high-resolution and low-resolution regions is not completed in one step, but rather through a dynamic iterative process. The real-time analysis module calculates a dynamic priority score for each monitored region. This priority score integrates real-time fluorescence intensity, historical trends, and the degree of matching with pathogen characteristic templates. The system dynamically allocates acquisition resources based on the priority score, ensuring that it always focuses on the most suspicious region. This adaptive focus adjustment mechanism of the biomimetic visual perception system can be understood to optimize resource utilization, avoid unnecessary fine scanning of sample regions without significant features, thereby improving overall detection efficiency. Simultaneously, by dynamically responding to feature changes, it reduces the possibility of missing emerging risk points. In specific implementations, the raw data acquired by multispectral imaging equipment, fluorescence microscopes, and spectrometers need to be preprocessed. Preprocessing includes timestamp alignment, spatial registration, and format standardization of data from different sensors to form a unified multimodal fluorescence data block for subsequent use by deep neural network models.

[0043] Optionally, the bionic visual perception system can integrate a mechanical positioning platform. This platform carries the sample and enables precise two-dimensional or three-dimensional movement. When the system decides to expand or shrink the high-resolution area, the mechanical positioning platform works with the zoom and scanning mechanisms of the optical equipment to quickly and accurately move the acquisition focus to the target location. In some embodiments, the feature recognition algorithm used by the real-time analysis module can be based on traditional image processing techniques, such as edge detection, region growing, and spectral angle mapping algorithms. Alternatively, it can integrate a lightweight machine learning model for preliminary screening. The machine learning model can quickly determine whether the acquired signal belongs to a type that requires special attention. It is understood that the quality of the multimodal fluorescence data dynamically captured by the bionic visual perception system directly affects the accuracy of subsequent pathogen surface distribution heatmap generation. Therefore, ensuring the synchronization accuracy of multiple devices, the response speed of dynamic region division, and the reliability of feature recognition are key considerations in the implementation process.

[0044] Optionally, for different target pathogens, such as wheat dwarf smut and onion pink root rot, due to differences in their fluorescent markers and the characteristics of the tissue lesions they induce, the biomimetic visual perception system's knowledge base stores a feature template library for different pathogens. This feature template library contains typical spectral characteristic curves and morphological descriptors for each pathogen. When dynamically adjusting the high-resolution region, the real-time analysis module performs feature matching by referring to the specific pathogen template targeted by the current detection task, thereby improving the specificity of region segmentation. In specific implementation, the focus adjustment mechanism of the simulated biological visual system is also reflected in the control of depth of field. The fluorescence microscope can have an autofocus function, ensuring that a clear image can be obtained within the high-resolution region, regardless of whether the sample surface is completely flat. This is achieved through laser ranging or a contrast detection algorithm based on image sharpness. The entire process of the biomimetic visual perception system operates automatically under the control of a computer program, forming a closed-loop perception adaptation process from initial region segmentation, synchronous data acquisition, real-time feature analysis to dynamic adjustment of the resolution region. This process ensures that the system can intelligently allocate attention resources in multi-pathogen detection scenarios.

[0045] Example 2: See Figure 3In practice, the first step is to standardize the multimodal fluorescence data. Standardization preprocessing includes dark current correction and flat field correction of fluorescence intensity data from multispectral imaging equipment to eliminate sensor noise and illumination inhomogeneity, scale normalization and rotation correction of spatial morphological feature images from fluorescence microscope to ensure spatial consistency, wavelength calibration and intensity normalization of spectral feature data from spectrometer, and then pixel-level registration of the corrected data according to spatial coordinates, merging them into a multi-channel data block. Each spatial position of the multi-channel data block corresponds to a point on the sample surface, and each channel represents a quantized value of a specific mode.

[0046] In the specific implementation, the constructed deep neural network model adopts a convolutional neural network with an encoder-decoder structure. The encoder consists of five convolutional layers, each followed by a rectified linear unit activation function and a max-pooling layer. Through convolutional operations, multi-scale features of multi-channel data blocks are extracted progressively, while the feature map size decreases layer by layer. The decoder consists of five deconvolutional layers, each responsible for upsampling the spatial size of the feature map to a larger scale. The final output layer has the same size as the input multi-channel data block. A cross-layer connection is added between the encoder and decoder, directly passing the feature map output from each convolutional operation in the encoder to the corresponding deconvolutional layer in the decoder for channel concatenation. This transfers the local features extracted in the encoder stage to the decoder, aiding in the restoration of spatial details. The output layer uses a sigmoid activation function, mapping the final output value to the interval between 0 and 1, generating a pathogen surface distribution heatmap. Each pixel value in the pathogen surface distribution heatmap represents an estimate of the probability of the presence of the target pathogen at the corresponding sample surface location.

[0047] The process of dividing the pathogen surface distribution heatmap into regions based on a preset risk threshold follows immediately after heatmap generation. In practice, it's necessary to calculate the risk value for each pixel region in the pathogen surface distribution heatmap. This risk value isn't calculated directly using the probability value from the heatmap, but rather based on a comprehensive calculation of three features: fluorescence intensity, texture complexity, and gradient change. The fluorescence intensity feature is taken from the normalized fluorescence intensity value at the corresponding position in the multi-channel data block. The texture complexity feature is calculated by the entropy value of the gray-level co-occurrence matrix within the pixel's neighborhood. The gradient change feature is obtained by calculating the response amplitude of the Sobel operator within the pixel's neighborhood. The risk value for each pixel region is calculated using a weighted formula, expressed as:

[0048]

[0049] in: This represents the calculated risk value. This represents the normalized fluorescence intensity value. This represents the texture entropy value calculated based on the gray-level co-occurrence matrix. Represents the magnitude of the image gradient. , , These are pre-set weighting coefficients that satisfy... Set a low threshold for risk values. and high risk threshold The calculated risk value Compared with these two thresholds, if If so, the pixel region is classified as a low-risk region; If so, it is classified as a medium-risk area; if If so, it is classified as a high-risk area.

[0050] In some embodiments, the calculation of risk values ​​can incorporate neighborhood information. Specifically, the average or maximum value of all pixel features within a window surrounding a central pixel is calculated as the representative feature value for that window region, and then substituted into the risk value calculation formula. This helps smooth noise and reflect the overall risk characteristics of the region. It is understood that the setting of low and high risk thresholds needs to be calibrated based on historical detection data or expert knowledge to ensure the accuracy of region division. Setting the threshold too high may lead to the omission of high-risk areas, while setting it too low may lead to low-risk areas being overclassified as medium- or high-risk areas. In specific implementations, using different colors to label the classification results is the final step in region division. Typically, green is used to fill low-risk areas, yellow to fill medium-risk areas, and red to fill high-risk areas, and these results are overlaid on the original sample image on a visualization interface to form an intuitive pathogen risk distribution map.

[0051] Optionally, the results of region segmentation can not only be used for visualization but also generate structured data files. These files record the boundary coordinates of each segmented region, region category labels, and the average risk value within that region, for subsequent use by the optical detection array configuration module. In some embodiments, more complex feature descriptors, such as local binary patterns or histograms of oriented gradients, can be used to calculate texture complexity and gradient changes, in order to capture more finely the microscopic morphological changes associated with pathogen infection. It is understood that the performance of deep neural network models in generating pathogen surface distribution heatmaps depends on training with a large amount of labeled data. This training data should include multimodal fluorescence data of samples under various pathogen infection levels and background conditions, along with their corresponding true distribution labels.

[0052] See Figure 4This image is a core visualization result in the generation of pathogen surface distribution heatmaps and risk region classification in multiplex immunofluorescence analysis methods for detecting various microbial pathogens. Based on multimodal fluorescence data, it generates a pathogen surface distribution heatmap after processing by a deep neural network model. Then, by comprehensively considering fluorescence intensity, texture complexity, and gradient changes to calculate risk values, and combining these with preset low and high thresholds, the regions are classified into low-risk, medium-risk, and high-risk categories. The value of this image lies in providing crucial support for subsequent steps: Firstly, the spatial risk region classification guides the dynamic configuration of the optical detection array, employing the densest detection point spacing in high-risk areas, followed by medium-risk areas, and the sparsest in low-risk areas, achieving precise allocation of detection resources. Secondly, its clear region classification and spatial coordinate information form the spatial basis for the subsequent spatiotemporal alignment of the pathogen surface distribution heatmap with its internal three-dimensional distribution, ensuring that multimodal data are fused and analyzed within a unified spatial framework. This provides structured surface node information for the graph neural network to construct heterogeneous graph structures and generate detection confidence scores, ultimately improving the accuracy and efficiency of pathogen detection.

[0053] Example 3: In this implementation, the region division results are derived from the risk assessment of the pathogen surface distribution heatmap, which clearly identifies high-risk, medium-risk, and low-risk areas on the sample surface. The configuration of the optical detection array is directly driven by the region division results. In this implementation, the optical detection array consists of multiple independently addressable laser emitting units and corresponding photoelectric detection units. The distribution density of the detection points is set according to the risk level of the area they project onto the sample surface. For areas classified as high-risk in the surface heatmap, the spacing between the laser emitting units above their corresponding sample interior space is set to a minimum, for example, 100 micrometers, to achieve high spatial resolution internal detection. For medium-risk areas, the spacing between the laser emitting units is increased to 200 micrometers. For low-risk areas, the spacing between the laser emitting units is set to a maximum of 500 micrometers. This gradient density configuration achieves optimized allocation of detection resources for potentially risky areas.

[0054] In practice, the emission of coded light signals into the sample is achieved through a spatial light modulator. The spatial light modulator receives instructions from a control computer to generate a sequence of coded light patterns. This sequence contains a series of binary or grayscale images with specific spatial structures, such as sinusoidal fringes, Hadamard matrix patterns, or random speckle patterns. The spatial light modulator sequentially loads these patterns and modulates the incident excitation laser beam, projecting the coded excitation patterns into the sample. A photoelectric sensor array operates synchronously with the pattern switching of the spatial light modulator. This array is typically a high-sensitivity scientific-grade CCD or sCMOS camera, responsible for acquiring the fluorescence signals emitted after scattering and absorption by fluorescent markers within the sample, and recording time-series fluorescence image data corresponding to each coded light pattern. Solving for the internal fluorescence source distribution using an iterative algorithm based on a light transport model is the core of calculating the three-dimensional distribution. In practice, the light transport model uses a diffusion approximation equation suitable for light propagation in biological tissues, treating the sample as a homogeneous medium with specific absorption and scattering coefficients. The relationship between the measured fluorescence signal and the internal fluorescence source distribution can be expressed as a linear system:

[0055]

[0056] in: It is a large sparse matrix representing a system matrix constructed based on the diffusion equation and specific boundary conditions, whose elements describe the photon propagation probability from each internal potential light source location to each photoelectric sensor pixel; It is a vector to be solved, representing the fluorescence source intensity of each voxel unit in the discretized three-dimensional space inside the sample; It is a vector composed of the fluorescence intensity values ​​of all pixels collected by the photoelectric sensor array, arranged in sequence.

[0057] In practical implementation, iterative algorithms, such as algebraic reconstruction techniques or the conjugate gradient method, are used to solve the above linear system. Algebraic reconstruction techniques continuously update the estimated value of X through iteration, minimizing the difference between the signal calculated by forward simulation and the measured signal B. The iterative process continues until a preset convergence criterion is met, such as the residual norm being less than a threshold or reaching the maximum number of iterations. The finally obtained vector X is rearranged and interpolated to generate grid data of the three-dimensional distribution inside the pathogen. The grid data is stored in the form of a three-dimensional array, where the value of each array element represents the relative fluorescence source intensity at the corresponding voxel position. Establishing the spatiotemporal alignment between the pathogen surface distribution heatmap and the three-dimensional distribution inside the pathogen is a prerequisite for subsequent data fusion. In practical implementation, spatial alignment is achieved by setting physical reference points on the sample surface to establish a world coordinate system. The physical reference points are usually composed of micro-markers with highly reflective or specific fluorescence properties, which can be clearly identified during both surface imaging and internal detection. The intrinsic and extrinsic parameters of the optical sensor are calibrated using calibration tools. The calibration process uses a calibration plate of known size and dot matrix arrangement. By taking images of the calibration plate in different poses, the intrinsic and extrinsic parameter matrices of the multispectral imaging device, fluorescence microscope, and photoelectric sensor array are calculated.

[0058] In some embodiments, the world coordinate system can be established by selecting a fixed corner point on the sample carrier as the origin, thereby transforming the pixel coordinates in the pathogen surface distribution heatmap to the world coordinate system through the camera extrinsic matrix. It can be understood that the voxel coordinates of the pathogen's internal 3D distribution data are defined based on the world coordinate system during reconstruction. Therefore, after the surface heatmap coordinate transformation is completed, the surface and internal data possess a unified spatial reference frame. Time alignment is achieved by attaching precise time stamps to the pathogen's internal 3D distribution data, accurate to the millisecond level. A network time protocol is used to synchronize the clock signals of all sensors involved in data acquisition. The network time protocol server ensures that the system clock deviations of the multispectral imaging device, fluorescence microscope, spectrometer, spatial light modulator, and photoelectric sensor array remain below the millisecond level.

[0059] Optionally, the accuracy of spatial alignment can be verified through reprojection error, i.e., calculating the difference between the known coordinates of the surface reference point in the world coordinate system after reprojection onto the image plane and the actual detected pixel coordinates. Typically, the average reprojection error is required to be less than one pixel. In some embodiments, for dynamic monitoring scenarios with extremely high time synchronization requirements, hardware trigger signals can be used instead of network time protocols for synchronization. A master device generates a synchronization trigger pulse, and other slave devices strictly synchronize their acquisition actions according to the pulse signal, thereby achieving higher precision time alignment. It can be understood that accurate spatiotemporal alignment ensures that the two-dimensional risk information reflected by the surface distribution heatmap can be correlated with the pathogen information in the depth direction reflected by the internal three-dimensional distribution within the same spatiotemporal framework, providing structured and aligned input data for the fusion computation of graph neural networks.

[0060] Example 4: In a specific implementation, each independently divided risk region (e.g., a continuous high-risk block) in the pathogen surface distribution heatmap is defined as a surface node. Each surface node contains the set of all pixels it covers. Each discrete voxel unit in the pathogen's internal three-dimensional distribution data is defined as an internal node, thus constructing a heterogeneous graph structure containing both surface and internal nodes. Each node needs to be assigned a feature vector as its attribute. The feature vector of a surface node typically includes the average risk value of the risk region, the risk region category label (e.g., 0, 1, and 2 representing low, medium, and high risk, respectively), the two-dimensional coordinates of the geometric center of the risk region in the world coordinate system, the pixel area of ​​the risk region, and the average fluorescence intensity and texture complexity of the region extracted from multimodal fluorescence data. The feature vector of an internal node includes the fluorescence intensity value of the voxel unit, its three-dimensional spatial coordinates, and the local gradient magnitude calculated from its neighboring voxels.

[0061] In its implementation, the graph neural network comprises multiple layers of graph convolution operations. Each layer includes three steps: message passing, message aggregation, and node state update. In the message passing step, each node receives information from its neighbors, who are connected to it via edges. The message aggregation step combines information from all neighbors using a learnable aggregation function. The node state update step combines the aggregated neighbor information with the node's current feature vector and generates a new feature vector for the node using a neural network. After multiple layers of such graph convolution operations, each node's final state vector not only encodes its own attributes but also contains its topological relationships and contextual information within a specific graph structure neighborhood. Finally, the final feature vectors of all nodes are input into a graph-level classification layer. This layer typically aggregates all node features into a single graph representation vector using a global pooling operation. This representation vector is then input into one or more fully connected layers. Finally, a scalar value between 0 and 1, representing the detection confidence score, is output through a sigmoid function. This score represents the overall confidence level in the presence of the target pathogen in the sample after fusing surface and internal information.

[0062] The key to constructing a heterogeneous graph structure lies in defining the rules for edge connections between nodes. In practice, edge establishment is based on a dual criterion of spatial distance and feature similarity. First, the spatial distance between each surface node and each internal node in the world coordinate system is calculated. The position of a surface node is represented by the two-dimensional coordinates of the geometric center of its risk region, with its Z-coordinate considered as 0 (sample surface). The position of an internal node is represented by the three-dimensional coordinates of its voxel center. The three-dimensional Euclidean distance between them is calculated. If the calculated spatial distance is less than a preset connection threshold D_connect, an edge connection is established between the surface node and the internal node. Simultaneously, the cosine similarity of the node feature vectors is calculated. If the cosine similarity is higher than a preset similarity threshold S_sim, an edge connection is also added, even if the spatial distance is slightly greater than D_connect. For each established edge, a weight needs to be assigned. The edge weight is calculated by weighting the spatial distance and feature similarity that generated the edge. The calculation formula can be expressed as:

[0063]

[0064] in: Represents the weight of the edge. This represents the spatial distance between two nodes. It is a normalization factor, usually taking the estimate of the maximum distance between all possible node pairs. The cosine similarity value represents the feature vectors of two nodes. and It is a weighting coefficient that controls the relative importance of the distance factor and the similarity factor, and satisfies Referring to Table 1, this weighting method gives higher weight to connections between nodes that are spatially closer and have more similar features, resulting in greater influence during message transmission.

[0065] Table 1: Feature Vector Table of Heterogeneous Graph Nodes

[0066] Node type Node ID Feature 1 (Average Risk Value) Feature 2 (Center X Coordinate) Feature 3 (Center Y-coordinate) Feature 4 (Area) Feature 5 (average fluorescence intensity) Surface nodes S1 0.85 102.5 58.3 150 1250.6 Surface nodes S2 0.45 205.7 120.1 85 850.3 Internal nodes V1 - 101.8 59.0 2.5 - Internal nodes V2 - 204.2 121.5 1.8 -

[0067] In some embodiments, the construction of heterogeneous graph structures can consider establishing edge connections only between high-risk surface nodes and their spatially neighboring internal nodes, while ignoring connections between low-risk surface nodes and internal nodes. This simplifies the graph structure and focuses on key regions. This requires setting different connection thresholds, such as using a larger D_connect_high for high-risk surface nodes and a smaller D_connect_low for low-risk surface nodes. It is understood that the performance of graph neural networks largely depends on the quality of the heterogeneous graph structure, including the representativeness of node features, the rationality of edge connections, and the accuracy of edge weights. Therefore, feature engineering and threshold selection need to be carefully performed. In specific implementations, training graph neural networks requires a large number of labeled graph structure data samples. Each sample graph corresponds to a sample with a known real pathogen presence state. The training objective is to minimize the loss function between the network output detection confidence score and the true label (0 for negative, 1 for positive), such as binary cross-entropy loss.

[0068] Optionally, in addition to edges between surface nodes and internal nodes, edges can also be established between internal nodes. If two internal nodes are directly adjacent in 3D space (sharing a surface), edges can be established and weighted. These weights can be set based on the gradient of their fluorescence intensity, thereby capturing the continuity information of the internal pathogen distribution. In some embodiments, graph convolution operations can employ more complex models, such as graph attention networks. Graph attention networks assign an adaptive attention weight to each neighbor node during message aggregation, rather than using fixed edge weights or simple aggregation functions. This allows the model to more flexibly learn the importance of different neighbor nodes. It can be understood that by fusing spatiotemporally aligned distribution data through graph neural networks, the complex correlations between pathogen distribution on the surface and in the internal space can be effectively modeled, resulting in a more reliable detection confidence score than analyzing surface or internal data separately. Optionally, the detection confidence score can be further subdivided into confidence scores for different pathogen species. This requires modifying the output of the graph-level classification layer to multiple neurons and employing a corresponding loss function.

[0069] See Figure 5This graph is a key visualization of the pathogen surface and internal distribution data fused spatiotemporally aligned by a graph neural network, generating a detection confidence score. It clearly presents the discriminatory power of the three sample types in terms of confidence level by statistically analyzing the probability density distribution of the detection confidence scores for negative, ambiguous, and positive samples. Simultaneously, setting a judgment threshold and a high-confidence threshold provides a quantitative basis for the subsequent support vector machine classifier to convert the confidence scores into detection instructions. The graph intuitively reflects the effect of the graph neural network on multimodal data fusion: the confidence scores of negative samples are concentrated in the low range, positive samples in the high range, and ambiguous samples are distributed in the intermediate transition zone. This distribution characteristic ensures the accuracy of subsequent instruction conversion. Through clear threshold division, the detection system can efficiently distinguish different infection states, improving the automation and intelligence level of pathogen detection and laying a data support foundation for the final reliable detection conclusion.

[0070] Example 5: In practical implementation, collecting historical detection data is a prerequisite for training the support vector machine (SVM) classifier. Historical detection data includes a large number of detection confidence score samples recorded in previous detections, and the corresponding operation instruction labels for each score sample, which are ultimately deemed correct after expert interpretation or verification using a standard gold-standard method. Operation instruction types include confirmation instructions, exclusion instructions, and retest instructions. During the training phase, the SVM classifier learns a decision boundary to clearly distinguish sample points belonging to different instruction types in the feature space. After training, the SVM classifier model can output a discrete detection instruction type classification result based on a newly input, single detection confidence score value.

[0071] Monitoring sample state changes after the execution of detection commands is a key step in achieving closed-loop system optimization. In practice, once the system executes the corresponding detection command based on the output of the support vector machine classifier, a monitoring cycle is immediately initiated. This involves continuously acquiring fluorescence image sequences of the samples using a multispectral imaging device. The fluorescence image sequences are acquired at fixed time intervals over a period of time to capture the dynamic response after command execution. Calculating the state change deviation requires analyzing the continuously acquired fluorescence image sequences. The change in sample state is quantified by calculating the intensity difference between corresponding pixels in adjacent image frames. The calculated difference image reflects the change in fluorescence intensity distribution between adjacent time points. The absolute value sequence of all calculated pixel differences is compared with a preset baseline threshold. This baseline threshold is a typical range of normal state changes expected after command execution, derived from a large amount of historical data. The state change deviation ratio can be calculated using the following formula:

[0072]

[0073] in: This represents the calculated deviation ratio of state changes. This represents the total number of difference values ​​involved in the calculation (i.e., the number of difference values ​​for all pixels across all adjacent frame pairs). Representing the The specific numerical values ​​of each difference. This represents the preset baseline threshold. It is an indicator function that executes when the condition within the parentheses is true (i.e., ...). The function value is 1 when the threshold is reached, and 0 otherwise. This formula essentially calculates the proportion of changes in the observed state that exceed the expected change range (benchmark threshold).

[0074] The effectiveness index of the detection strategy is generated based on the calculated state change deviation ratio. Effectiveness indicators of detection strategy It can be defined as This indicator reflects the degree of consistency between actual and expected changes in the state. The closer the value is to 1, the more closely the actual response matches expectations, and the higher the effectiveness of the current detection strategy. Dynamically adjusting detection parameters is crucial for the effectiveness of the detection strategy. Below a preset adjustment threshold Triggered at time, when When the system determines that the current combination of detection parameters is not ideal and needs adjustment, the main optical detection parameters to be modified include excitation light intensity and exposure time. The adjustment logic is usually based on optimization algorithms or predefined rules. For example, if the deviation ratio... If the excitation light intensity is too high and manifests as insufficient signal response, the system may increase the excitation light intensity or extend the exposure time in certain steps to enhance the signal acquisition capability; conversely, if the signal is oversaturated or the noise is too high, the excitation light intensity may be appropriately reduced or the exposure time shortened.

[0075] In some embodiments, the support vector machine classifier can employ a nonlinear kernel function, such as the radial basis function, to handle the complex nonlinear relationship that may exist between the detection confidence score and the optimal detection instruction. This requires more historical data for training to avoid overfitting. It is understood that the quality and quantity of historical detection data directly determine the accuracy of the support vector machine classifier in instruction conversion; therefore, a continuously updated annotation database needs to be established. When monitoring changes in sample state, the continuously acquired fluorescence image sequences may need to be registered first to eliminate displacement artifacts caused by minute sample movements, ensuring that the calculated difference value truly reflects the change in fluorescence intensity rather than positional shift. Optionally, the calculation of state change bias can focus on the region of interest, such as areas previously classified as high-risk or medium-risk, rather than calculating for all pixels in the entire image, thus improving the specificity and efficiency of the calculation. In some embodiments, dynamically adjusting detection parameters can be not limited to excitation light intensity and exposure time, but can also include the bandwidth of the optical filter, the contrast of the encoded pattern of the spatial light modulator, etc., forming a multi-parameter joint optimization problem.

[0076] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multiplex immunofluorescence assay for detecting various microbial pathogens, characterized in that, The method includes: A biomimetic visual perception system is used to dynamically capture multimodal fluorescence data of pathogens. The biomimetic visual perception system simulates the focus adjustment mechanism of the biological visual system, dividing the sample surface into high-resolution and low-resolution regions. The high-resolution region corresponds to the potential aggregation site of pathogens and adopts a dense sampling strategy, while the low-resolution region adopts a sparse sampling strategy. The system analyzes the feature changes in the collected data in real time and dynamically adjusts the range of the high-resolution region. When new feature changes are found, the high-resolution region is expanded, and when the feature disappears, the high-resolution region is shrunk. Multimodal fluorescence data is input into a deep neural network model to generate a pathogen surface distribution heatmap, and the pathogen surface distribution heatmap is divided into regions based on a preset risk threshold; Based on the regional division results, an optical detection array is configured and coded light signals are emitted into the sample. After receiving the internal fluorescence response, the three-dimensional distribution of the pathogen inside the sample is calculated through a reconstruction algorithm. Establish a spatiotemporal alignment between the pathogen surface distribution heatmap and the pathogen's internal three-dimensional distribution, and use a clock synchronization mechanism to align time information; A graph neural network is used to fuse spatiotemporally aligned distribution data to generate a detection confidence score; a machine learning model is then applied to convert the detection confidence score into a detection instruction. Monitor the changes in sample status after the execution of the detection command, calculate the deviation of the status change, generate the effectiveness index of the detection strategy, and dynamically adjust the detection parameters.

2. The multiplex immunofluorescence assay method for detecting multiple microbial pathogens according to claim 1, characterized in that, The multimodal fluorescence data of the samples dynamically captured using a biomimetic visual perception system includes: The pathogens include wheat dwarf smut, wheat leaf blight, wheat chrysophagus, onion pink root rot, sorghum root rot, grape stem blight, and cruciferous vegetable blackleg. Multispectral imaging equipment, fluorescence microscope, and spectrometer are deployed to simultaneously acquire the fluorescence intensity distribution, spatial morphological characteristics, and spectral characteristics of the samples.

3. The multiplex immunofluorescence assay method for detecting multiple microbial pathogens according to claim 1, characterized in that, The step of inputting multimodal fluorescence data into a deep neural network model to generate a pathogen surface distribution heatmap includes: Multimodal fluorescence data are standardized and preprocessed to merge into multi-channel data blocks; a convolutional neural network with an encoder-decoder structure is constructed, where the encoder extracts multi-scale features through convolution operations and the decoder recovers spatial dimensions through deconvolution operations; cross-layer connections are added between the encoder and decoder to transfer local features; and the output layer uses an activation function to generate a pathogen surface distribution heatmap.

4. The multiplex immunofluorescence assay method for detecting multiple microbial pathogens according to claim 1, characterized in that, The process of dividing the pathogen surface distribution heatmap into regions based on a preset risk threshold includes: The risk value of each pixel region in the pathogen surface distribution heatmap is calculated. The risk value is calculated based on a combination of fluorescence intensity, texture complexity, and gradient change. Low and high risk thresholds are set. Regions with risk values ​​below the low risk threshold are classified as low-risk regions, regions with risk values ​​between the low and high risk thresholds are classified as medium-risk regions, and regions with risk values ​​above the high risk threshold are classified as high-risk regions. Different colors are used to mark the classification results.

5. The multiplex immunofluorescence assay method for detecting multiple microbial pathogens according to claim 4, characterized in that, The process of configuring an optical detection array based on the regional division results and emitting coded light signals into the sample, receiving the internal fluorescence response, and then calculating the three-dimensional distribution of the pathogen inside the sample using a reconstruction algorithm includes: setting the distribution density of optical detection points according to the risk level of the region, with the smallest spacing between detection points in high-risk regions, followed by medium-risk regions, and the largest spacing in low-risk regions; using a spatial light modulator to generate a sequence of coded light patterns and projecting it into the sample; collecting fluorescence signals and recording time-series data through a photoelectric sensor array; and using an iterative algorithm based on a light transmission model to solve for the distribution of internal fluorescence sources and generate three-dimensional grid data.

6. The multiplex immunofluorescence assay method for detecting multiple microbial pathogens according to claim 5, characterized in that, The spatiotemporal alignment of establishing the pathogen surface distribution heatmap with the pathogen's internal three-dimensional distribution includes: A world coordinate system is established by setting reference points on the sample surface; the internal and external parameters of the optical sensor are calibrated using calibration tools, and the surface distribution heat map of the pathogen is converted to the world coordinate system; time stamps are added to the three-dimensional distribution data inside the pathogen, and the clock signals of all sensors are synchronized using a network time protocol.

7. The multiplex immunofluorescence assay method for detecting multiple microbial pathogens according to claim 1, characterized in that, The process of using a graph neural network to fuse spatiotemporally aligned distribution data to generate a detection confidence score includes: Each risk region of the pathogen surface distribution heatmap is used as a surface node, and each voxel unit of the pathogen's internal three-dimensional distribution is used as an internal node to construct a heterogeneous graph structure. Feature vectors of surface nodes and internal nodes are extracted as node attributes. The graph neural network contains multiple layers of graph convolution operations, and each layer aggregates information from adjacent nodes to update the node state. Finally, the detection confidence score is output through the classification layer.

8. The multiplex immunofluorescence assay method for detecting multiple microbial pathogens according to claim 7, characterized in that, The construction of the heterogeneous graph structure includes: Calculate the spatial distance between surface nodes and internal nodes. If the distance is less than the connection threshold, establish an edge connection. At the same time, calculate the similarity of node feature vectors. If the similarity is higher than the similarity threshold, add an edge. The weight of the edge is weighted according to the distance and similarity.

9. The multiplex immunofluorescence assay method for detecting multiple microbial pathogens according to claim 8, characterized in that, The application of the machine learning model to convert the detection confidence score into detection instructions includes: Collect historical detection confidence scores and corresponding operation instructions to train a support vector machine classifier; input the current detection confidence score into the support vector machine classifier and output the detection instruction type; the detection instruction type includes confirmation instruction, exclusion instruction and retest instruction.

10. The multiplex immunofluorescence assay method for detecting multiple microbial pathogens according to claim 9, characterized in that, The process of monitoring and detecting changes in sample state after the execution of the detection command, calculating state change deviations, generating detection strategy effectiveness indicators, and dynamically adjusting detection parameters includes: Continuously acquire sample fluorescence image sequences and calculate the difference value between adjacent image frames; compare the difference value with a benchmark threshold to obtain the deviation ratio; calculate the detection strategy effectiveness index based on the deviation ratio; when the detection strategy effectiveness index is lower than the adjustment threshold, modify the optical detection parameters, including excitation light intensity and exposure time.

Images