A SPR response region identification method based on image semantic segmentation and temporal alignment

Abstract

This invention discloses a method for SPR response region identification based on image semantic segmentation and temporal alignment, comprising the following steps: acquiring SPR image frame sequence data to construct an original image sequence; performing image preprocessing operations on the original image sequence to output a standardized image sequence; constructing a temporal window image set composed of multiple consecutive frames; inputting the temporal window image set into an improved SegFormer model to generate a response region segmentation mask map corresponding to each frame; performing cross-frame temporal alignment operations on the response regions to output a temporal consistency identifier mapping of the response regions; performing area statistics, intensity analysis, and temporal localization operations; and generating a structured response region identification result. This invention can effectively identify the spatiotemporal evolution process of response regions in SPR image sequences, improving the accuracy and stability of response region identification, and is suitable for high-precision biological detection and real-time molecular analysis scenarios.

Description

Technical Field

[0001] This invention relates to the fields of image processing and biosensing technology, and in particular to a method for identifying SPR response regions based on image semantic segmentation and temporal alignment. Background Technology

[0002] With the widespread application of surface plasmon resonance (SPR) technology in biomolecule detection, drug screening, and real-time response monitoring, accurately identifying response regions and quantifying their temporal changes from SPR image sequences has become a key issue for improving detection sensitivity and automated analysis capabilities. Existing SPR response identification methods mainly rely on traditional image processing techniques such as image gray-level difference thresholding, regional mean changes, or global background subtraction to roughly determine the response region. However, these methods generally suffer from the following problems in practical applications:

[0003] Response regions often exhibit characteristics such as blurred edges, weak grayscale changes, and irregular shapes. Traditional methods based on fixed thresholds or simple image differencing are difficult to adapt to the subtle response changes under various experimental conditions, resulting in high false negative rates and inaccurate spatial localization in response region detection results. The response process in SPR image sequences has obvious temporal evolution characteristics. Existing methods mostly process each frame independently, lacking the ability to model the cross-frame continuity and dynamic trajectory of the response region, and cannot achieve consistent tracking of the same response region in the time dimension. In addition, existing methods usually do not perform sufficient registration and preprocessing of displacement and rotation errors in image acquisition, which easily leads to error accumulation in response region recognition due to image perturbation, reducing the stability and repeatability of analysis results.

[0004] Therefore, how to provide a method for SPR response region recognition based on image semantic segmentation and temporal alignment is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an SPR response region recognition method based on image semantic segmentation and temporal alignment. This invention fully integrates deep visual segmentation networks and dynamic time warping algorithms to construct an improved SegFormer model for segmenting response regions in SPR images, and introduces a cross-frame trajectory matching mechanism and response feature analysis process, which has the advantages of high recognition accuracy, strong temporal consistency and high degree of automation.

[0006] The SPR response region identification method based on image semantic segmentation and temporal alignment according to embodiments of the present invention includes the following steps:

[0007] Acquire SPR image frame sequence data and construct the original image sequence;

[0008] Perform image preprocessing operations on the original image sequence to output a normalized image sequence;

[0009] Based on the standardized image sequence and its time index information, a time-series window image set consisting of multiple consecutive frames is constructed.

[0010] Input the temporal window image set into the improved SegFormer model to generate the response region segmentation mask map corresponding to each frame;

[0011] Based on the response region segmentation mask map of each frame, cross-frame temporal alignment of the response region is performed. The dynamic time warping method is used to perform trajectory matching of the response region positions at multiple time points and outputs the response region temporal consistency identifier mapping.

[0012] Based on the mapping between the response region segmentation mask and the temporal consistency identifier, area statistics, response intensity numerical analysis and time index positioning operations are performed on each response region to generate the corresponding response region area change curve, intensity change trend curve and maximum response time point, and to construct a structured response region temporal feature representation.

[0013] The response region segmentation mask, time index information, and structured response region temporal feature representation are associated to generate structured response region recognition results.

[0014] Optionally, the acquisition of SPR image frame sequence data and the construction of the original image sequence specifically includes:

[0015] The original image frames of the sensor chip surface are continuously acquired within a set time interval. The original image frames include image information reflecting changes in surface plasmon resonance.

[0016] Record the corresponding time index at each time an image frame is acquired, so that the image frame is matched with its acquisition time.

[0017] All image frames are arranged in chronological order according to their time indices to form a temporally continuous original image sequence.

[0018] Optionally, the step of performing image preprocessing on the original image sequence to output a normalized image sequence specifically includes:

[0019] A Gaussian filtering method is used to perform filtering operations on each frame of the original image sequence;

[0020] The image feature point matching method is used to perform spatial registration operation on image frames in the original image sequence to correct displacement or rotation deviations during the acquisition process.

[0021] Perform brightness normalization on the registered image frames to unify the grayscale distribution range of the images;

[0022] The image frames, after filtering, registration, and normalization, are reorganized in chronological order to construct a standardized image sequence.

[0023] Optionally, the step of constructing a temporal window image set consisting of multiple consecutive frames based on the standardized image sequence and its time index information specifically includes:

[0024] Read the image frames and their corresponding time index information from the standardized image sequence;

[0025] At each target time point, determine the time index corresponding to several adjacent time points, and select the image frame corresponding to the current time point and the adjacent time points from the standardized image sequence.

[0026] The selected image frames are arranged in chronological order, and size and channel consistency processing is performed to construct a time-series window image set consisting of multiple consecutive frames.

[0027] Optionally, the step of inputting the temporal window image set into the improved SegFormer model to generate a response region segmentation mask for each frame specifically includes:

[0028] An improved SegFormer model is constructed, including an encoder with enhanced frame position coding, a temporal cross-attention mechanism, a saliency differential guidance structure, a lightweight decoder, and a response consistency supervision module;

[0029] The temporal window image set is input to the encoder, which is constructed using a hybrid vision Transformer. It performs multi-scale feature embedding operations on each frame image and generates a frame position encoding vector by embedding the time index through the embedding mapping. This vector is then fused with the original image embedding to output a multi-scale encoded feature representation containing frame position information.

[0030] The multi-scale encoded feature representation is input into the temporal cross-attention mechanism. Cross-frame channel attention coupling is performed on the feature vectors of the current time frame and the adjacent time frame that are in the same spatial position to construct temporal fusion features. The output is a set of cross-frame enhanced encoded features.

[0031] The cross-frame enhanced coding feature set is fused with the saliency guide map. The saliency guide map is generated by calculating the grayscale difference map between the current frame image and the previous frame image, extracting response change features through convolution, and normalizing it with the Sigmoid function. After channel expansion, the guide map is fused with the coding feature set through a channel-wise weighted fusion to output a response-enhanced feature set.

[0032] The response-enhanced feature set is input into the lightweight decoder, which performs layer-by-layer upsampling and feature restoration operations to restore the spatial size to the same as the input image frame, and outputs the response region segmentation mask map of each frame image.

[0033] Receive response region segmentation mask images of adjacent time frames, and construct a dynamic response consistency regularization loss function based on Euclidean distance;

[0034] The temporal consistency loss term for the prediction results of consecutive frames is used in conjunction with the main loss function for model training optimization.

[0035] Optionally, the step of using a dynamic time warping method to perform trajectory matching on the response region locations at multiple time points and outputting a temporal consistency identifier mapping for the response region specifically includes:

[0036] Perform boundary extraction on the response region segmentation mask for each frame, determine the spatial centroid coordinates or boundary contour information of the response region in each frame image based on connected component analysis, and construct the response region location sequence.

[0037] The dynamic time warping method is applied to the response region location sequence to calculate the minimum distance cost path between the current frame and the adjacent frames for each response region location, and a cross-frame response region mapping relationship is established based on the minimum cumulative distance.

[0038] Based on the dynamic time warping calculation results, the response region trajectories with spatial continuity and minimum displacement cost in the time series are numbered and matched, and the inter-frame response region trajectory number labels are output.

[0039] Construct a temporal consistency identifier mapping for the response region based on the trajectory number label of the response region.

[0040] Optionally, the construction of the structured response region temporal feature representation specifically includes:

[0041] The response region segmentation mask map is mapped to the response region temporal consistency identifier, and the response region mask maps belonging to the same identifier number are combined into a response trajectory sequence according to the time index;

[0042] Perform pixel statistics on the mask region corresponding to each response trajectory sequence, calculate the number of effective pixels in the response region in each frame, convert it into an area value according to the image spatial resolution, and construct the area change curve of the response region over time.

[0043] By combining the original image frame sequence, the gray values ​​of the area covered by the response region mask in each frame are averaged to extract numerical indicators representing the response intensity, and then combined in chronological order to form an intensity change trend curve.

[0044] Search for the global maximum point in the area change curve or intensity change curve, locate its corresponding time index, and determine the maximum response time point of the current response area;

[0045] The area change curve, intensity change trend curve, and maximum response time point corresponding to each response trajectory are combined into a set of structured response region temporal feature representations.

[0046] Optionally, the step of associating the response region segmentation mask, time index information, and structured response region temporal feature representation to generate a structured response region identification result specifically includes:

[0047] Receive response region segmentation mask, structured response region temporal feature representation, and time index information corresponding to each frame image;

[0048] Based on the temporal consistency identifier mapping, the segmentation mask of each response region is associated with the corresponding area change curve, intensity change trend curve and the maximum response time point;

[0049] In each response trajectory, the response duration interval of the response region is determined based on the range of consecutively appearing frame indexes in the response region segmentation mask image, and the complete spatiotemporal trajectory is marked by combining the spatial location of the response region in each frame image.

[0050] The spatial location set, response duration range, and structured response region temporal feature representation of each response region are organized into response region identification result data items according to a predetermined field format, and the structured response region identification result is output.

[0051] The beneficial effects of this invention are:

[0052] (1) This invention constructs an improved SegFormer model that includes frame position coding enhancement, temporal cross attention mechanism and saliency difference guidance structure. It makes full use of the multi-frame information in the temporal window image set to achieve high-precision semantic-level segmentation of response regions with weak gray-level changes and blurred edges in SPR image sequences. Compared with traditional image difference or thresholding methods, it can significantly improve the detection capability of weak response targets.

[0053] (2) By introducing a dynamic time warping method, the present invention performs optimal trajectory matching on the spatial position of the response region in each frame image and constructs a temporal consistency identifier mapping of the response region. This solves the problem that the traditional method of processing each frame independently leads to discontinuous response trajectories and the inability of the same region to correspond across frames. It realizes accurate association and numbering tracking of the response region in the time dimension and enhances the system's ability to model the complete response process.

[0054] (3) The present invention further combines the segmentation mask map and trajectory marker to carry out the operation of response area statistics, intensity change trend analysis and maximum response time point location, generate a structured response region temporal feature representation, and organize it in a unified manner with time index and spatial location information, output the structured response region identification result, and improve the SPR system's ability to automatically interpret the detection process and quantitatively express the dynamic changes of the response. Attached Figure Description

[0055] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0056] Figure 1 This is a flowchart of the SPR response region recognition method based on image semantic segmentation and temporal alignment proposed in this invention;

[0057] Figure 2 This is a schematic diagram of the SPR response region recognition method based on image semantic segmentation and temporal alignment proposed in this invention;

[0058] Figure 3 This is a framework diagram of the improved SegFormer model in the SPR response region recognition method based on image semantic segmentation and temporal alignment proposed in this invention. Detailed Implementation

[0059] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0060] refer to Figure 1-3 The SPR response region recognition method based on image semantic segmentation and temporal alignment includes the following steps:

[0061] Step 1: Acquire SPR image frame sequence data and construct the original image sequence;

[0062] Step 2: Perform image preprocessing operations on the original image sequence to output a standardized image sequence;

[0063] Step 3: Based on the standardized image sequence and its time index information, construct a time-series window image set consisting of multiple consecutive frames;

[0064] Step 4: Input the temporal window image set into the improved SegFormer model to generate the response region segmentation mask map corresponding to each frame;

[0065] Step 5: Perform cross-frame temporal alignment of the response region based on the response region segmentation mask map of each frame. Use dynamic time warping to perform trajectory matching on the response region positions at multiple time points and output the temporal consistency identifier mapping of the response region.

[0066] Step 6: Based on the mapping between the response region segmentation mask and the temporal consistency identifier, perform area statistics, response intensity numerical analysis and time index positioning operations on each response region to generate the corresponding response region area change curve, intensity change trend curve and maximum response time point, and construct a structured temporal feature representation of the response region;

[0067] Step 7: Associate the response region segmentation mask, time index information, and structured response region temporal feature representation to generate structured response region recognition results.

[0068] In this embodiment, the acquisition of SPR image frame sequence data and the construction of the original image sequence specifically includes:

[0069] The original image frames of the sensor chip surface are continuously acquired within a set time interval. The original image frames include image information reflecting changes in surface plasmon resonance.

[0070] Record the corresponding time index at each time an image frame is acquired, so that the image frame is matched with its acquisition time.

[0071] All image frames are arranged in chronological order according to their time indices to form a temporally continuous original image sequence.

[0072] This implementation method effectively constructs a temporally continuous original image sequence by continuously acquiring raw image frames from the surface of the sensor chip within a set time interval and arranging them sequentially in conjunction with the time index information corresponding to each frame. This approach not only ensures the temporal integrity of the SPR image data but also provides a high-quality input foundation for subsequent temporal modeling and response trajectory analysis, significantly improving the accuracy and consistency of response region identification.

[0073] In this embodiment, the step of performing image preprocessing on the original image sequence to output a standardized image sequence specifically includes:

[0074] A Gaussian filtering method is used to perform filtering operations on each frame of the original image sequence;

[0075] The image feature point matching method is used to perform spatial registration operation on image frames in the original image sequence to correct displacement or rotation deviations during the acquisition process.

[0076] Perform brightness normalization on the registered image frames to unify the grayscale distribution range of the images;

[0077] The image frames, after filtering, registration, and normalization, are reorganized in chronological order to construct a standardized image sequence.

[0078] This implementation method effectively suppresses image noise, corrects inter-frame displacement and rotation errors, and unifies the image grayscale by sequentially performing Gaussian filtering, image feature point spatial registration, and brightness normalization on the original image sequence. This generates a standardized image sequence with stable quality and continuous temporal sequence, providing accurate and consistent input data for subsequent semantic segmentation and response feature extraction, and significantly improving the robustness and accuracy of the overall recognition effect.

[0079] In this embodiment, the step of constructing a time-series window image set consisting of multiple consecutive frames based on the standardized image sequence and its time index information specifically includes:

[0080] Read the image frames and their corresponding time index information from the standardized image sequence;

[0081] At each target time point, determine the time index corresponding to several adjacent time points, and select the image frame corresponding to the current time point and the adjacent time points from the standardized image sequence.

[0082] The selected image frames are arranged in chronological order, and size and channel consistency processing is performed to construct a time-series window image set consisting of multiple consecutive frames.

[0083] This implementation selects image frames corresponding to adjacent time points at each target time point, constructs a temporal window image set of multiple consecutive frames by combining time index information, and performs size unification and channel consistency processing on the selected image frames to ensure the structural consistency and contextual coherence of the input image in the temporal modeling process. This effectively enhances the subsequent model's ability to perceive temporal dynamic features and improves the temporal stability and accuracy of response region recognition.

[0084] In this embodiment, the step of inputting the temporal window image set into the improved SegFormer model to generate a response region segmentation mask for each frame specifically includes:

[0085] An improved SegFormer model is constructed, including an encoder with enhanced frame position coding, a temporal cross-attention mechanism, a saliency differential guidance structure, a lightweight decoder, and a response consistency supervision module;

[0086] The temporal window image set is input to the encoder, which is constructed using a hybrid vision Transformer. It performs multi-scale feature embedding operations on each frame image and generates a frame position encoding vector by embedding the time index through the embedding mapping. This vector is then fused with the original image embedding to output a multi-scale encoded feature representation containing frame position information.

[0087] The multi-scale encoded feature representation is input into a temporal cross-attention mechanism, which performs cross-frame channel attention coupling on feature vectors of the current time frame and adjacent time frames that are in the same spatial position to construct temporal fusion features. The specific calculation method is as follows:

[0088] ;

[0089] in, , , These are the query vector for the current frame obtained through linear transformation, and the key vector and value vector for the previous frame, respectively. The scaling factor is constructed based on the principle of cross-frame attention calculation in Transformer, and the output is a set of encoded features enhanced across frames;

[0090] The cross-frame enhanced coding feature set is fused with the saliency guide map. The saliency guide map is generated by calculating the grayscale difference map between the current frame image and the previous frame image, extracting response change features through convolution, and normalizing it with the Sigmoid function. After channel expansion, the guide map is fused with the coding feature set through a channel-wise weighted fusion to output a response-enhanced feature set.

[0091] The response-enhanced feature set is input into the lightweight decoder, which performs layer-by-layer upsampling and feature restoration operations to restore the spatial size to the same as the input image frame, and outputs the response region segmentation mask map of each frame image.

[0092] Receive response region segmentation masks from adjacent time frames, and construct a dynamic response consistency regularization loss function based on Euclidean distance. :

[0093] ;

[0094] in and The model represents the first time. The response region segmentation mask map output on the frame image and the previous frame image;

[0095] The temporal consistency loss term for the prediction results of consecutive frames is used in conjunction with the main loss function for model training optimization.

[0096] This implementation constructs an improved SegFormer model, employing a frame position encoding-enhanced encoder, a temporal cross-attention mechanism, and a saliency difference-guided structure to model deep features of consecutive image frames. Temporal index information is introduced to enhance the model's temporal awareness, and a cross-frame attention mechanism is used to achieve dynamic fusion of response features, effectively improving the ability to identify weak response regions. Simultaneously, a dynamic response consistency regularization loss function is constructed to apply Euclidean distance constraints to the segmentation results of adjacent frames, strengthening the model's consistent representation of response regions evolving over time, ultimately generating a more accurate and temporally continuous response region segmentation mask. This approach significantly enhances the robustness and dynamic tracking stability of SPR image response region segmentation.

[0097] In this embodiment, the step of using a dynamic time warping method to perform trajectory matching on the response region locations at multiple time points and outputting a temporal consistency identifier mapping for the response region specifically includes:

[0098] Perform boundary extraction on the response region segmentation mask for each frame, determine the spatial centroid coordinates or boundary contour information of the response region in each frame image based on connected component analysis, and construct the response region location sequence.

[0099] The dynamic time warping method is applied to the response region location sequence to calculate the minimum distance cost path between the current frame and the adjacent frames for each response region location, and a cross-frame response region mapping relationship is established based on the minimum cumulative distance.

[0100] Based on the dynamic time warping calculation results, the response region trajectories with spatial continuity and minimum displacement cost in the time series are numbered and matched, and the inter-frame response region trajectory number labels are output.

[0101] Construct a temporal consistency identifier mapping for the response region based on the trajectory number label of the response region.

[0102] This implementation method accurately obtains the position coordinates of the response regions by performing boundary extraction and connected component analysis on the response region segmentation mask map of each frame. Furthermore, it utilizes a dynamic time warping method to match and number the response region trajectories across different time frames, achieving stable tracking and consistent numbering of response regions across frames. This method effectively improves the temporal consistency and spatial continuity of response region identification results in SPR images, contributing to the accuracy and interpretability of subsequent response change trend analysis.

[0103] In this embodiment, the construction of the structured response region temporal feature representation specifically includes:

[0104] The response region segmentation mask map is mapped to the response region temporal consistency identifier, and the response region mask maps belonging to the same identifier number are combined into a response trajectory sequence according to the time index;

[0105] Perform pixel statistics on the mask region corresponding to each response trajectory sequence, calculate the number of effective pixels in the response region in each frame, convert it into an area value according to the image spatial resolution, and construct the area change curve of the response region over time.

[0106] By combining the original image frame sequence, the gray values ​​of the area covered by the response region mask in each frame are averaged to extract numerical indicators representing the response intensity, and then combined in chronological order to form an intensity change trend curve.

[0107] Search for the global maximum point in the area change curve or intensity change curve, locate its corresponding time index, and determine the maximum response time point of the current response area;

[0108] The area change curve, intensity change trend curve, and maximum response time point corresponding to each response trajectory are combined into a set of structured response region temporal feature representations.

[0109] This implementation combines the response region segmentation mask map with the temporal consistency identifier mapping result to construct a complete temporal trajectory sequence of the response region. Pixel statistics and grayscale analysis are performed on this region in each frame of the image, generating area change curves and intensity change trend curves, respectively. Simultaneously, the maximum response time point is accurately extracted, and a structured temporal feature representation of the response region is constructed. This method enables quantitative analysis and feature modeling of the SPR response region in the temporal dimension, improving the visualization, monitoring, and intelligent interpretation capabilities of the dynamic response process.

[0110] In this embodiment, the step of associating the response region segmentation mask, time index information, and structured response region temporal feature representation to generate a structured response region recognition result specifically includes:

[0111] Receive response region segmentation mask, structured response region temporal feature representation, and time index information corresponding to each frame image;

[0112] Based on the temporal consistency identifier mapping, the segmentation mask of each response region is associated with the corresponding area change curve, intensity change trend curve and the maximum response time point;

[0113] In each response trajectory, the response duration interval of the response region is determined based on the range of consecutively appearing frame indexes in the response region segmentation mask image, and the complete spatiotemporal trajectory is marked by combining the spatial location of the response region in each frame image.

[0114] The spatial location set, response duration range, and structured response region temporal feature representation of each response region are organized into response region identification result data items according to a predetermined field format, and the structured response region identification result is output.

[0115] This implementation associates a response region segmentation mask, a structured temporal feature representation of the response region, and time index information, and maps these together with temporal consistency identifiers to achieve comprehensive annotation of the spatial location, duration, and dynamic features of the response trajectory. This method can accurately extract and output structured response region identification results, possessing excellent interpretability and visualization features, and significantly improving the automatic identification accuracy and analysis efficiency of SPR response regions.

[0116] Example 1:

[0117] To verify the feasibility of this invention in practice, it was applied to an SPR (surface plasmon resonance) molecular recognition experiment conducted at a research institution. The aim was to achieve automatic identification and cross-frame tracking of response regions, further improving the accuracy and efficiency of SPR response analysis. The experimental scenario was based on a real-world SPR image acquisition system. During a continuous 10-second observation period, one frame of SPR grayscale image was acquired per second, with a resolution of 512×512 pixels, forming a complete image sequence. The experimental setup included two target protein molecules acting on the chip surface to produce local responses, designated R1 and R2, corresponding to the binding regions of different molecules. After data acquisition, the images were input into the automatic SPR response region identification system constructed using the method of this invention for processing.

[0118] In the data processing flow, the original image sequence is first subjected to Gaussian filtering and brightness normalization to suppress background random noise and enhance local contrast. Next, an improved SegFormer model is used to perform frame-by-frame semantic segmentation of response regions. This model effectively improves the ability to identify weak response regions by introducing a cross-scale attention mechanism and a structured nesting guidance strategy. To address the temporal variation behavior of response regions, an intra-frame saliency prediction enhancement branch is constructed and fused with contextual feature maps from adjacent frames. A time-aware attention mechanism maintains the consistency of region response boundaries. The segmentation result for each frame is output as a binary mask map, identifying the salient response regions in the current frame image.

[0119] Subsequently, a Dynamic Time Warping (DTW) method is introduced to perform cross-frame response trajectory alignment. The system performs trajectory pairing and adjustment for each response region based on the region centroid coordinate sequence and grayscale response intensity change trend, generating a complete temporal consistency identifier mapping for the response regions. Based on the temporal consistency mask and the original response region mask, the area (number of pixels), average response intensity (mean grayscale), and maximum response time point of each region at different time points are further statistically analyzed and encoded into a structured temporal feature representation.

[0120] Table 1 shows the SPR response region detection results, displaying the area, intensity, and semantic segmentation accuracy (IoU) of the two response regions R1 and R2 across 10 time indices:

[0121] Table 1. SPR Response Area Detection Results

[0122] Time Index Response region R1 - area Response region R1 - intensity Response region R1-IoU Response region R2 - area Response region R2-intensity Response region R2-IoU 0 126.65 0.874 0.874 161.42 0.782 0.919 1 140.71 0.778 0.876 172.71 0.812 0.797 2 137.10 0.728 0.790 169.08 0.712 0.887 3 128.91 0.539 0.773 163.03 0.830 0.769 4 117.30 0.746 0.902 143.69 0.612 0.761 5 110.18 0.864 0.795 157.33 0.963 0.812 6 105.97 0.688 0.926 180.00 0.991 0.951 7 118.65 0.718 0.864 162.50 0.901 0.824 8 121.38 0.759 0.904 149.36 0.710 0.850 9 130.00 0.804 0.892 145.71 0.660 0.843

[0123] Analysis of the data in Table 1 reveals that the area of ​​the response region R1 gradually increases from 126.65 to 140.71 within 0 to 2 seconds, while the response intensity decreases slightly, indicating that the initial binding region gradually expands but the unit response decreases. Between 4 and 6 seconds, the area decreases but the intensity increases, indicating local binding focusing and the generation of a stronger reflected signal. The IoU accuracy of semantic segmentation remains stable between 0.77 and 0.92, demonstrating the model's stable recognition capability.

[0124] The area of ​​the response region R2 reaches its maximum value of 180.00 between 5 and 6 seconds, and its intensity also peaks at 0.991 during this stage, indicating that it is the main reaction region. The segmented IoU value also reaches its highest value of 0.951 during this period, demonstrating that the method has good accuracy under large response conditions. In addition, the area of ​​the R2 region gradually shrinks and its intensity decreases in the later stage, reflecting the molecular dissociation process, and the response characteristics decay significantly over time.

[0125] By combining the time-series consistency identifiers generated by this method, the maximum response time point in each region can be accurately located, and a complete response change trend chart can be generated, effectively supporting dynamic response analysis in SPR biological experiments.

[0126] This embodiment significantly improves the accuracy of response region identification and the stability of cross-frame temporal analysis by applying the method of this invention in a real SPR experimental scenario. It solves the problems of response boundary ambiguity, region drift and time trajectory breakage in traditional methods, demonstrating good engineering practical value and intelligent automation potential.

[0127] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for SPR response region recognition based on image semantic segmentation and temporal alignment, characterized in that, Includes the following steps: Acquire SPR image frame sequence data and construct the original image sequence; Perform image preprocessing operations on the original image sequence to output a normalized image sequence; Based on the standardized image sequence and its time index information, a time-series window image set consisting of multiple consecutive frames is constructed. The temporal window image set is input into the improved SegFormer model to generate a response region segmentation mask for each frame, specifically including: An improved SegFormer model is constructed, including an encoder with enhanced frame position coding, a temporal cross-attention mechanism, a saliency differential guidance structure, a lightweight decoder, and a response consistency supervision module; The temporal window image set is input to the encoder, which is constructed using a hybrid vision Transformer. It performs multi-scale feature embedding operations on each frame image and generates a frame position encoding vector by embedding the time index through the embedding mapping. This vector is then fused with the original image embedding to output a multi-scale encoded feature representation containing frame position information. The multi-scale encoded feature representation is input into the temporal cross-attention mechanism. Cross-frame channel attention coupling is performed on the feature vectors of the current time frame and the adjacent time frame that are in the same spatial position to construct temporal fusion features. The output is a set of cross-frame enhanced encoded features. The cross-frame enhanced coding feature set is fused with the saliency guide map. The saliency guide map is generated by calculating the grayscale difference map between the current frame image and the previous frame image, extracting response change features through convolution, and normalizing it with the Sigmoid function. After channel expansion, the guide map is fused with the coding feature set through a channel-wise weighted fusion to output a response-enhanced feature set. The response-enhanced feature set is input into the lightweight decoder, which performs layer-by-layer upsampling and feature restoration operations to restore the spatial size to the same as the input image frame, and outputs the response region segmentation mask map of each frame image. Receive response region segmentation mask images of adjacent time frames, and construct a dynamic response consistency regularization loss function based on Euclidean distance; The temporal consistency loss term for the prediction results of consecutive frames is used in conjunction with the main loss function for model training optimization. Based on the response region segmentation mask map of each frame, cross-frame temporal alignment of the response region is performed. The dynamic time warping method is used to perform trajectory matching of the response region positions at multiple time points and outputs the response region temporal consistency identifier mapping. Based on the mapping between the response region segmentation mask and the temporal consistency identifier, area statistics, response intensity numerical analysis and time index positioning operations are performed on each response region to generate the corresponding response region area change curve, intensity change trend curve and maximum response time point, and to construct a structured response region temporal feature representation. The response region segmentation mask, time index information, and structured response region temporal feature representation are associated to generate structured response region recognition results.

2. The SPR response region identification method based on image semantic segmentation and temporal alignment according to claim 1, characterized in that, The acquisition of SPR image frame sequence data and the construction of the original image sequence specifically includes: The original image frames of the sensor chip surface are continuously acquired within a set time interval. The original image frames include image information reflecting changes in surface plasmon resonance. Record the corresponding time index at each time an image frame is acquired, so that the image frame is matched with its acquisition time. All image frames are arranged in chronological order according to their time indices to form a temporally continuous original image sequence.

3. The SPR response region identification method based on image semantic segmentation and temporal alignment according to claim 1, characterized in that, The step of performing image preprocessing on the original image sequence to output a standardized image sequence specifically includes: A Gaussian filtering method is used to perform filtering operations on each frame of the original image sequence; The image feature point matching method is used to perform spatial registration operation on image frames in the original image sequence to correct displacement or rotation deviations during the acquisition process. Perform brightness normalization on the registered image frames to unify the grayscale distribution range of the images; The image frames, after filtering, registration, and normalization, are reorganized in chronological order to construct a standardized image sequence.

4. The SPR response region identification method based on image semantic segmentation and temporal alignment according to claim 1, characterized in that, The step of constructing a time-series window image set consisting of multiple consecutive frames based on the standardized image sequence and its time index information specifically includes: Read the image frames and their corresponding time index information from the standardized image sequence; At each target time point, determine the time index corresponding to several adjacent time points, and select the image frame corresponding to the current time point and the adjacent time points from the standardized image sequence. The selected image frames are arranged in chronological order, and size and channel consistency processing is performed to construct a time-series window image set consisting of multiple consecutive frames.

5. The SPR response region identification method based on image semantic segmentation and temporal alignment according to claim 1, characterized in that, The method of using dynamic time warping to perform trajectory matching on the response region locations at multiple time points and outputting a temporal consistency identifier mapping for the response region specifically includes: Perform boundary extraction on the response region segmentation mask for each frame, determine the spatial centroid coordinates or boundary contour information of the response region in each frame image based on connected component analysis, and construct the response region location sequence. The dynamic time warping method is applied to the response region location sequence to calculate the minimum distance cost path between the current frame and the adjacent frames for each response region location, and a cross-frame response region mapping relationship is established based on the minimum cumulative distance. Based on the dynamic time warping calculation results, the response region trajectories with spatial continuity and minimum displacement cost in the time series are numbered and matched, and the inter-frame response region trajectory number labels are output. Construct a temporal consistency identifier mapping for the response region based on the trajectory number label of the response region.

6. The SPR response region identification method based on image semantic segmentation and temporal alignment according to claim 1, characterized in that, The construction of the structured response region temporal feature representation specifically includes: The response region segmentation mask map is mapped to the response region temporal consistency identifier, and the response region mask maps belonging to the same identifier number are combined into a response trajectory sequence according to the time index; Perform pixel statistics on the mask region corresponding to each response trajectory sequence, calculate the number of effective pixels in the response region in each frame, convert it into an area value according to the image spatial resolution, and construct the area change curve of the response region over time. By combining the original image frame sequence, the gray values ​​of the area covered by the response region mask in each frame are averaged to extract numerical indicators representing the response intensity, and then combined in chronological order to form an intensity change trend curve. Search for the global maximum point in the area change curve or intensity change curve, locate its corresponding time index, and determine the maximum response time point of the current response area; The area change curve, intensity change trend curve, and maximum response time point corresponding to each response trajectory are combined into a set of structured response region temporal feature representations.

7. The SPR response region identification method based on image semantic segmentation and temporal alignment according to claim 1, characterized in that, The step of associating the response region segmentation mask, time index information, and structured response region temporal feature representation to generate structured response region recognition results specifically includes: Receive response region segmentation mask, structured response region temporal feature representation, and time index information corresponding to each frame image; Based on the temporal consistency identifier mapping, the segmentation mask of each response region is associated with the corresponding area change curve, intensity change trend curve and the maximum response time point; In each response trajectory, the response duration interval of the response region is determined based on the range of consecutively appearing frame indexes in the response region segmentation mask image, and the complete spatiotemporal trajectory is marked by combining the spatial location of the response region in each frame image. The spatial location set, response duration range, and structured response region temporal feature representation of each response region are organized into response region identification result data items according to a predetermined field format, and the structured response region identification result is output.

Images