SPR response region identification method based on image semantic segmentation and time sequence alignment

Abstract

The invention discloses an SPR response region identification method based on image semantic segmentation and time sequence alignment, and the method comprises the following steps: collecting SPR image frame sequence data, and constructing an original image sequence; performing image preprocessing operation on the original image sequence, and outputting a standardized image sequence; constructing a time sequence window image set composed of multiple continuous frames; inputting the time sequence window image set into an improved SegFormer model, and generating a response region segmentation mask image corresponding to each frame; executing cross-frame time sequence alignment operation of the response area, and outputting time sequence consistency identification mapping of the response area; performing area statistics, intensity analysis and time positioning operation; and generating a structured response region recognition result. The spatial-temporal evolution process of the response area in the SPR image sequence can be effectively recognized, the accuracy and stability of response area recognition are improved, and the method is suitable for high-precision biological detection and real-time molecular analysis scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing and biosensing technology, and particularly relates to a SPR response region identification method based on image semantic segmentation and time sequence alignment. BACKGROUND

[0002] With the wide application of surface plasmon resonance (SPR) technology in biomolecular detection, drug screening and real-time reaction monitoring, how to accurately identify the response region from the SPR image sequence and quantify its time sequence change has become a key problem to improve the detection sensitivity and automatic analysis capability. The existing SPR response identification methods mainly rely on traditional image processing methods such as image gray difference threshold, regional mean change or global background subtraction to roughly judge the response region, but in actual application, the following problems are generally present: The response region often presents the characteristics of fuzzy edge, weak gray change and irregular shape, and the traditional methods based on fixed threshold or simple image difference are difficult to adapt to the weak response change under various experimental conditions, resulting in high missed detection rate of response region detection results and inaccurate spatial positioning; the response process in the SPR image sequence has obvious time sequence evolution characteristics, and the existing methods are mostly processed independently frame by frame, lacking the modeling capability of the continuity and dynamic trajectory of the response region across frames, and unable to realize the consistency tracking of the same response region in the time dimension; in addition, the existing methods usually do not fully calibrate and preprocess the displacement and rotation error in image acquisition, which is easy to cause error accumulation of response region identification affected by image disturbance, and reduce the stability and repeatability of analysis results.

[0003] Therefore, how to provide a SPR response region identification method based on image semantic segmentation and time sequence alignment is a problem to be solved by those skilled in the art. SUMMARY

[0004] One object of the present application is to provide a SPR response region identification method based on image semantic segmentation and time sequence alignment. The present application fully combines a deep vision segmentation network and a dynamic time warping algorithm to construct an improved SegFormer model for segmenting the response region in the SPR image, and introduces a cross-frame trajectory matching mechanism and a response feature analysis process, which has the advantages of high identification accuracy, strong time sequence consistency and high automation degree.

[0005] The SPR response region identification method based on image semantic segmentation and time sequence alignment according to the embodiment of the present application comprises the following steps: Collecting SPR image frame sequence data to construct an original image sequence; Performing image preprocessing operation on the original image sequence to output a standardized image sequence; Based on the standardized image sequence and its time index information, a time-series window image set consisting of multiple consecutive frames is constructed. Input the temporal window image set into the improved SegFormer model to generate the response region segmentation mask map corresponding to each frame; Based on the response region segmentation mask map of each frame, cross-frame temporal alignment of the response region is performed. The dynamic time warping method is used to perform trajectory matching of the response region positions at multiple time points and outputs the response region temporal consistency identifier mapping. Based on the mapping between the response region segmentation mask and the temporal consistency identifier, area statistics, response intensity numerical analysis and time index positioning operations are performed on each response region to generate the corresponding response region area change curve, intensity change trend curve and maximum response time point, and to construct a structured response region temporal feature representation. The response region segmentation mask, time index information, and structured response region temporal feature representation are associated to generate structured response region recognition results.

[0006] Optionally, the acquisition of SPR image frame sequence data and the construction of the original image sequence specifically includes: The original image frames of the sensor chip surface are continuously acquired within a set time interval. The original image frames include image information reflecting changes in surface plasmon resonance. Record the corresponding time index at each time an image frame is acquired, so that the image frame is matched with its acquisition time. All image frames are arranged in chronological order according to their time indices to form a temporally continuous original image sequence.

[0007] Optionally, the step of performing image preprocessing on the original image sequence to output a normalized image sequence specifically includes: A Gaussian filtering method is used to perform filtering operations on each frame of the original image sequence; The image feature point matching method is used to perform spatial registration operation on image frames in the original image sequence to correct displacement or rotation deviations during the acquisition process. Perform brightness normalization on the registered image frames to unify the grayscale distribution range of the images; The image frames, after filtering, registration, and normalization, are reorganized in chronological order to construct a standardized image sequence.

[0008] Optionally, the step of constructing a temporal window image set consisting of multiple consecutive frames based on the standardized image sequence and its time index information specifically includes: Read the image frames and their corresponding time index information from the standardized image sequence; At each target time point, determine the time index corresponding to its adjacent time points, and select the image frames corresponding to the current time point and the adjacent time points from the standardized image sequence; Arrange the selected multiple image frames in chronological order, perform size unification and channel consistency processing, and construct a time sequence window image set composed of continuous multiple frames.

[0009] Optionally, the time sequence window image set is input into the improved SegFormer model to generate a response region segmentation mask corresponding to each frame, specifically including: An improved SegFormer model is constructed, including an encoder with frame position coding enhancement, a time sequence cross attention mechanism, a saliency difference guide structure, a lightweight decoder, and a response consistency supervision module; The time sequence window image set is input into the encoder, and the encoder is constructed using a hybrid visual Transformer. Multi-scale feature embedding operations are performed on each frame image, and the time index is mapped to generate a frame position coding vector, which is fused with the original image embedding in the channel to output a multi-scale coding feature representation containing frame position information; The multi-scale coding feature representation is input into the time sequence cross attention mechanism to perform cross-frame channel attention coupling on the feature vectors at the same spatial position of the current time frame and the adjacent time frame, construct a time fusion feature, and output a cross-frame enhanced coding feature set; The cross-frame enhanced coding feature set is fused with the saliency guide map, which is generated by calculating the gray difference map of the current frame image and the previous frame image, extracting the response change feature through convolution, and normalizing through the Sigmoid function. After the guide map is expanded in the channel, it is fused with the coding feature set through channel-by-channel weighted fusion to output a response enhanced feature set; The response enhanced feature set is input into the lightweight decoder to perform layer-by-layer upsampling and feature restoration operations to restore to the spatial size consistent with the input image frame, and output the response region segmentation mask map of each frame image; The response region segmentation mask map of the adjacent time frame is received, and a dynamic response consistency regularization loss function is constructed based on the Euclidean distance; The time consistency loss term of the output continuous frame prediction result is combined with the main loss function for model training and optimization.

[0010] Optionally, the dynamic time warping method is used to perform trajectory matching on the response region positions of multiple time points to output a response region time sequence consistency identification mapping, specifically including: Boundary extraction operations are performed on each frame response region segmentation mask map to determine the spatial centroid coordinates or boundary contour information of the response region in each frame image based on connected domain analysis to construct a response region position sequence; Applying dynamic time warping method to the response region position sequence, calculating the minimum distance cost path of each response region position between the current frame and the adjacent frame, and establishing the mapping relationship of the response region across frames based on the minimum cumulative distance; According to the dynamic time warping calculation result, the response region trajectory with spatial continuity and minimum displacement cost in the time sequence is numbered and matched, and the inter-frame response region trajectory number label is output; Based on the response region trajectory number label, the response region time sequence consistency identification mapping is constructed.

[0011] Optionally, the constructing structured response region time sequence feature representation specifically comprises: Receiving the response region segmentation mask map and the response region time sequence consistency identification mapping, combining the response region mask maps belonging to the same identification number into response trajectory sequences according to the time index; Performing pixel statistical operation on the corresponding mask region in each response trajectory sequence, calculating the effective pixel number of the response region in each frame, and converting it into area value according to the image spatial resolution, and constructing the area change curve of the response region with time change; Combining the original image frame sequence, performing average processing on the gray value in the area covered by the response region mask map in each frame, extracting the numerical index representing the response intensity, and combining them in time sequence to form the intensity change trend curve; Searching for the global maximum value point in the area change curve or the intensity change curve, locating the time index corresponding to it, and determining the maximum response time point of the current response region; Combining the area change curve, the intensity change trend curve and the maximum response time point corresponding to each response trajectory into a set of structured response region time sequence feature representation.

[0012] Optionally, the associating the response region segmentation mask map, the time index information and the structured response region time sequence feature representation to generate the structured response region identification result specifically comprises: Receiving the response region segmentation mask map, the structured response region time sequence feature representation, and the time index information corresponding to each frame image; According to the time sequence consistency identification mapping, associating each response region segmentation mask map with the corresponding area change curve, intensity change trend curve and maximum response time point; In each response trajectory, according to the frame index range continuously appearing in the response region segmentation mask map, the response duration interval of the response region is determined, and combined with the spatial position of the response region in each frame image, the complete space-time trajectory is labeled; The spatial position set of each response area, the response duration range and the structured response area timing feature representation are organized as a response area identification result data item in a predetermined field format, and a structured response area identification result is output.

[0013] The present application has the following advantages: (1) The improved SegFormer model of the present application comprises frame position coding enhancement, timing cross attention mechanism and significant difference guide structure, fully utilizes the multi-frame information in the timing window image set, realizes high-precision semantic-level segmentation of the response area with weak gray scale change and fuzzy edge in the SPR image sequence, and significantly improves the detection ability of weak response targets compared with the traditional image difference or threshold method. (2) The present application introduces a dynamic time warping method to perform optimal trajectory matching on the spatial position of the response area in each frame image, constructs a timing consistency identifier mapping of the response area, solves the problem of incoherent response trajectory and the inability of the same area to correspond across frames caused by the traditional method of independent processing frame by frame, realizes accurate association and number tracking of the response area in the time dimension, and enhances the complete modeling ability of the system to the response process. (3) The present application further combines the segmentation mask image and the trajectory identifier to carry out response area area statistics, intensity change trend analysis and maximum response time point positioning operation, generates a structured response area timing feature representation, and uniformly organizes the time index and spatial position information, outputs a structured response area identification result, and improves the automatic interpretation ability of the SPR system to the detection process and the quantitative expression ability of the response dynamic change. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, together with the embodiments of the present application, to explain the present application, and do not constitute a limitation of the present application. In the drawings: Figure 1 Flow chart of the SPR response area identification method based on image semantic segmentation and timing alignment proposed by the present application; Figure 2 Schematic diagram of the SPR response area identification method based on image semantic segmentation and timing alignment proposed by the present application; Figure 3 Framework diagram of the improved SegFormer model in the SPR response area identification method based on image semantic segmentation and timing alignment proposed by the present application. DETAILED DESCRIPTION

[0015] The present application will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, and only illustrate the basic structure of the present application in a schematic manner, and therefore only show the components related to the present application.

[0016] Reference Figures 1-3 The SPR response region identification method based on image semantic segmentation and time sequence alignment includes the following steps: Step one: collect SPR image frame sequence data and construct an original image sequence; Step two: perform image preprocessing operations on the original image sequence to output a standardized image sequence; Step three: construct a time sequence window image set composed of multiple consecutive frames according to the standardized image sequence and its time index information; Step four: input the time sequence window image set into the improved SegFormer model to generate a response region segmentation mask map corresponding to each frame; Step five: perform cross-frame time sequence alignment operations on the response region based on the response region segmentation mask map of each frame, and use the dynamic time warping method to perform trajectory matching on the response region positions at multiple time points to output a response region time sequence consistency identification mapping; Step six: according to the response region segmentation mask map and the time sequence consistency identification mapping, perform area statistics, response intensity numerical analysis and time index positioning operations on each response region to generate corresponding response region area change curves, intensity change trend curves and maximum response time points, and construct a structured response region time sequence feature representation; Step seven: associate the response region segmentation mask map, time index information and structured response region time sequence feature representation to generate a structured response region identification result.

[0017] In this embodiment, the collection of SPR image frame sequence data and the construction of the original image sequence specifically include: Continuously collect original image frames on the surface of the sensing chip within a set time interval, and the original image frames include image information reflecting surface plasmon resonance changes; Record the corresponding time index at the same time as each image frame is collected, so that the image frame and its collection time correspond one-to-one; Arrange all the image frames in the order of time index to form a time-continuous original image sequence.

[0018] This embodiment effectively constructs a time-continuous original image sequence by continuously collecting original image frames on the surface of the sensing chip within a set time interval and sequentially arranging them in combination with the corresponding time index information of each frame. This method not only ensures the time sequence integrity of the SPR image data, but also provides a high-quality input basis for subsequent time sequence modeling and response trajectory analysis, significantly improving the accuracy and consistency of response region identification.

[0019] In this embodiment, the image preprocessing operations performed on the original image sequence to output a standardized image sequence specifically include: performing a filtering operation on each image frame in the original image sequence by using a Gaussian filtering method; performing a spatial registration operation on the image frames in the original image sequence based on an image feature point matching method to correct displacement or rotation deviation in the acquisition process; performing a brightness normalization operation on the image frames after registration to unify the image gray scale distribution range; reorganizing the image frames after filtering, registration and normalization processing in time index order to construct a standardized image sequence.

[0020] The embodiment effectively suppresses image noise, corrects inter-frame displacement and rotation error, and unifies image gray scale by sequentially performing Gaussian filtering, image feature point spatial registration and brightness normalization operations on the original image sequence, thereby generating a standardized image sequence with stable quality and continuous time sequence, providing accurate and consistent input data for subsequent semantic segmentation and response feature extraction, and significantly improving the robustness and accuracy of the overall recognition effect.

[0021] In the embodiment, the time sequence window image set composed of continuous multiple frames is constructed according to the standardized image sequence and its time index information, specifically including: reading the image frames in the standardized image sequence and their corresponding time index information; at each target time point, determining the time indexes corresponding to the adjacent time points, and selecting the image frames corresponding to the current time point and the adjacent time points from the standardized image sequence; arranging the selected multiple image frames in time sequence order, performing size unification and channel consistency processing to construct a time sequence window image set composed of continuous multiple frames.

[0022] The embodiment selects the image frames corresponding to the adjacent time points at each target time point, constructs a time sequence window image set composed of continuous multiple frames in combination with the time index information, and performs size unification and channel consistency processing on the selected image frames, thereby ensuring the structural consistency and contextual coherence of the input image in the time sequence modeling process, effectively enhancing the perception ability of the subsequent model to the time dynamic features, and improving the time sequence stability and accuracy of the response region identification.

[0023] In the embodiment, the time sequence window image set is input into the improved SegFormer model to generate a response region segmentation mask corresponding to each frame, specifically including: constructing an improved SegFormer model, including an encoder with frame position encoding enhancement, a time sequence cross-attention mechanism, a saliency difference guided structure, a lightweight decoder and a response consistency supervision module; The time window image set is input to an encoder, the encoder adopts a hybrid vision Transformer to construct, performs multi-scale feature embedding operation on each frame image, and generates a frame position encoding vector through embedding mapping of a time index, fuses a channel with an original image embedding, and outputs a multi-scale coding feature representation containing frame position information; The multi-scale coding feature representation is input to a time cross attention mechanism, the feature vectors of the current time frame and adjacent time frames at the same spatial position are coupled through cross-frame channel attention, and time fusion features are constructed The specific calculation method is as follows: ; Among them, 、 、 are a current frame query vector and a key vector and a value vector of the previous frame obtained through linear transformation, is a scaling factor constructed based on the cross-frame cross attention calculation principle of the Transformer, and the output is a cross-frame enhanced coding feature set; The cross-frame enhanced coding feature set is fused with a saliency guide map, the saliency guide map is generated by calculating a gray difference map of the current frame image and the previous frame image, extracting response change features through convolution, and normalizing through a Sigmoid function, the guide map is expanded in channel and fused with the coding feature set through channel-by-channel weighted fusion, and a response enhanced feature set is output; The response enhanced feature set is input to a lightweight decoder to perform layer-by-layer upsampling and feature restoration operations, and is restored to a spatial size consistent with the input image frame, and a response region segmentation mask map of each frame image is output; The response region segmentation mask map of the adjacent time frame is received, and a dynamic response consistency regularization loss function is constructed based on the Euclidean distance : ; Among them and represent the response region segmentation mask maps output by the model on the frame image and the previous frame image; The time consistency loss term of the output continuous frame prediction result is combined with the main loss function for model training and optimization.

[0024] The embodiment constructs an improved SegFormer model, uses an encoder enhanced by frame position coding, a time cross attention mechanism, and a saliency difference guide structure to model the deep features of continuous image frames, introduces time index information to enhance the time perception ability of the model, and uses a cross-frame attention mechanism to realize dynamic fusion of response features, effectively improving the recognition ability of weak response areas. At the same time, a dynamic response consistency regularization loss function is constructed to perform Euclidean distance constraint on the segmentation results of adjacent frames, and the consistency of the response area evolution over time is strengthened, and finally a more accurate and more time-continuous response area segmentation mask graph is generated. This approach significantly enhances the robustness and dynamic tracking stability of the SPR image response area segmentation.

[0025] In the embodiment, the dynamic time warping method is used to match the trajectories of the response area positions at multiple time points, and a response area time consistency identification mapping is output, which specifically includes: Performing a boundary extraction operation on each frame of response area segmentation mask graph, determining the spatial centroid coordinates or boundary contour information of the response area in each frame of image based on connected component analysis, and constructing a response area position sequence; Applying the dynamic time warping method to the response area position sequence, calculating the minimum distance cost path of each response area position between the current frame and the adjacent frame, and establishing the mapping relationship of the cross-frame response area based on the minimum cumulative distance; According to the dynamic time warping calculation result, the response area trajectories with spatial continuity and minimum displacement cost in time sequence are numbered and matched, and the inter-frame response area trajectory number label is output; Based on the response area trajectory number label, a response area time consistency identification mapping is constructed.

[0026] The embodiment precisely obtains the position coordinates of the response area by performing boundary extraction and connected component analysis on each frame of response area segmentation mask graph, and matches and numbers the response area trajectories between different time frames using the dynamic time warping method, realizing stable tracking and number consistency labeling of cross-frame response areas. This method effectively improves the time consistency and spatial continuity of the response area recognition result in the SPR image, which helps to improve the accuracy and interpretability of subsequent response trend analysis.

[0027] In the embodiment, the construction of the structured response area time sequence feature representation specifically includes: Receiving the response area segmentation mask graph and the response area time consistency identification mapping, and combining the response area mask graphs belonging to the same identification number into a response trajectory sequence according to the time index; Performing pixel statistics operation on the corresponding mask area in each response trajectory sequence, calculating the number of effective pixels in the response area of each frame, and converting it into area value according to the image space resolution, constructing the area change curve of the response area with time; In combination with the original image frame sequence, the gray value in the area covered by the response area mask graph in each frame is averaged, the numerical index representing the response intensity is extracted, and the intensity change trend curve is formed by combining in time sequence; Search for the global maximum value point in the area change curve or the intensity change curve, locate the time index corresponding to it, and determine the maximum response time point of the current response area; Combine the area change curve and the intensity change trend curve corresponding to each response trajectory with the maximum response time point to form a set of structured response area time sequence feature representation.

[0028] The embodiment combines the response area segmentation mask graph with the time sequence consistency identification mapping result, constructs the complete time sequence trajectory sequence of the response area, and performs pixel statistics and gray analysis on the area in each frame of image, respectively generating the area change curve and the intensity change trend curve. At the same time, the maximum response time point is accurately extracted and the structured response area time sequence feature representation is constructed. This method can realize the quantitative analysis and feature modeling of the SPR response area in the time dimension, and improve the visualization, monitoring and intelligent interpretation ability of the dynamic response process.

[0029] In the embodiment, the response area segmentation mask graph, time index information and structured response area time sequence feature representation are associated to generate a structured response area identification result, specifically including: Receiving response area segmentation mask graph, structured response area time sequence feature representation, and time index information corresponding to each frame of image; According to the time sequence consistency identification mapping, each response area segmentation mask graph is associated with the corresponding area change curve, intensity change trend curve and maximum response time point; In each response trajectory, according to the frame index range continuously appearing in the response area segmentation mask graph, the response duration interval of the response area is determined, and the complete space-time trajectory is labeled in combination with the spatial position of the response area in each frame of image; Organize the spatial position set, response duration range and structured response area time sequence feature representation of each response area into response area identification result data items according to the predetermined field format, and output the structured response area identification result.

[0030] The embodiment is associated with time index information by dividing the response area segmentation mask map and the structured response area time sequence feature representation, and completes the comprehensive labeling of the spatial position, duration and dynamic characteristics of the response trajectory according to the time sequence consistency identification mapping. The method can accurately extract and output the structured response area identification result, has good interpretability and visualization features, and significantly improves the automatic identification accuracy and analysis efficiency of the SPR response area.

[0031] Embodiment 1 In order to verify the feasibility of the application in implementation, the application is applied to the SPR (surface plasmon resonance) molecular recognition experiment carried out by a research institution, aiming to realize the automatic identification and cross-frame tracking of the response area, and further improve the accuracy and efficiency of the SPR response analysis. The experimental scene is selected from the actual SPR image acquisition system, and in the observation process of 10 seconds, one frame of SPR gray image is obtained every second, the image resolution is 512*512 pixels, and a complete image sequence is formed. The experimental setting includes two kinds of target protein molecules acting on the chip surface to produce local response, numbered as R1 and R2, corresponding to the binding area of different molecules. After data acquisition, the images are input into the SPR response area automatic identification system constructed by the application method for processing.

[0032] In the data processing process, first, the original image sequence is subjected to Gaussian filtering and brightness normalization operation to suppress background random noise and enhance local contrast. Then, an improved SegFormer model is used to perform frame-by-frame response area semantic segmentation. The model introduces a cross-scale attention mechanism and a structure nesting guidance strategy, effectively improving the identification ability of weak response areas. For the time sequence change behavior of the response area, an intra-frame saliency prediction enhancement branch is constructed, and the context feature maps of adjacent frames are fused to maintain the consistency of the response area boundary through a time perception attention mechanism. The segmentation result of each frame is output as a binary mask map, which identifies the significant response area in the current frame image.

[0033] Subsequently, the dynamic time warping (DTW) method is introduced to perform cross-frame response trajectory alignment operation. The system performs trajectory pairing and adjustment on each response area according to the region centroid coordinate sequence and gray response intensity change trend, and generates a complete response area time sequence consistency identification mapping. Based on the time sequence consistency mask and the original response area mask, the area (pixel number), average response intensity (gray mean) and maximum response time point of each region at different time points are further calculated and encoded as structured time sequence feature representation.

[0034] Table 1 is the SPR response area detection result table, which shows the area, intensity and semantic segmentation accuracy (IoU) of R1 and R2 response areas at 10 time indexes: Table 1 SPR response region detection result table Time Index Response Area R1 - Area Response Area R1 - Intensity Response Area R1 - IoU Response Area R2 - Area Response Area R2 - Intensity Response Area R2 - IoU 0 126.65 0.874 0.874 161.42 0.782 0.919 1 140.71 0.778 0.876 172.71 0.812 0.797 2 137.10 0.728 0.790 169.08 0.712 0.887 3 128.91 0.539 0.773 163.03 0.830 0.769 4 117.30 0.746 0.902 143.69 0.612 0.761 5 110.18 0.864 0.795 157.33 0.963 0.812 6 105.97 0.688 0.926 180.00 0.991 0.951 7 118.65 0.718 0.864 162.50 0.901 0.824 8 121.38 0.759 0.904 149.36 0.710 0.850 9 130.00 0.804 0.892 145.71 0.660 0.843 Analysis of the data in Table 1 can find that the area of response region R1 gradually increases from 0 to 2 seconds, from 126.65 to 140.71, and the response intensity slightly decreases, indicating that the initial binding region gradually diffuses but the unit response decreases. During 4 to 6 seconds, the area decreases but the intensity increases, indicating that the local binding focuses and produces stronger reflection signals. The IoU accuracy of semantic segmentation is stable between 0.77 and 0.92, indicating that the model has stable recognition ability.

[0035] The area of response region R2 reaches a maximum of 180.00 at 5 to 6 seconds, and the intensity also reaches a peak of 0.991 at this stage, indicating that it is the main reaction region. The segmentation IoU value also reaches a maximum of 0.951 during this period, proving that the method has good accuracy in the case of large response. In addition, the R2 region gradually shrinks in area and the intensity falls in the later stage, reflecting the dissociation process of molecules, and the response characteristics decay significantly over time.

[0036] The time sequence consistency identifier generated by the method can accurately locate the maximum response time point of each region and generate a complete response trend graph, effectively supporting dynamic reaction analysis in SPR biological experiments.

[0037] The embodiment significantly improves the accuracy of response region identification and the stability of cross-frame time sequence analysis by applying the method of the present application in real SPR experimental scenarios, solves the problems of traditional methods in response boundary ambiguity, region drift and time trajectory fracture, and demonstrates good engineering practical value and intelligent automation potential.

[0038] The above describes only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can make equivalent replacements or changes to the technical solutions and inventive concepts of the present application within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for SPR response region recognition based on image semantic segmentation and temporal alignment, characterized in that, Includes the following steps: Acquire SPR image frame sequence data and construct the original image sequence; Perform image preprocessing operations on the original image sequence to output a normalized image sequence; Based on the standardized image sequence and its time index information, a time-series window image set consisting of multiple consecutive frames is constructed. Input the temporal window image set into the improved SegFormer model to generate the response region segmentation mask map corresponding to each frame; Based on the response region segmentation mask map of each frame, cross-frame temporal alignment of the response region is performed. The dynamic time warping method is used to perform trajectory matching of the response region positions at multiple time points and outputs the response region temporal consistency identifier mapping. Based on the mapping between the response region segmentation mask and the temporal consistency identifier, area statistics, response intensity numerical analysis and time index positioning operations are performed on each response region to generate the corresponding response region area change curve, intensity change trend curve and maximum response time point, and to construct a structured response region temporal feature representation. The response region segmentation mask, time index information, and structured response region temporal feature representation are associated to generate structured response region recognition results.

2. The SPR response region identification method based on image semantic segmentation and temporal alignment according to claim 1, characterized in that, The acquisition of SPR image frame sequence data and the construction of the original image sequence specifically includes: The original image frames of the sensor chip surface are continuously acquired within a set time interval. The original image frames include image information reflecting changes in surface plasmon resonance. Record the corresponding time index at each time an image frame is acquired, so that the image frame is matched with its acquisition time. All image frames are arranged in chronological order according to their time indices to form a temporally continuous original image sequence.

3. The SPR response region identification method based on image semantic segmentation and temporal alignment according to claim 1, characterized in that, The step of performing image preprocessing on the original image sequence to output a standardized image sequence specifically includes: A Gaussian filtering method is used to perform filtering operations on each frame of the original image sequence; The image feature point matching method is used to perform spatial registration operation on image frames in the original image sequence to correct displacement or rotation deviations during the acquisition process. Perform brightness normalization on the registered image frames to unify the grayscale distribution range of the images; The image frames, after filtering, registration, and normalization, are reorganized in chronological order to construct a standardized image sequence.

4. The SPR response region identification method based on image semantic segmentation and temporal alignment according to claim 1, characterized in that, The step of constructing a time-series window image set consisting of multiple consecutive frames based on the standardized image sequence and its time index information specifically includes: Read the image frames and their corresponding time index information from the standardized image sequence; At each target time point, determine the time index corresponding to several adjacent time points, and select the image frame corresponding to the current time point and the adjacent time points from the standardized image sequence. The selected image frames are arranged in chronological order, and size and channel consistency processing is performed to construct a time-series window image set consisting of multiple consecutive frames.

5. The SPR response region identification method based on image semantic segmentation and temporal alignment according to claim 1, characterized in that, The step of inputting the temporal window image set into the improved SegFormer model to generate a response region segmentation mask for each frame specifically includes: An improved SegFormer model is constructed, including an encoder with enhanced frame position coding, a temporal cross-attention mechanism, a saliency differential guidance structure, a lightweight decoder, and a response consistency supervision module; The temporal window image set is input to the encoder, which is constructed using a hybrid vision Transformer. It performs multi-scale feature embedding operations on each frame image and generates a frame position encoding vector by embedding the time index through the embedding mapping. This vector is then fused with the original image embedding to output a multi-scale encoded feature representation containing frame position information. The multi-scale encoded feature representation is input into the temporal cross-attention mechanism. Cross-frame channel attention coupling is performed on the feature vectors of the current time frame and the adjacent time frame that are in the same spatial position to construct temporal fusion features. The output is a set of cross-frame enhanced encoded features. The cross-frame enhanced coding feature set is fused with the saliency guide map. The saliency guide map is generated by calculating the grayscale difference map between the current frame image and the previous frame image, extracting response change features through convolution, and normalizing it with the Sigmoid function. After channel expansion, the guide map is fused with the coding feature set through a channel-wise weighted fusion to output a response-enhanced feature set. The response-enhanced feature set is input into the lightweight decoder, which performs layer-by-layer upsampling and feature restoration operations to restore the spatial size to the same as the input image frame, and outputs the response region segmentation mask map of each frame image. Receive response region segmentation mask images of adjacent time frames, and construct a dynamic response consistency regularization loss function based on Euclidean distance; The temporal consistency loss term for the prediction results of consecutive frames is used in conjunction with the main loss function for model training optimization.

6. The SPR response region identification method based on image semantic segmentation and temporal alignment according to claim 1, characterized in that, The method of using dynamic time warping to perform trajectory matching on the response region locations at multiple time points and outputting a temporal consistency identifier mapping for the response region specifically includes: Perform boundary extraction on the response region segmentation mask for each frame, determine the spatial centroid coordinates or boundary contour information of the response region in each frame image based on connected component analysis, and construct the response region location sequence. The dynamic time warping method is applied to the response region location sequence to calculate the minimum distance cost path between the current frame and the adjacent frames for each response region location, and a cross-frame response region mapping relationship is established based on the minimum cumulative distance. Based on the dynamic time warping calculation results, the response region trajectories with spatial continuity and minimum displacement cost in the time series are numbered and matched, and the inter-frame response region trajectory number labels are output. Construct a temporal consistency identifier mapping for the response region based on the trajectory number label of the response region.

7. The SPR response region identification method based on image semantic segmentation and temporal alignment according to claim 1, characterized in that, The construction of the structured response region temporal feature representation specifically includes: The response region segmentation mask map is mapped to the response region temporal consistency identifier, and the response region mask maps belonging to the same identifier number are combined into a response trajectory sequence according to the time index; Perform pixel statistics on the mask region corresponding to each response trajectory sequence, calculate the number of effective pixels in the response region in each frame, convert it into an area value according to the image spatial resolution, and construct the area change curve of the response region over time. By combining the original image frame sequence, the gray values ​​of the area covered by the response region mask in each frame are averaged to extract numerical indicators representing the response intensity, and then combined in chronological order to form an intensity change trend curve. Search for the global maximum point in the area change curve or intensity change curve, locate its corresponding time index, and determine the maximum response time point of the current response area; The area change curve, intensity change trend curve, and maximum response time point corresponding to each response trajectory are combined into a set of structured response region temporal feature representations.

8. The SPR response region identification method based on image semantic segmentation and temporal alignment according to claim 1, characterized in that, The step of associating the response region segmentation mask, time index information, and structured response region temporal feature representation to generate structured response region recognition results specifically includes: Receive response region segmentation mask, structured response region temporal feature representation, and time index information corresponding to each frame image; Based on the temporal consistency identifier mapping, the segmentation mask of each response region is associated with the corresponding area change curve, intensity change trend curve and the maximum response time point; In each response trajectory, the response duration interval of the response region is determined based on the range of consecutively appearing frame indexes in the response region segmentation mask image, and the complete spatiotemporal trajectory is marked by combining the spatial location of the response region in each frame image. The spatial location set, response duration range, and structured response region temporal feature representation of each response region are organized into response region identification result data items according to a predetermined field format, and the structured response region identification result is output.

Images