Space-time inconsistency face forgery detection method based on state space model

Abstract

The invention relates to the field of computer vision, in particular to a space-time inconsistency face forgery detection method based on a state space model (State Space Model). The method comprises the following steps: extracting a continuous face frame sequence, and constructing a double-branch detection model based on a space-time hybrid architecture; the method comprises the following steps: firstly, explicitly modeling motion change between video frames by using a time sequence difference module; then the features are respectively sent to a time branch and a space branch, the time branch adopts a visual state space model (Vision Mama) to combine with a time sequence grid splicing strategy to capture global time sequence dependence of a long sequence with linear complexity, and the space branch extracts local space-time textures by using a 3D convolutional network; in the process, semantic alignment and fusion of the two branches are realized by utilizing a time channel modulation (TCM) module and a time sequence channel attention (TCA) module; and finally, inputting the fusion features into a classifier to obtain a counterfeit detection result. According to the method, the detection precision and generalization ability of the model on the face counterfeit visual s frequency under the low-image-quality and complex scene are improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a method for detecting spatiotemporal inconsistencies in face forgery based on a state-space model. Background Technology

[0002] In recent years, with the rapid development of deep learning technology, face spoofing techniques have undergone iterative evolution from simple image stitching and autoencoders to generative adversarial networks and the latest diffusion models. The spoofed face videos generated by these techniques pose a serious challenge to public safety. Therefore, developing efficient and robust face spoofing detection technologies has become an urgent need in the fields of computer vision and multimedia forensics.

[0003] Early face forgery detection techniques primarily focused on using convolutional neural networks to uncover spatial artifacts or texture anomalies in single-frame images. However, the latest generative techniques can now produce forged videos that obscure most spatial artifact details. Especially with video compression and transcoding during network transmission, previously subtle spatial artifacts are often destroyed or smoothed out, causing detection methods relying solely on spatial texture features to experience a sharp decline in performance when faced with low-quality or high-quality forged videos.

[0004] Faced with the limitations of spatial features, researchers have gradually turned their attention to temporal features, utilizing the differences in motion continuity between real and fake faces (i.e., spatiotemporal inconsistency) between frames for discrimination. Most mainstream temporal detection methods currently employ the Transformer architecture, utilizing its self-attention mechanism to capture long-range dependencies between video frames. While the Transformer performs excellently in temporal modeling, the computational complexity of its attention mechanism is significant. This means that processing long sequences of video frames incurs a huge computational overhead.

[0005] In summary, designing a network framework that has low computational overhead but can accurately capture spatiotemporal inconsistencies in fake videos has become a key technical problem that urgently needs to be solved in current research on face forgery detection. Summary of the Invention

[0006] To address the aforementioned problems, this invention proposes a spatiotemporal inconsistency face forgery detection method based on a state-space model and 3DCNN. This method aims to overcome the limitations of existing spatial texture-based detection models in handling highly compressed videos, and the high computational complexity and resource consumption of Transformer-based temporal detection models. By combining a visual state-space model with 3DCNN, this invention can explicitly model the spatiotemporal inconsistencies implicit in forged videos, thereby significantly improving the detection accuracy and generalization ability of the model in both low-quality and high-quality forgery scenarios while maintaining computational efficiency.

[0007] To achieve the above-mentioned technical objectives, the specific technical solution of the present invention is as follows:

[0008] A method for detecting spatiotemporal inconsistencies in face forgery based on a state-space model, which mainly includes the following steps:

[0009] S1: Collect a video dataset containing real and fake faces, and perform face detection on the video to obtain face location information and a fixed-length sequence of face video frames;

[0010] S2: Construct a dual-branch spatiotemporal hybrid network model suitable for face forgery detection;

[0011] S3: Input the video frame sequence processed in S1 into the model constructed in S2 for training. Calculate the loss function and update the network parameters through backpropagation to obtain the trained network model.

[0012] S4: Obtain the target video to be detected, preprocess it according to the rules described in S1, input it into the model trained in S3, and output the probability score of the video being a fake face.

[0013] Furthermore, in S1, T consecutive video frames are randomly extracted from each input video, where T is set to a perfect square. Then, facial features are extracted from each video frame and used as input to the model.

[0014] Furthermore, in S2, the dual-branch spatiotemporal hybrid network model consists of the following modules or branches: a learnable temporal difference module (TDM) for displaying dynamic change information between modeling frames; a temporal branch for extracting global dependencies of long sequences, which is constructed based on a visual state space model; a spatial branch for extracting local spatiotemporal textures, which is constructed based on a 3D convolutional neural network; and a temporal channel attention module (TCA) and a temporal channel modulation module (TCM) for connecting the two branches and performing feature interaction.

[0015] Furthermore, the workflow of the learnable temporal difference module includes: extracting shallow features of the input sequence using a convolutional neural network and calculating the difference between features at adjacent time steps. This process includes:

[0016]

[0017] in, This represents the inter-frame difference sequence of a video frame sequence. This represents a convolutional neural network used for modeling temporal spaces. This represents a sequence of video frames that have passed through the Stem layer.

[0018] After obtaining the inter-frame difference sequence, zero-padding is performed at the end of the sequence's time dimension, and temporal channel attention is used to enhance the difference features. This process can be represented as:

[0019]

[0020] in, This represents the learnable parameters used to dynamically scale the difference features. TCA stands for Temporal Channel Attention Module.

[0021] Furthermore, the time branch employs a temporal grid stitching strategy: the inter-frame difference sequence obtained in S1 is arranged in a spatial grid according to temporal order, folding the temporal dimension into the spatial dimension to reconstruct a high-resolution two-dimensional feature map, the length and width of which are both [missing information]. The feature map is then embedded and input into the visual state space model for bidirectional sequence modeling. This process can be represented as:

[0022]

[0023] Furthermore, in the spatial branching network structure: In the bottleneck structure of the 3D convolutional layer, an asymmetric convolutional kernel with a temporal dimension kernel size of 2 is used. This is used to align inter-frame semantic offsets caused by differential operations and to capture local motion changes between adjacent frames.

[0024] Furthermore, the working process of the temporal channel attention module includes: firstly, performing global average pooling on the input feature map in the spatial dimension to aggregate and obtain channel statistical features that retain the temporal dimension. This process can be represented as:

[0025]

[0026] in, This represents the input feature map of the representation module. This represents the average pooling operation on spatial dimensions H and W. This represents the intermediate features after spatial pooling.

[0027] Subsequently, a multilayer perceptron is used to perform a nonlinear transformation on the statistical feature, and a channel attention weight is generated using a sigmoid activation function. This process can be represented as follows:

[0028]

[0029] in, and Let represent the one-dimensional convolution weight parameters of the first and second layers in a multilayer perceptron, respectively. Used for dimensionality reduction. Used for dimensional recovery of channel count. Represents the ReLU activation function. This represents the Sigmoid activation function, used to map attention weights to the (0, 1) interval. This represents the generated temporal channel attention weight map.

[0030] Obtain attention weights The sequence then needs to be applied to the original sequence and residual joined. This process includes:

[0031]

[0032] in This represents element-wise multiplication, using a broadcast mechanism to apply weights to each spatial pixel. This represents the output features after attention enhancement.

[0033] In the branching loop, it is also necessary to... Pooling and projection are performed to transform the spatial dimension into the temporal dimension. This process includes:

[0034]

[0035] Furthermore, the operation of the time channel modulation module includes: firstly, acquiring the sequence features of the time branch output, and then mapping them to gate weights with values ​​between 0 and 1 through a linear projection layer. This process can be expressed as:

[0036]

[0037] in, This represents the sequence features of the time branch output. This represents the projection matrix, used to increase dimensionality. This indicates that features are broadcast to the spatial dimension. This represents the Sigmoid activation function.

[0038] The gate weights are then aligned dimensionally with the feature maps of the spatial branch. Finally, a learnable scaling factor initialized to zero is introduced. The aligned gate weights are multiplied by the spatial features, scaled, and then added back to the original spatial features via a residual connection. This process injects the global temporal context into the spatial branch, which can be represented as:

[0039]

[0040] in, This indicates element-wise multiplication. To introduce a learnable scaling factor, : Represents the output characteristics after modulation by global timing information.

[0041] Furthermore, in S3, the main process of inputting the processed video frame sequence from S1 into the model constructed in S2 for training and obtaining the trained network model includes:

[0042] First, the preprocessed video frame sequence is input into the network. It first undergoes preliminary dimensionality reduction through a shallow convolutional network, and then the input is fed into the spatial branch and the temporal branch as initial inputs.

[0043] Then, in the spatial branch, the initial input passes through stacked 3D convolutional bottleneck modules layer by layer; at each stage, the spatial features first receive the gating signal from the temporal branch through the temporal channel modulation module to perform feature calibration, integrate global temporal anomaly information into the spatial features, and then pass through the convolutional layer to extract local texture information and perform downsampling to enter the next stage.

[0044] Secondly, in the temporal branch, the initial input first passes through the learnable temporal difference module to extract inter-frame motion differences; then it is transformed into a two-dimensional sequence through the grid stitching strategy; subsequently, before each stage, this sequence is fused with spatial information through the temporal channel attention module described in the claim. After passing through the embedding layer, the sequence enters the visual state space model block for bidirectional state space scanning and global modeling. This process can be represented as:

[0045]

[0046] in This represents a learnable location embedding that includes spatial and temporal location information. Represents the projection matrix. This represents the flattened vector of the j-th image patch segmented from the temporal grid feature map. This indicates the [CLS] token. This indicates positional embedding.

[0047] Generated global time-series token In addition to being used for its own deep feature extraction, it is also used as a global guiding signal, input to the aforementioned time channel modulation module, and adaptively modulates the spatial branch features of the same level in terms of channel dimension by generating gating weights.

[0048] Finally, after multiple iterations of the above process, the resulting feature vector is input into a fully connected classification layer to obtain the predicted probability value and output the prediction result.

[0049] The present invention has at least the following beneficial effects

[0050] Compared with existing face forgery detection methods, this invention significantly improves the performance degradation caused by the loss of spatial texture due to video compression. Based on a state-space 3DCNN model, this method explicitly models the spatiotemporal inconsistencies between video frames, achieving accurate capture of subtle temporal jitter in low-quality and complex scenes. Furthermore, thanks to the linear computational complexity of the state-space model, this invention can process longer video sequences with low computational overhead, effectively balancing detection accuracy and efficiency.

[0051] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0052] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0053] Figure 1 This is a flowchart of the method of the present invention. Figure 2 This is a structural diagram of the hybrid spatiotemporal network model mentioned in this invention. Detailed Implementation

[0054] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0055] Please see Figure 1 This invention provides a method for detecting spatiotemporal inconsistencies in face forgery based on a state-space model. The method mainly includes the following steps:

[0056] (1): Collect video datasets containing real faces and fake faces, and perform face detection on the videos to obtain face location information and a fixed-length sequence of face video frames;

[0057] (2): Construct a dual-branch spatiotemporal hybrid network model suitable for face forgery detection;

[0058] (3): Input the video frame sequence processed in (1) into the model constructed in (2) for training. Calculate the loss function and backpropagate to update the network parameters to obtain the trained network model.

[0059] (4): Obtain the target video to be detected, preprocess it according to the rules described in (1), input it into the model trained in (3), and output the probability score of the video being a fake face.

[0060] In a preferred embodiment of the present invention, in step (1), T consecutive video frames are randomly extracted from each input video, where T is set to a perfect square. Subsequently, the face portion is extracted from each video frame as the input to the model.

[0061] As a preferred embodiment of the present invention, in step (2), the dual-branch spatiotemporal hybrid network model consists of the following modules or branches: a learnable temporal difference module (TDM) for displaying dynamic change information between modeling frames; a temporal branch for extracting global dependencies of long sequences, which is constructed based on a visual state space model; a spatial branch for extracting local spatiotemporal textures, which is constructed based on a 3D convolutional neural network; and a temporal channel attention module (TCA) and a temporal channel modulation module (TCM) for connecting the two branches and performing feature interaction. For the overall architecture of the model, please refer to [link to relevant documentation]. Figure 2 .

[0062] The steps for model construction in step (2) are as follows:

[0063] Step (2-1): Constructing the temporal branch input: Using a convolutional neural network, shallow features of the input sequence are extracted and the differences between features at adjacent time steps are calculated. This process includes:

[0064]

[0065] in, This represents the inter-frame difference sequence of a video frame sequence. This represents a convolutional neural network used for modeling temporal spaces. This represents a sequence of video frames that have passed through the Stem layer.

[0066] After obtaining the inter-frame difference sequence, zero-padding is performed at the end of the sequence's time dimension, and temporal channel attention is used to enhance the difference features. This process can be represented as:

[0067]

[0068] in, This represents the learnable parameters used to dynamically scale the difference features. TCA stands for Temporal Channel Attention Module.

[0069] Step (2-2): Arrange the inter-frame difference sequences obtained in step (2-1) in a spatial grid according to temporal order, folding the temporal dimension into the spatial dimension to reconstruct a high-resolution two-dimensional feature map. The length and width of this feature map are both [missing information]. The feature map is then embedded and input into the visual state space model for bidirectional sequence modeling. This process can be represented as:

[0070]

[0071] Step (2-3): Constructing Spatial Branches: In the bottleneck structure of the 3D convolutional layer, an asymmetric convolutional kernel with a temporal dimension kernel size of 2 is used. This is used to align inter-frame semantic offsets caused by differential operations and to capture local motion changes between adjacent frames.

[0072] Step (2-4): Constructing the temporal channel attention module: First, global average pooling of the spatial dimension is performed on the input feature map to aggregate the channel statistical features that retain the temporal dimension. This process can be represented as:

[0073]

[0074] in, This represents the input feature map of the representation module. This represents the average pooling operation on spatial dimensions H and W. This represents the intermediate features after spatial pooling.

[0075] Subsequently, a multilayer perceptron is used to perform a nonlinear transformation on the statistical feature, and a channel attention weight is generated using a sigmoid activation function. This process can be represented as follows:

[0076]

[0077] in, and Let represent the one-dimensional convolution weight parameters of the first and second layers in a multilayer perceptron, respectively. Used for dimensionality reduction. Used for dimensional recovery of channel count. Represents the ReLU activation function. This represents the Sigmoid activation function, used to map attention weights to the (0, 1) interval. This represents the generated temporal channel attention weight map.

[0078] Obtain attention weights The sequence then needs to be applied to the original sequence and residual joined. This process includes:

[0079]

[0080] in This represents element-wise multiplication, using a broadcast mechanism to apply weights to each spatial pixel. This represents the output features after attention enhancement.

[0081] In the branching loop, it is also necessary to... Pooling and projection are performed to transform the spatial dimension into the temporal dimension. This process includes:

[0082]

[0083] Step (2-5): Constructing the time channel modulation module: First, obtain the sequence features of the time branch output, and map them to gate weights with values ​​between 0 and 1 through a linear projection layer. This process can be represented as:

[0084]

[0085] in, This represents the sequence features of the time branch output. This represents the projection matrix, used to increase dimensionality. This indicates that features are broadcast to the spatial dimension. This represents the Sigmoid activation function.

[0086] The gate weights are then aligned dimensionally with the feature maps of the spatial branch. Finally, a learnable scaling factor initialized to zero is introduced. The aligned gate weights are multiplied by the spatial features, scaled, and then added back to the original spatial features via a residual connection. This process injects the global temporal context into the spatial branch, which can be represented as:

[0087]

[0088] in, This indicates element-wise multiplication. To introduce a learnable scaling factor, : Represents the output characteristics after modulation by global timing information.

[0089] As a preferred embodiment of the present invention, the main process described in step (3) of inputting the processed video frame sequence in step (1) into the model constructed in step (2) for training and obtaining the trained network model includes:

[0090] Step (3-1): Input the preprocessed video frame sequence into the network. First, it is passed through a shallow convolutional network for preliminary dimensionality reduction. Then, the input is sent to the spatial branch and temporal branch described in step (2) as the initial input.

[0091] Step (3-2): Then, in the spatial branch, the initial input passes through stacked 3D convolutional bottleneck modules layer by layer; at each stage, the spatial features first receive the gating signal from the temporal branch through the temporal channel modulation module described in step (2-5) for feature calibration, integrate global temporal anomaly information into the spatial features, and then pass through the convolutional layer to extract local texture information and perform downsampling to enter the next stage.

[0092] Step (3-3): In the temporal branch, the initial input first passes through the learnable temporal difference module described in step (2-1) to extract inter-frame motion differences; then it is transformed into a two-dimensional sequence through the grid stitching strategy described in step (2-2); subsequently, before each stage, this sequence is fused with spatial information through the temporal channel attention module described in step (2-4). After passing through the embedding layer, the sequence enters the visual state space model block for bidirectional state space scanning and global modeling. This process can be represented as:

[0093]

[0094] in This represents a learnable location embedding that includes spatial and temporal location information. Represents the projection matrix. This represents the flattened vector of the j-th image patch segmented from the temporal grid feature map. This indicates the [CLS] token. This indicates positional embedding.

[0095] Generated global time-series token In addition to being used for its own deep feature extraction, it also serves as a global guiding signal, inputting into the time channel modulation module described in step (2-5), and performing adaptive modulation of the spatial branch features at the same level in the channel dimension by generating gating weights.

[0096] Step (3-4): After multiple iterations of the above steps, the final feature vector is input into the fully connected classification layer to obtain the predicted probability value and output the prediction result.

Claims

1. A method for detecting spatiotemporal inconsistencies in face forgery based on a state-space model, characterized in that, This method mainly includes the following steps: S1: Collect a video dataset containing real and fake faces, and perform face detection on the video to obtain face location information and a fixed-length sequence of face video frames; S2: Construct a dual-branch spatiotemporal hybrid network model suitable for face forgery detection; S3: Input the video frame sequence processed in S1 into the model constructed in S2 for training. Calculate the loss function and update the network parameters through backpropagation to obtain the trained network model. S4: Obtain the target video to be detected, preprocess it according to the rules described in S1, input it into the model trained in S3, and output the probability score of the video being a fake face.

2. The method for detecting spatiotemporal inconsistencies in face forgery based on a state-space model according to claim 1, characterized in that, In S1, T consecutive video frames are randomly extracted from each input video, where T is set to a perfect square. Then, the face portion is extracted from each video frame as input to the model.

3. The method for detecting spatiotemporal inconsistencies in face forgery based on a state-space model according to claim 1, characterized in that, In S2, the dual-branch spatiotemporal hybrid network model consists of the following modules or branches: a learnable temporal difference module (TDM) for displaying dynamic change information between modeling frames; a temporal branch for extracting global dependencies of long sequences, which is constructed based on a visual state space model; a spatial branch for extracting local spatiotemporal textures, which is constructed based on a 3D convolutional neural network; and a temporal channel attention module (TCA) and a temporal channel modulation module (TCM) for connecting the two branches and performing feature interaction.

4. The method for detecting spatiotemporal inconsistencies in face forgery based on a state-space model according to claim 3, characterized in that, The workflow of the learnable temporal difference module includes: extracting shallow features of the input sequence using a convolutional neural network and calculating the difference between features at adjacent time steps. This process includes: in, This represents the inter-frame difference sequence of a video frame sequence. This represents a convolutional neural network used for modeling temporal spaces. This represents a sequence of video frames that have passed through the Stem layer. After obtaining the inter-frame difference sequence, zero-padding is performed at the end of the sequence's time dimension, and temporal channel attention is used to enhance the difference features. This process can be represented as: in, This represents the learnable parameters used to dynamically scale the difference features. TCA stands for Temporal Channel Attention Module.

5. The method for detecting spatiotemporal inconsistencies in face forgery based on a state-space model according to claim 3, characterized in that, The time branch employs a temporal grid stitching strategy: the inter-frame difference sequence obtained in claim 1 is arranged in a grid according to temporal order in the spatial dimension, folding the temporal dimension into the spatial dimension to reconstruct a high-resolution two-dimensional feature map, the length and width of which are both [missing information]. The feature map is then embedded and input into the visual state space model for bidirectional sequence modeling. This process can be represented as:

6. The method for detecting spatiotemporal inconsistencies in face forgery based on a state-space model according to claim 3, characterized in that, In the network structure of the spatial branch: In the bottleneck structure of the 3D convolutional layer, an asymmetric convolutional kernel with a temporal dimension kernel size of 2 is used. This is used to align inter-frame semantic offsets caused by differential operations and to capture local motion changes between adjacent frames.

7. The method for detecting spatiotemporal inconsistencies in face forgery based on a state-space model according to claim 3, characterized in that, The working process of the temporal channel attention module includes: firstly, performing global average pooling on the input feature map in the spatial dimension to aggregate and obtain channel statistical features that retain the temporal dimension. This process can be represented as: in, This represents the input feature map of the representation module. This represents the average pooling operation on spatial dimensions H and W. This represents the intermediate features after spatial pooling. Subsequently, a multilayer perceptron is used to perform a nonlinear transformation on the statistical feature, and a channel attention weight is generated using a sigmoid activation function. This process can be represented as follows: in, and Let represent the one-dimensional convolution weight parameters of the first and second layers in a multilayer perceptron, respectively. Used for dimensionality reduction. Used for dimensional recovery of channel count. Represents the ReLU activation function. This represents the Sigmoid activation function, used to map attention weights to the (0, 1) interval. This represents the generated temporal channel attention weight map. Obtain attention weights The sequence then needs to be applied to the original sequence and residual joined. This process includes: in This represents element-wise multiplication, using a broadcast mechanism to apply weights to each spatial pixel. This represents the output features after attention enhancement. In the branching loop, it is also necessary to... Pooling and projection are performed to transform the spatial dimension into the temporal dimension. This process includes: According to claim 3, the spatiotemporal inconsistency face forgery detection method based on state-space model is characterized in that the working process of the time channel modulation module includes: firstly, acquiring the sequence features of the time branch output, and mapping them to gate weights with values ​​between 0 and 1 through a linear projection layer. This process can be expressed as: in, This represents the sequence features of the time branch output. This represents the projection matrix, used to increase dimensionality. This indicates that features are broadcast to the spatial dimension. This represents the Sigmoid activation function. The gate weights are then aligned dimensionally with the feature maps of the spatial branch. Finally, a learnable scaling factor initialized to zero is introduced. The aligned gate weights are multiplied by the spatial features, scaled, and then added back to the original spatial features via a residual connection. This process injects the global temporal context into the spatial branch, which can be represented as: in, This indicates element-wise multiplication. To introduce a learnable scaling factor, : Represents the output characteristics after modulation by global timing information.

8. The method for detecting spatiotemporal inconsistencies in face forgery based on a state-space model according to claim 1, characterized in that, In S3, the main process of inputting the processed video frame sequence from S1 into the model constructed in S2 for training and obtaining the trained network model includes: S3-1, The preprocessed video frame sequence is input into the network. First, it is passed through a shallow convolutional network for preliminary dimensionality reduction. Then, the input is sent to the spatial branch and temporal branch as described in claim 3 as initial inputs. S3-2, In the spatial branch, the initial input passes through stacked 3D convolutional bottleneck modules layer by layer; at each stage, the spatial features first receive the gating signal from the temporal branch through the temporal channel modulation module as described in claim 8 for feature calibration, integrate global temporal anomaly information into the spatial features, and then pass through the convolutional layer to extract local texture information and perform downsampling to enter the next stage. S3-3, in the temporal branch, the initial input first passes through the learnable temporal difference module described in claim 4 to extract inter-frame motion differences; then it is transformed into a two-dimensional sequence through the grid stitching strategy described in claim 5; subsequently, before each stage, this sequence is first fused with spatial information through the temporal channel attention module described in claim 7. After passing through the embedding layer, this sequence enters the visual state space model block for bidirectional state space scanning and global modeling. This process can be represented as: in This represents a learnable location embedding that includes spatial and temporal location information. Represents the projection matrix. This represents the flattened vector of the j-th image patch segmented from the temporal grid feature map. This indicates the [CLS] token. This indicates positional embedding. Generated global time-series token In addition to being used for its own deep feature extraction, it is also used as a global guiding signal, input to the time channel modulation module of claim 7, and performs adaptive modulation of the spatial branch features of the same level in the channel dimension by generating gating weights. S3-4, after multiple iterations of S3-2 and S3-3, inputs the final feature vector into the fully connected classification layer to obtain the predicted probability value and output the prediction result.

Images