Space-frequency domain pedestrian gait feature extraction method based on spiking neural network driving

Abstract

The invention relates to the technical field of computer image processing, and particularly provides a space-frequency domain pedestrian gait feature extraction method based on spiking neural network driving, and the method comprises the steps: carrying out the gait feature extraction of a gait image through employing a space-frequency domain pedestrian gait feature extraction model; the space-frequency domain pedestrian gait feature extraction model comprises a backbone network, a multi-stage time domain global feature interaction enhancement module and a distinguishing feature aggregation module; the backbone network comprises N stages which are connected in sequence, and each stage comprises a space-frequency feature fusion module for fusing space-domain features and frequency-domain features; a multi-stage time domain global feature interaction enhancement module extracts and enhances spatial domain global features and time domain global features; and the distinguishing feature aggregation module aggregates the output of the backbone network and the output of the multi-stage time domain global feature interaction enhancement module. According to the method, the capturing capability of the model on gait micro-detail features can be improved, and the adaptability to complex scenes is enhanced.

Description

Technical Field

[0001] This invention relates to the field of computer image processing technology, and more specifically, to a method for extracting spatial frequency domain pedestrian gait features based on a spiking neural network. Background Technology

[0002] Gait recognition technology uses an individual's unique walking pattern to identify them, representing an important direction in the field of biometrics. This technology has shown great potential in several key areas, including intelligent security, crime prevention, forensic identification, and dynamic crowd tracking.

[0003] The practical application of this technology still faces several significant obstacles. For example, occlusion of the target object, sudden changes in ambient lighting conditions, difficulties in recognition due to differences in perspective, and high complexity of the background environment can all affect recognition accuracy. In addition, factors such as an individual's health status, emotional fluctuations, and changes in walking speed can also lead to natural fluctuations in gait characteristics, requiring the recognition algorithm to have higher environmental adaptability and robustness.

[0004] In 2016, Shiraga et al. proposed GEINet, a model that uses a six-layer convolutional network to extract features from the gait energy map (GEI). Since then, various deep learning models have been applied to gait recognition. Nair et al. developed the DBNGait model based on a deep Bayesian network. The GaitSet model designed an efficient and lightweight network architecture, achieving a 70.3% cross-view recognition accuracy for the first time on the CASIA-B dataset under clothing-changing conditions. GaitBase, employing a residual network architecture, deepened the network, significantly improving recognition performance on field datasets; the subsequent DeepGaitV2-2D further increased the network depth and verified the positive correlation between recognition accuracy and model depth on field datasets. The BigGait model introduced a self-supervised learning module, extracting general features through a general feature extraction module (LVM) and then using a gait recognition expert module (GRE) to transform these general features into features specifically for gait recognition.

[0005] The rapid development of spiking neural networks (SNNs) has led to their expansion into multiple fields. In gait recognition, the GaitSpike model pioneered the combination of traditional SNNs with gait recognition algorithms. However, research on SNN-based gait recognition methods is relatively limited, necessitating the design of new frameworks to further advance the application of such networks in gait recognition.

[0006] In visual tasks such as object detection, pose estimation, and machine temporal signal processing, frequency-domain-based networks (such as Fourier transform or wavelet transform) have been applied. These methods transform data into the frequency domain for filtering and feature extraction. For example, in pose estimation, the Discrete Fourier Transform (DTC) is used to extract frequency-domain features from the entire time series as supplementary information for joint spatial position and motion details. These frequency-domain features are combined with the joint spatial domain analysis results of a single frame and then input into a Transformer for cross-frame temporal dependency modeling. Regarding attention mechanisms, the GFN model proposes learnable frequency filters for efficient mixing of visual tags, and AFNO further improves the performance of the frequency-domain tag mixer through operator learning. However, the aforementioned frequency-domain-based methods have not yet been fully explored and applied in the field of gait recognition.

[0007] The core shortcomings of existing gait recognition technology: (i) Insufficient mining of spatial frequency domain feature dimension information: Existing methods mostly rely on spatial contour features and do not mine frequency domain dynamic information (such as step frequency harmonic components and joint motion spectrum), resulting in the loss of motion details and difficulty in distinguishing key identity markers such as slight changes in stride and differences in step frequency. (ii) Lack of high and low frequency feature extraction and fusion mechanism: The lack of a dedicated module to separate low and high frequency details (joint micro-movements) results in insufficient adaptability of the algorithm to occlusion and viewpoint changes; (III) The potential of spiking neural networks in gait recognition has not been fully realized: the pulse temporal coding characteristics of SNNs are naturally adapted to gait dynamic modeling, but existing explorations (such as GaitSpike) have not combined spatial domain characteristics in their architecture design, and the spatiotemporal pulse coding capabilities have not been fully utilized. (iv) Inefficient interaction of global features in the spatiotemporal domain: The lack of a cross-dimensional global spatial domain and global frequency domain feature fusion mechanism results in weak spatiotemporal feature correlation; (v) Multi-stage feature fusion method is not robust enough: The lack of hierarchical fusion of multi-scale temporal features (such as gait cycle and single-step micro-movements) leads to a sharp drop in recognition rate in complex scenarios. Summary of the Invention

[0008] To overcome the shortcomings and deficiencies of the existing technology, the purpose of this invention is to provide a spatial-frequency domain pedestrian gait feature extraction method based on a spiking neural network. This method integrates spatial and frequency domain features, models high and low frequencies separately, leverages the advantages of spiking neural networks, and performs global spatial-temporal feature interaction to improve the model's ability to capture gait micro-detail features and enhance its adaptability to complex scenes.

[0009] To achieve the above objectives, the present invention is implemented through the following technical solution: a spatial frequency domain pedestrian gait feature extraction method based on a spiking neural network, which uses a spatial frequency domain pedestrian gait feature extraction model to extract gait features from gait images; The spatial-frequency domain pedestrian gait feature extraction model includes: a backbone network, a multi-stage temporal-domain global feature interaction enhancement module, and a discriminative feature aggregation module. The backbone network comprises N sequentially connected stages, each stage including a spatial-frequency feature fusion module for fusing spatial and frequency domain features. The multi-stage temporal-domain global feature interaction enhancement module extracts and enhances spatial and temporal global features from the output feature maps Si, i=1,2,…,N of each stage of the backbone network, to obtain the output feature map S. T_total The discriminative feature aggregation module combines the backbone network output with the feature map S output by the multi-stage temporal global feature interaction enhancement module. T_total Aggregation is performed to obtain the gait features finally output by the spatial frequency domain pedestrian gait feature extraction model.

[0010] Preferably, each space-frequency feature fusion module includes a spatial domain extraction branch and a frequency domain extraction branch; each space-frequency feature fusion module divides the feature map input to the space-frequency feature fusion module into two feature maps I1 and I2; the spatial domain extraction branch extracts spatial features from feature map I1 through two convolutional kernels; the frequency domain extraction branch performs a Fourier transform on feature map I2 and then separates the high-frequency and low-frequency components after the Fourier transform through two convolutional kernels, and then inputs them into the corresponding convolutional networks after inverse transform to obtain the low-frequency feature U in the spatial domain. 0L and high-frequency features U 0H High-frequency feature U 0H The high-frequency component weights W and low-frequency features U are obtained through a pulsating neural network LIF. 0L After adding the spatial features extracted by the spatial extraction branch, the dual-branch low-frequency fusion feature U1 is obtained; the dual-branch low-frequency fusion feature U1 is multiplied by the low-frequency component weight (1-W) to obtain the fused low-frequency feature U. 1L The low-frequency fusion feature U1 of the dual-branch system is multiplied by the high-frequency component weight W and then combined with the high-frequency feature U. 0H Add them together to obtain the fused high-frequency feature U. 1H Then, the fused low-frequency features U 1L and high-frequency features U 1H Further addition and fusion are performed to obtain the hybrid output feature map U2; The feature map I is processed through a channel attention mechanism to obtain channel weights, which are then multiplied with the weights obtained from the spiking neural network LIF to obtain the updated channel attention weights W. C The channel attention weight W C Multiplying the mixed output feature map U2 with the attention-based feature map U2 yields the attention-based feature map U.C After normalization, the updated features are input into the FFN feedforward neural network module. Finally, the features are added to the feature map I through the residual linking module to obtain the final output feature map Si of the space-frequency feature fusion module.

[0011] Preferably, the feature maps I1, I2 are: I1, I2 = split(I); where split is a segmentation function; The dual-branch low-frequency fusion feature U1 is: U1=Conv2D ×2 (I1)+ Conv2D(IFFT([Conv2D ×2 (FFT(I2))] LF )) ; Where Conv2D() is a 2D convolution; Conv2D ×2 ( ) represents a 2D convolution with two kernels; FFT() represents the Fourier transform; IFFT() represents the inverse Fourier transform; [ ] LF Represents low-frequency components; The weight W of the high-frequency components is: W=LIF(Conv2D(IFFT([Conv2D ×2 (FFT(I2))] HF ))) ; Wherein, LIF() represents a spiking neural network; [ ] HF Represents high-frequency components; The fused low-frequency feature U 1L and high-frequency features U 1H They are respectively: U 1L =U1×(1-W); U 1H =U1×W+Conv2D(IFFT([Conv2D ×2 (FFT(I2))] HF )) ; The mixed output feature map U2 is: U2=Conv2D(BN(Conv2D(U 1L ))+BN(Conv2D(U 1H ))) ; Where BN() is the batch normalization operation; Channel attention weight W C for: W C = Tanh(attention(I) ×LIF(attention(I))); Where attention() is the attention mechanism; Tanh() is the activation function; Feature map U based on attention mechanism C for: U C =U2×W C ; The final output feature map Si of the space-frequency feature fusion module is: Si = FFN(Norm(U C ))+I; Here, FFN() is the feedforward neural network module; Norm() is the normalization operation.

[0012] Preferably, the multi-stage temporal global feature interaction enhancement module includes a temporal global feature fusion module, a spatial global feature enhancement network branch, and a temporal global feature enhancement network branch; The output feature maps Si of each stage of the backbone network are used to calculate the spatial global features S through spatial pooling and temporal pooling, respectively. si and temporal global features S Ti Spatial global features S si and temporal global features S Ti The common inputs are fed into the temporal global feature fusion module to obtain the spatiotemporal global feature S. STi ; The spatial global features S of each stage si With the global features in the space-time domain S STi Merging into merged features (S) along the time dimension si , S STi ); Merging characteristics at each stage (S) si , S STi The inputs are fed into the spatial global feature enhancement network branch; the spatial global feature enhancement network branch consists of N stages of spatial enhancement convolutional groups; the merged features (S) of the current stage are input into the spatial global feature enhancement network branch. si , S STi The output feature map (S) of the previous stage spatial enhancement convolution group s i-1 ,S ST i-1 The sums are then fed into the current stage's spatial enhancement convolutional group for feature extraction to obtain the output feature map (S) of the current stage's spatial enhancement convolutional group. s i ,S ST i ); Temporal global feature map S at each stage TiThe inputs are fed into the temporal global feature enhancement network branch; the temporal global feature enhancement network branch consists of N stages of temporal enhancement convolutional groups; the temporal global feature map S of the current stage... Ti Compared with the output feature map S of the previous stage temporal enhancement convolution group T i-1 The sums are then fed into the current stage's temporal augmentation convolutional group for feature extraction, yielding the output feature map S of the current stage's temporal augmentation convolutional group. T i ; The output feature map (S) of the final stage spatial enhancement convolution group s N ,S ST N Segmented into features S s N ,S ST N ; with feature S s N As a global feature in the spatial domain S s(N+1) The output feature map S of the final stage temporal enhancement convolution group T N As a time-domain global feature S T(N+1) The common input temporal global feature fusion module obtains the output spatiotemporal global feature S. ST(N+1) , with feature S ST N The feature map S is obtained by fusion. ST_temp Feature map S ST_temp With feature map S T N The weights obtained through the activation function are weighted and fused to obtain the final output feature map S of the multi-stage temporal global feature interaction enhancement module. T_total .

[0013] Preferably, in the multi-stage temporal global feature interaction enhancement module, the calculation method is as follows: Initialize S s 0 =0;S ST 0 =0;S T 0 =0; Spatial global features S si and temporal global features S Ti They are respectively: ; ; Where T is the number of time frames, and C, H, and W are the number of channels, height, and width of feature map Si, respectively; Output feature maps of spatial enhancement convolutional groups at each stage (S s i ,S ST i )for: (S s i ,S ST i =Conv2D ×2 (concat(S s i-1 ,S ST i-1 )+concat(S si ,S STi ) ; Output feature maps S of temporal augmentation convolutional groups at each stage T i for: S T i =Conv2D ×2 (S T i-1 +S Ti ); Feature map S ST_temp for: S ST_temp =Conv2D ×2 (f st (S s N ,S T N )+S ST N ); The final output feature map S of the multi-stage temporal global feature interaction enhancement module T_total for: S T_total =sigmoid(S ST_temp ) ×sigmoid(S T N ) ×S ST_temp +S T N .

[0014] Preferably, the spatial global feature S si and temporal global features S Ti The common inputs are into the temporal global feature fusion module: spatial global feature S si After feature extraction using two layers of 3D convolution, a temporal attention score is obtained through a temporal attention mechanism module consisting of two layers of 3D convolution and activated in the temporal dimension. This temporal attention score is then compared with the spatial global feature S. siThe fused features S are obtained by performing dot product weighting. si_temp Temporal global features S Ti After max pooling and average pooling, it is combined with the feature S si_temp The features are added together, and the spatiotemporal global features S output by the temporal global feature fusion module are obtained through two layers of 3D convolution. STi .

[0015] Preferably, the spatiotemporal global feature S output by the temporal global feature fusion module is... STi for: S STi =f st (S si ,S Ti ) =Conv3D ×2 (Softmax(Conv3D ×2 (R(S si ))) ×Conv3D ×2 (R(S si ))+maxpool(S Ti )+meanpool(S Ti )); Among them, f st ( ) represents the temporal global feature fusion module; R() is the reshape operation; Conv3D ×2 ( ) represents two layers of 3D convolution; Softmax() is the activation function; maxpool() and meanpool() are the spatial max pooling and average pooling operations, respectively.

[0016] Preferably, the output feature map SN of the final stage of the backbone network is subjected to temporal pooling to obtain feature map S. m In the feature aggregation module, the feature map S is... m The feature map S is output by the multi-stage temporal global feature interaction enhancement module. T_total After aggregation and feature fusion via convolution, the results are input into the HPM operation to obtain the output S of the discriminative feature aggregation module. out : S out =HPM(Conv2D ×2 (Cancat(S T_total ,S m ))); HPM() is the block operation; Conv2D ×2 ( ) represents a 2D convolution with two kernels; Cancat() represents an aggregation operation.

[0017] A readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the described method for extracting spatial frequency domain pedestrian gait features based on a spiking neural network.

[0018] A computer device includes a processor and a memory for storing a processor-executable program, wherein when the processor executes the program stored in the memory, it implements the aforementioned method for extracting spatial frequency domain pedestrian gait features based on a spiking neural network.

[0019] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. The method of this invention significantly improves the model's performance in gait feature extraction, thereby enhancing the accuracy and robustness of gait recognition: First, through the spatial-frequency feature fusion module, this invention can efficiently separate and fuse high- and low-frequency features, strengthening the main structural features and edge texture details, thus improving the richness and accuracy of feature representation; Second, the design of the multi-stage temporal global feature interaction enhancement module enables the model to progressively extract and enhance global features in the spatial and temporal domains, further improving the model's capabilities; In addition, the discriminative feature aggregation module further enhances the discriminativeness of features by fusing the output feature map of the backbone network with the feature map enhanced by the multi-stage temporal global feature interaction enhancement module and performing HPM operation, enabling the model to maintain excellent recognition performance even in complex scenarios; 2. The spatial-frequency feature fusion module of the present invention effectively fuses the spatial and frequency domain features in gait images, improving the accuracy and robustness of gait recognition; the spatial-frequency feature fusion module utilizes a spiking neural network screening mechanism to achieve the separation and fusion of high and low frequency features, enhances edge texture details, and improves detail sensitivity, thereby enhancing the expressive power of gait features. 3. The multi-stage temporal global feature interaction enhancement module of the present invention realizes the interaction enhancement of spatial global features and temporal global features through iterative progressive calculation, which further improves the gait feature extraction capability; the multi-stage temporal global feature interaction enhancement module uses the interaction attention mechanism module to perform temporal weighted fusion of feature maps, resulting in more robust gait features. 4. The temporal global feature fusion module of the present invention accurately captures the temporal dynamic information in the gait sequence through the temporal attention mechanism. By fusing global spatial features and global temporal features, it enhances the model's sensitivity to gait changes, enabling gait recognition to maintain high performance under different speeds and rhythms. 5. The discriminative feature aggregation module of the present invention obtains richer and more accurate gait features by fusing the output feature map of the backbone network with the feature map enhanced by the multi-stage temporal global feature interaction enhancement module; the discriminative feature aggregation module uses fixed block operations and 2D convolution groups to perform feature fusion, thereby improving the accuracy and efficiency of gait recognition. Attached Figure Description

[0020] Figure 1 This is a structural block diagram of the spatial frequency domain pedestrian gait feature extraction model of the present invention; Figure 2 This is a structural block diagram of the space-frequency feature fusion module of the present invention; Figure 3 This is a structural block diagram of the multi-stage temporal global feature interaction enhancement module of the present invention; Figure 4 This is a structural block diagram of the temporal global feature fusion module of the present invention; Figure 5 This is a structural block diagram of the feature aggregation module of the present invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0022] Example 1 This embodiment presents a spatial frequency domain pedestrian gait feature extraction method based on a spiking neural network, which uses a spatial frequency domain pedestrian gait feature extraction model to extract features from gait images.

[0023] like Figure 1 As shown, the spatial frequency domain pedestrian gait feature extraction model includes: a backbone network, a multi-stage temporal global feature interaction enhancement module, and a discriminative feature aggregation module.

[0024] The backbone network comprises N sequentially connected stages; before the first stage, a stem module is included; after the final stage, a temporal pooling module (TP) is included. Each stage includes a space-frequency feature fusion module for fusing spatial and frequency domain features. In this embodiment, the backbone network comprises four stages; each stage consists of a 2D residual module and a space-frequency feature fusion module connected in series; the number of feature map channels output by each stage is C, 2C, 4C, and 8C, respectively, and the feature map sizes are (H, W), (1 / 2H, 1 / 2W), (1 / 4H, 1 / 4W), and (1 / 4H, 1 / 4W), respectively. In practical applications, the number of stages in the backbone network can also be two, three, five, six, or even more.

[0025] The spatial-frequency feature fusion module consists of a spatial-frequency dual-domain decoupled processing branch, a high- and low-frequency feature extraction and dynamic fusion mechanism, a cross-domain channel attention interaction based on a spiking neural network, and a feedforward neural network. The module primarily separates and fuses high- and low-frequency features using weights obtained from the dual-branch structure and spiking neural network filtering mechanism. Low-frequency components retain the main structural features, while high-frequency components enhance edge texture details. The channel attention mechanism further enhances detail sensitivity.

[0026] like Figure 2 As shown, each space-frequency feature fusion module includes a spatial domain extraction branch and a frequency domain extraction branch; each space-frequency feature fusion module divides the feature map input to the space-frequency feature fusion module into two feature maps I1, I2: I1, I2 = split(I); where split is the splitting function.

[0027] The spatial domain extraction branch extracts spatial features from feature map I1 using two convolutional kernels; the frequency domain extraction branch extracts high- and low-frequency components from feature map I2 after Fourier transform using two convolutional kernels, and then inputs them into the corresponding convolutional networks after inverse transform to obtain the low-frequency features U in the spatial domain. 0L and high-frequency features U 0H High-frequency feature U 0H The high-frequency component weights W are obtained through the spiking neural network LIF: W=LIF(Conv2D(IFFT([Conv2D ×2 (FFT(I2))] HF ))) ; Wherein, LIF() represents a spiking neural network; [ ] HF Represents high-frequency components; Low-frequency characteristics U 0L After adding the spatial features extracted by the spatial extraction branch, the dual-branch low-frequency fusion feature U1 is obtained: U1=Conv2D ×2 (I1)+ Conv2D(IFFT([Conv2D ×2 (FFT(I2))] LF )) ; Where Conv2D() is a 2D convolution; Conv2D ×2 ( ) represents a 2D convolution with two kernels; FFT() represents the Fourier transform; IFFT() represents the inverse Fourier transform; [ ] LF This represents the low-frequency component.

[0028] The fused low-frequency feature U1 is obtained by multiplying the low-frequency component weight (1-W) by the dual-branch low-frequency fusion feature U1. 1LThe low-frequency fusion feature U1 of the dual-branch system is multiplied by the high-frequency component weight W and then combined with the high-frequency feature U. 0H Add them together to obtain the fused high-frequency feature U. 1H The fused low-frequency feature U 1L and high-frequency features U 1H They are respectively: U 1L =U1×(1-W); U 1H =U1×W+Conv2D(IFFT([Conv2D ×2 (FFT(I2))] HF )) ; Then the fused low-frequency features U 1L and high-frequency features U 1H Further addition and fusion are performed to obtain the mixed output feature map U2: U2=Conv2D(BN(Conv2D(U 1L ))+BN(Conv2D(U 1H ))) ; BN() is the batch normalization operation.

[0029] The feature map I is processed through a channel attention mechanism to obtain channel weights, which are then multiplied with the weights obtained from the spiking neural network LIF to obtain the updated channel attention weights W. C : W C = Tanh(attention(I) ×LIF(attention(I))); Where attention() is the attention mechanism; Tanh() is the activation function; The channel attention weight W C Multiplying the mixed output feature map U2 with the attention-based feature map U2 yields the attention-based feature map U. C : U C =U2×W C ; After normalization, the updated features are input into the FFN feedforward neural network module. Finally, these features are added to feature map I through the residual linking module, allowing feature map I to provide cross-layer feature supplementation and avoid information loss, resulting in the final output feature map Si of the space-frequency feature fusion module. Si = FFN(Norm(U C ))+I; Here, FFN() is the feedforward neural network module; Norm() is the normalization operation.

[0030] The multi-stage temporal global feature interaction enhancement module extracts and enhances spatial and temporal global features from the output feature maps Si, i=1,2,…,N of each stage of the backbone network, to obtain the output feature map S. T_total .

[0031] like Figure 3 As shown, the multi-stage temporal global feature interaction enhancement module includes a temporal global feature fusion module, a spatial global feature enhancement network branch, and a temporal global feature enhancement network branch.

[0032] First, initialize S s 0 =0;S ST 0 =0;S T 0 =0; The output feature maps Si of each stage of the backbone network are used to calculate the spatial global feature S through spatial pooling and temporal pooling, respectively. si and temporal global features S Ti : ; ; Spatial global features S si and temporal global features S Ti The spatiotemporal global features S are obtained together through the temporal global feature fusion module. STi .

[0033] Spatiotemporal global features S STi The feature dimension is consistent with the feature dimension of the corresponding stage output feature map Si; the spatial global feature S of each stage is... si With the global features in the space-time domain S STi Merging into merged features (S) along the time dimension si , S STi ); Merging characteristics at each stage (S) si , S STi The inputs are fed into the spatial global feature enhancement network branch; the spatial global feature enhancement network branch consists of N stages of spatial enhancement convolutional groups; the merged features (S) of the current stage are input into the spatial global feature enhancement network branch. si , S STi The output feature map (S) of the previous stage spatial enhancement convolution group s i-1 ,S ST i-1 The sums are then fed into the current stage's spatial enhancement convolutional group for feature extraction to obtain the output feature map (S) of the current stage's spatial enhancement convolutional group. s i ,S STi ): (S s i ,S ST i =Conv2D ×2 (concat(S s i-1 ,S ST i-1 )+concat(S si ,S STi ) ; Temporal global feature map S at each stage Ti The inputs are fed into the temporal global feature enhancement network branch; the temporal global feature enhancement network branch consists of N stages of temporal enhancement convolutional groups; the temporal global feature map S of the current stage... Ti Compared with the output feature map S of the previous stage temporal enhancement convolution group T i-1 The sums are then fed into the current stage's temporal augmentation convolutional group for feature extraction, yielding the output feature map S of the current stage's temporal augmentation convolutional group. T i : S T i =Conv2D ×2 (S T i-1 +S Ti ); The output feature map (S) of the final stage spatial enhancement convolution group s N ,S ST N Segmented into features S s N ,S ST N ; with feature S s N As a global feature in the spatial domain S s(N+1) The output feature map S of the final stage temporal enhancement convolution group T N As a time-domain global feature S T(N+1) The common input temporal global feature fusion module obtains the output spatiotemporal global feature S. ST(N+1) , with feature S ST N The feature map S is obtained by fusion. ST_temp : S ST_temp =Conv2D ×2 (f st (S s N ,ST N )+S ST N ); Feature map S ST_temp With feature map S T N The weights obtained through the activation function are weighted and fused to obtain the final output feature map S of the multi-stage temporal global feature interaction enhancement module. T_total : S T_total =sigmoid(S ST_temp ) ×sigmoid(S T N ) ×S ST_temp +S T N .

[0034] like Figure 4 As shown, the spatial global feature S si and temporal global features S Ti The common inputs are into the temporal global feature fusion module: spatial global feature S si After feature extraction using two layers of 3D convolution, a temporal attention score is obtained through a temporal attention mechanism module consisting of two layers of 3D convolution and activated in the temporal dimension. This temporal attention score is then compared with the spatial global feature S. si The fused features S are obtained by performing dot product weighting. si_temp Temporal global features S Ti After max pooling and average pooling, it is combined with the feature S si_temp The features are added together, and the spatiotemporal global features S output by the temporal global feature fusion module are obtained through two layers of 3D convolution. STi : S STi =f st (S si ,S Ti ) =Conv3D ×2 (Softmax(Conv3D ×2 (R(S si ))) ×Conv3D ×2 (R(S si ))+maxpool(S Ti )+meanpool(S Ti )); Among them, f st ( ) represents the temporal global feature fusion module; R() is the reshape operation; Conv3D ×2( ) represents two layers of 3D convolution; Softmax() is the activation function; maxpool() and meanpool() are the spatial max pooling and average pooling operations, respectively; here i=1,2,…,N,N+1.

[0035] The output feature map SN from the final stage of the backbone network is subjected to temporal pooling (TP) to obtain the feature map S. m .

[0036] like Figure 5 As shown, the feature aggregation module will output feature map S from the backbone network. m The feature map S is output by the multi-stage temporal global feature interaction enhancement module. T_total After aggregation and feature fusion via convolution, the results are input into the HPM operation to obtain the output S of the discriminative feature aggregation module. out The gait features are the final output of the spatial frequency domain pedestrian gait feature extraction model. S out =HPM(Conv2D ×2 (Cancat(S T_total ,S m ))); HPM() is the block operation; Conv2D ×2 ( ) represents a 2D convolution with two kernels; Cancat() represents an aggregation operation.

[0037] The training of the spatial frequency domain pedestrian gait feature extraction model is as follows: The model input data is a gait silhouette sequence with dimensions (B, T, C, H, W), where B is the batch size (a common parameter in deep learning models), T is the number of time frames, C is the number of channels, H is the height of the feature map, and W is the width of the feature map. The data is processed through the STEM module and the backbone network. Simultaneously, the output feature map from each stage is input into a multi-stage temporal global feature interaction enhancement module to obtain enhanced feature maps. The output feature maps from the backbone network and the multi-stage temporal global feature interaction enhancement module are then input into a discriminative feature aggregation module to obtain the final gait feature vector. This vector is then processed through L... triplet Triple loss and L ce Supervised training was performed using a contrast loss to obtain the final spatial frequency domain pedestrian gait feature extraction model.

[0038] This invention significantly improves the model's gait feature extraction performance, thereby enhancing the accuracy and robustness of gait recognition. First, through the spatial-frequency feature fusion module, this invention can efficiently separate and fuse high- and low-frequency features, strengthening the main structural features and edge texture details, thus improving the richness and accuracy of feature representation. Second, the design of the multi-stage temporal global feature interaction enhancement module enables the model to progressively extract and enhance global features in both the spatial and temporal domains, further improving the model's capabilities. In addition, the discriminative feature aggregation module further enhances the discriminativeness of features by fusing the output feature map of the backbone network with the feature map enhanced by the multi-stage temporal global feature interaction enhancement module and performing HPM operations, enabling the model to maintain excellent recognition performance even in complex scenarios.

[0039] Example 2 This embodiment provides a readable storage medium storing a computer program. When executed by a processor, the computer program causes the processor to perform the spatial frequency domain pedestrian gait feature extraction method based on a spiking neural network as described in Embodiment 1.

[0040] Example 3 This embodiment discloses a computer device, including a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, it implements the spatial frequency domain pedestrian gait feature extraction method based on spiking neural network driven as described in Embodiment 1.

[0041] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for extracting pedestrian gait features in the spatial frequency domain based on a spiking neural network, characterized in that: A spatial frequency domain pedestrian gait feature extraction model is used to extract gait features from gait images; The spatial frequency domain pedestrian gait feature extraction model includes: a backbone network, a multi-stage temporal global feature interaction enhancement module, and a discriminative feature aggregation module; The backbone network comprises N sequentially connected stages, each stage including a space-frequency feature fusion module for fusing spatial and frequency domain features. The multi-stage temporal global feature interaction enhancement module extracts and enhances spatial and temporal global features from the output feature maps Si, i=1,2,…,N of each stage of the backbone network, to obtain the output feature map S. T_total The discriminative feature aggregation module combines the backbone network output with the feature map S output by the multi-stage temporal global feature interaction enhancement module. T_total Aggregation is performed to obtain the gait features finally output by the spatial frequency domain pedestrian gait feature extraction model.

2. The method for extracting pedestrian gait features in the spatial frequency domain based on a spiking neural network according to claim 1, characterized in that: Each space-frequency feature fusion module includes a spatial domain extraction branch and a frequency domain extraction branch; each space-frequency feature fusion module divides the feature map input to the space-frequency feature fusion module into two feature maps I1 and I2; The spatial extraction branch extracts spatial features from feature map I1 using two convolutional kernels. The frequency domain extraction branch performs a Fourier transform on the feature map I2, then separates the high-frequency and low-frequency components into two convolutional kernels. These components are then inversely transformed and input into the corresponding convolutional networks to obtain the low-frequency feature U in the spatial domain. 0L and high-frequency features U 0H High-frequency feature U 0H The high-frequency component weights W are obtained through a spiking neural network LIF; low frequency Feature U 0L After adding the spatial features extracted by the spatial extraction branch, the dual-branch low-frequency fusion feature U1 is obtained; the dual-branch low-frequency fusion feature U1 is multiplied by the low-frequency component weight (1-W) to obtain the fused low-frequency feature U. 1L The low-frequency fusion feature U1 of the dual-branch system is multiplied by the high-frequency component weight W and then combined with the high-frequency feature U. 0H Add them together to obtain the fused high-frequency feature U. 1H Then, the fused low-frequency features U 1L and high-frequency features U 1H Further addition and fusion are performed to obtain the hybrid output feature map U2; The feature map I is processed through a channel attention mechanism to obtain channel weights, which are then multiplied with the weights obtained from the spiking neural network LIF to obtain the updated channel attention weights W. C The channel attention weight W C Multiplying the mixed output feature map U2 with the attention-based feature map U2 yields the attention-based feature map U. C After normalization, the updated features are input into the FFN feedforward neural network module. Finally, the features are added to the feature map I through the residual linking module to obtain the final output feature map Si of the space-frequency feature fusion module.

3. The method for extracting pedestrian gait features in the spatial frequency domain based on a spiking neural network according to claim 2, characterized in that: The feature maps I1 and I2 are: I1,I2=split(I); where split is the segmentation function; The dual-branch low-frequency fusion feature U1 is: U1=Conv2D ×2 (I1)+ Conv2D(IFFT([Conv2D ×2 (FFT(I2))] LF )) ; Where Conv2D() is a 2D convolution; Conv2D ×2 ( ) represents a 2D convolution with two kernels; FFT() represents the Fourier transform; IFFT() represents the inverse Fourier transform; [ ] LF Represents low-frequency components; The weight W of the high-frequency components is: W=LIF(Conv2D(IFFT([Conv2D ×2 (FFT(I2))] HF ))) ; Wherein, LIF() represents a spiking neural network; [ ] HF Represents high-frequency components; The fused low-frequency feature U 1L and high-frequency features U 1H They are respectively: U 1L =U1×(1-W); U 1H =U1×W+Conv2D(IFFT([Conv2D ×2 (FFT(I2))] HF )) ; The mixed output feature map U2 is: U2=Conv2D(BN(Conv2D(U 1L ))+BN(Conv2D(U 1H ))) ; Where BN() is the batch normalization operation; Channel attention weight W C for: W C = Tanh(attention(I) ×LIF(attention(I))) ; Where attention() is the attention mechanism; Tanh() is the activation function; Feature map U based on attention mechanism C for: IN C =U2×W C ; The final output feature map Si of the space-frequency feature fusion module is: Si = FFN(Norm(U C ))+I; Here, FFN() is the feedforward neural network module; Norm() is the normalization operation.

4. The method for extracting pedestrian gait features in the spatial frequency domain based on a spiking neural network according to claim 1, characterized in that: The multi-stage temporal global feature interaction enhancement module includes a temporal global feature fusion module, a spatial global feature enhancement network branch, and a temporal global feature enhancement network branch. The output feature maps Si of each stage of the backbone network are used to calculate the spatial global features S through spatial pooling and temporal pooling, respectively. si and temporal global features S Ti Spatial global features S si and temporal global features S Ti The common inputs are fed into the temporal global feature fusion module to obtain the spatiotemporal global feature S. STi ; The spatial global features S of each stage si With the global features in the space-time domain S STi Merging into merged features (S) along the time dimension si , S STi ); Merging characteristics at each stage (S) si , S STi The inputs are respectively fed into the spatial global feature enhancement network branch; The spatial global feature enhancement network branch consists of N stages of spatial enhancement convolutional groups; the merged features (S) of the current stage si , S STi The output feature map (S) of the previous stage spatial enhancement convolution group s i-1 ,S ST i-1 The sums are then fed into the current stage's spatial enhancement convolutional group for feature extraction to obtain the output feature map (S) of the current stage's spatial enhancement convolutional group. s i ,S ST i ); Temporal global feature map S at each stage Ti The inputs are respectively fed into the temporal global feature enhancement network branch; The temporal global feature enhancement network branch consists of N stages of temporal enhancement convolutional groups; the temporal global feature map S of the current stage. Ti Compared with the output feature map S of the previous stage temporal enhancement convolution group T i-1 The sums are then fed into the current stage's temporal augmentation convolutional group for feature extraction, yielding the output feature map S of the current stage's temporal augmentation convolutional group. T i ; The output feature map (S) of the final stage spatial enhancement convolution group s N ,S ST N Segmented into features S s N ,S ST N ; With feature S s N As a global feature in the spatial domain S s(N+1) The output feature map S of the final stage temporal enhancement convolution group T N As a time-domain global feature S T(N+1) The common input temporal global feature fusion module obtains the output spatiotemporal global feature S. ST(N+1) , with feature S ST N The feature map S is obtained by fusion. ST_temp Feature map S ST_temp With feature map S T N The weights obtained through the activation function are weighted and fused to obtain the final output feature map S of the multi-stage temporal global feature interaction enhancement module. T_total .

5. The method for extracting pedestrian gait features in the spatial frequency domain based on a spiking neural network according to claim 4, characterized in that: The calculation method in the multi-stage temporal global feature interaction enhancement module is as follows: Initialize S s 0 =0;S ST 0 =0;S T 0 =0; Spatial global features S si and temporal global features S Ti They are respectively: ； ； Where T is the number of time frames, and C, H, and W are the number of channels, height, and width of feature map Si, respectively; Output feature maps of spatial enhancement convolutional groups at each stage (S s i ,S ST i )for: (S s i ,S ST i )=Conv2D ×2 (concat(S s i-1 ,S ST i-1 )+concat(S si ,S STi ) ; Output feature maps S of temporal augmentation convolutional groups at each stage T i for: S T i =Conv2D ×2 (S T i-1 +S Ti ); Feature map S ST_temp for: S ST_temp =Conv2D ×2 (f st (S s N ,S T N )+S ST N ); The final output feature map S of the multi-stage temporal global feature interaction enhancement module T_total for: S T_total =sigmoid(S ST_temp ) ×sigmoid(S T N ) ×S ST_temp +S T N 。 6. The method for extracting pedestrian gait features in the spatial frequency domain based on a spiking neural network according to claim 4, characterized in that: Spatial global features S si and temporal global features S Ti The common inputs are into the temporal global feature fusion module: spatial global feature S si After feature extraction via two layers of 3D convolution, a temporal attention score is obtained through a temporal attention mechanism module consisting of two layers of 3D convolution and activated in the temporal dimension. This temporal attention score is then compared with the spatial global feature S. si The fused features S are obtained by performing dot product weighting. si_temp Temporal global features S Ti After max pooling and average pooling, it is combined with the feature S si_temp The features are added together, and the spatiotemporal global features S output by the temporal global feature fusion module are obtained through two layers of 3D convolution. STi .

7. The method for extracting pedestrian gait features in the spatial frequency domain based on a spiking neural network according to claim 6, characterized in that: The spatiotemporal global feature S output by the temporal global feature fusion module STi for: S STi =f st (S si ,S Ti ) =Conv3D ×2 (Softmax(Conv3D ×2 (R(S si ))) ×Conv3D ×2 (R(S si ))+maxpool(S Ti )+meanpool(S Ti )); Among them, f st ( ) represents the temporal global feature fusion module; R() is the reshape operation; Conv3D ×2 ( ) represents two layers of 3D convolution; Softmax() is the activation function; maxpool() and meanpool() are the spatial max pooling and average pooling operations, respectively.

8. The method for extracting pedestrian gait features in the spatial frequency domain based on a spiking neural network according to claim 1, characterized in that: The feature map S is obtained by performing temporal pooling on the output feature map SN of the final stage of the backbone network. m In the feature aggregation module, the feature map S is... m The feature map S is output by the multi-stage temporal global feature interaction enhancement module. T_total After aggregation and feature fusion via convolution, the results are input into the HPM operation to obtain the output S of the discriminative feature aggregation module. out : S out =HPM(Conv2D ×2 (Cancat(S T_total ,S m ))); HPM() is the block operation; Conv2D ×2 ( ) represents a 2D convolution with two kernels; Cancat() represents an aggregation operation.

9. A readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the spatial frequency domain pedestrian gait feature extraction method based on a spiking neural network as described in any one of claims 1-8.

10. A computer device comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the spatial frequency domain pedestrian gait feature extraction method based on spiking neural network driven by any one of claims 1-8.

Images