A lightweight cross-view gait recognition method based on a multi-layer perceptron network

Abstract

A kind of lightweight cross-view gait recognition method based on multilayer perceptron network, comprising the following steps: obtaining and preprocessing gait dataset including gait contour graph and gait skeleton graph;Gait recognition model including spatial compression enhancement module and space-time feature extraction module is constructed;Gait contour graph and gait skeleton graph are input into spatial compression enhancement module, compressed through multistage convolution, coded by pulse neural network LIF neuron and channel attention mechanism, to generate compressed and enhanced dual-mode fusion features;Dual-mode fusion features are input into space-time feature extraction module, to generate discriminative gait features through dynamic window space-time segmentation, frequency domain processing and multidimensional perceptron modeling;Gait recognition is carried out based on discriminative gait features;Gait recognition model is trained by jointly optimizing triplet loss and cross-entropy loss;The gait recognition model trained is tested.

Description

Technical Field

[0001] This invention relates to the field of gait recognition technology, and in particular to a lightweight cross-view gait recognition method based on a multilayer perceptron network. Background Technology

[0002] Existing gait recognition methods generally employ deep residual convolutional backbones to represent binary contour sequences, acquiring high semantic space features through multi-stage downsampling and channel expansion, and combining ensemble / pyramid-style temporal aggregation, 3D convolution, or pseudo-3D convolution in the temporal dimension to achieve cross-frame information modeling. In discriminative learning, batch-normalized neck structures are often used as feature heads, combined with cross-entropy loss and triplet loss for optimization, aiming to simultaneously consider identity classification and metric learning performance.

[0003] The drawback of existing technologies is that, while pursuing high accuracy, they generally suffer from large parameter scales and high computational complexity: deep backbones and multi-stage stacking significantly increase the number of parameters and GPU memory usage, and the introduction of 3D or pseudo-3D convolutional temporal units further increases floating-point operations and inference time. Due to these limitations, such methods are difficult to deploy on resource-constrained terminals and cannot meet the dual constraints of latency and cost for large-scale, real-time applications. Summary of the Invention

[0004] In view of the aforementioned shortcomings of existing technologies, this invention provides a lightweight cross-view gait recognition method based on a multilayer perceptron network. By introducing a spatial compression enhancement module, it addresses two heterogeneous gait representations—contour maps and skeletal maps—significantly reducing the feature space resolution while efficiently mining and enhancing discriminative edge information. Furthermore, a lightweight fusion mechanism effectively integrates complementary cross-modal information. Building upon this, a spatiotemporal feature extraction module is designed. Through dynamic window spatiotemporal slicing, combined with joint interactive modeling of the multilayer perceptron and a frequency domain module, multidimensional feature reconstruction and supplementation are performed in height, width, and channel dimensions to achieve a full feature representation of the gait sequence. Ultimately, this method can extract discriminative gait features for identity recognition, effectively mitigating the problem of significantly increased computational overhead during training and inference stages caused by excessive model parameters.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A lightweight cross-view gait recognition method based on a multilayer perceptron network includes the following steps:

[0007] S1. Obtain and preprocess the gait dataset, which includes gait contour maps and gait skeleton maps;

[0008] S2. Construct a gait recognition model including a spatial compression enhancement module and a spatiotemporal feature extraction module; input the gait contour map and gait skeleton map into the spatial compression enhancement module, and generate compressed and enhanced bimodal fusion features through multi-level convolutional compression, LIF neuron encoding of spiking neural networks and channel attention mechanism; input the bimodal fusion features into the spatiotemporal feature extraction module, and generate discriminative gait features through dynamic window spatiotemporal segmentation, frequency domain processing and multi-dimensional perceptron modeling; perform gait recognition based on discriminative gait features;

[0009] S3. Train the gait recognition model by jointly optimizing triplet loss and cross-entropy loss;

[0010] S4. Test the trained gait recognition model.

[0011] Preferably, S2 includes:

[0012] S21. The gait contour map and gait skeleton map are sequentially passed through the first convolutional layer group, the first LIF neuron module, and the first channel attention module of the spatial compression enhancement module for preliminary convolutional compression, pulse recoding, and channel dimension recalibration, respectively. Then, they are sequentially passed through the second convolutional layer group, the second LIF neuron module, and the second channel attention module for further spatial compression, pulse coding, and channel recalibration. After convolutional compression through the third convolutional layer group, the contour features and skeleton features after multi-level compression enhancement are adaptively weighted and fused by the fusion gating module to generate dual-modal fusion features.

[0013] As a preferred embodiment, the implementation of the space compression enhancement module includes:

[0014] S211. Input the gait contour map and gait skeleton image into two-dimensional convolutional layer a and two-dimensional convolutional layer c respectively to extract features and generate the first contour feature map and the first skeleton feature map;

[0015] S212. Input the first contour feature map and the first bone feature map into LIF neuron module a and LIF neuron module c respectively for pulse recoding to generate the first pulse contour enhancement feature and the first bone contour enhancement feature; input the first contour feature map and the first bone feature map into channel attention module a and channel attention module c respectively for channel dimension recalibration, and multiply them element-wise with the first pulse contour enhancement feature and the first bone contour enhancement feature respectively to generate the first contour calibration feature and the first bone calibration feature;

[0016] S213. Input the first contour calibration feature and the first bone calibration feature into the two-dimensional convolutional layer b and the two-dimensional convolutional layer d respectively for further spatial compression to generate the second contour feature map and the second bone feature map.

[0017] S214. Input the second contour feature map and the second bone feature map into LIF neuron module b and LIF neuron module d respectively for pulse coding to generate the second contour pulse enhancement feature and the second bone pulse enhancement feature; input the second contour feature map and the second bone feature map into channel attention module b and channel attention module d respectively to perform channel recalibration on the second contour pulse enhancement feature and the second bone pulse enhancement feature to generate the second contour calibration feature and the second bone calibration feature.

[0018] S215. Input the second contour calibration feature and the second bone calibration feature into the two-dimensional convolutional layer e and the two-dimensional convolutional layer f respectively to generate the third contour feature map and the third bone feature map; concatenate the third contour feature map and the third bone feature map along the channel dimension, and then input them into the fusion gating module for adaptive weighted fusion to obtain the dual-modal fusion feature.

[0019] As a preferred embodiment, S2 also includes:

[0020] S22. The dual-modal fusion features are input into the dynamic window segmentation module of the spatiotemporal feature extraction module for dynamic window segmentation to generate window segmentation features. The window segmentation features are then processed by pooling, convolution, and upsampling layers and input into the first frequency domain processing module to obtain frequency domain enhancement features. The frequency domain enhancement features are input into the multi-branch processing module, and the results are fused to generate multi-dimensional fusion attention features. The multi-dimensional fusion attention features are then processed by the second frequency domain processing module to generate discriminative gait features.

[0021] As a preferred embodiment, S2 also includes:

[0022] S23. Input the discriminative gait features into two fully connected layers respectively to obtain the metric feature vector and the classification feature vector.

[0023] Preferably, the multi-branch processing module for frequency domain enhancement feature input in S22 includes:

[0024] The frequency domain enhancement features are fed into the high-dimensional processing module of the first branch to obtain high-dimensional reconstructed features; the frequency domain enhancement features are fed into the wide-dimensional processing module of the second branch to obtain wide-dimensional reconstructed features; the frequency domain enhancement features are fed into the channel dimension processing module of the third branch to obtain channel dimension reconstructed features; the high-dimensional reconstructed features, wide-dimensional reconstructed features and channel dimension reconstructed features are added element-wise and then input into the channel attention module e to generate multi-dimensional fused attention features.

[0025] Preferably, the high-dimensional processing module, the wide-dimensional processing module, and the channel-dimensional processing module all contain two branches:

[0026] One branch is used to flatten the extracted sub-feature maps and feed them into a multilayer perceptron containing two layers of 2D convolutions and activations; the output of the multilayer perceptron is then used for feature reshaping through depthwise separable convolutions.

[0027] Another branch is used to pass the frequency domain enhanced features through Fast Fourier Transform, frequency domain attention, and Inverse Fast Fourier Transform in sequence; the outputs of the two branches are added element by element, and the processed sub-feature maps are then pieced back together as is to obtain the reconstructed features of each branch.

[0028] As a preferred embodiment, channel attention module a, channel attention module b, channel attention module c, channel attention module d, and channel attention module are implemented as follows:

[0029] Global max pooling and global average pooling operations are performed on the input features respectively, and the resulting two feature vectors are input into two multilayer perceptrons with the same structure. The output features of the two multilayer perceptrons are added element by element, and channel attention weights are generated by the sigmoid activation function. The channel attention weights are multiplied with the input feature map channel by channel to achieve feature recalibration at the channel level.

[0030] Preferably, S3 includes:

[0031] The loss function is constructed as follows:

[0032] L = L1 + L2

[0033] Where L1 is the identity classification loss calculated using Softmax Loss based on the classification feature vector, and L2 is the metric feature vector loss calculated using Triplet Loss.

[0034] Compared with the prior art, the beneficial effects of the present invention are reflected in:

[0035] 1. Unlike traditional gait recognition methods that only use gait contour maps as network input, this invention uses dual gait features—contour maps and skeleton maps—as initial input. After multi-level convolutional compression, LIF neuron modules from a spiking neural network are introduced to perform pulse-like recoding of the features. Simultaneously, a channel attention mechanism is used to selectively enhance the discriminative channel. Then, a lightweight fusion mechanism effectively integrates cross-modal complementary information, thereby significantly reducing the feature space resolution while improving the temporal sensitivity and discriminative ability of gait representation, reducing the model's parameter size, and improving overall computational efficiency.

[0036] 2. Unlike traditional techniques that uniformly mix and process gait features across all dimensions, this invention first employs a dynamic window segmentation mechanism to spatiotemporally slice the gait sequence. Then, it introduces a multilayer perceptron module and a frequency domain processing module to work collaboratively, interactively modeling gait information from both spatial and frequency domain perspectives. Based on this, a multi-dimensional processing module is constructed, incorporating high-dimensional, wide-dimensional, and channel-dimensional features, enabling the reconstruction and supplementation of features from different dimensions. This significantly reduces model complexity and inference overhead while maintaining high recognition accuracy.

[0037] 3. The key technical points of this invention lie in the design of a spatial compression enhancement module and a spatiotemporal feature extraction module. The spatial compression enhancement module, accommodating dual gait inputs of contour and skeleton maps, introduces LIF neurons from a spiking neural network into the gait recognition task for the first time. These neurons work synergistically with convolutional layers and channel attention mechanisms, and then, through a lightweight fusion mechanism, effectively integrates cross-modal complementary information, achieving selective enhancement of gait features and effective compression of feature spatial resolution. This provides a compact and more discriminative representation for subsequent spatiotemporal modeling. The spatiotemporal feature extraction module decouples temporal and spatial domain information for modeling and explicitly introduces frequency domain representation through Fourier transform. Simultaneously, corresponding processing branches are designed for high-dimensional, wide-dimensional, and channel-dimensional features, thereby effectively controlling the model parameter scale and computational complexity while effectively extracting discriminative features. The structural design and synergistic effect of these two modules enable the model to significantly reduce the number of parameters and computational overhead while maintaining recognition performance, thus improving the efficiency and applicability of gait recognition. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the space compression enhancement module in Embodiment 1;

[0039] Figure 2 This is a schematic diagram of the channel attention module in the spatial compression enhancement module of Embodiment 1;

[0040] Figure 3 This is a schematic diagram of the spatiotemporal feature extraction module in Example 1;

[0041] Figure 4 It is the high-dimensional, wide-dimensional, and channel-dimensional processing module in the spatiotemporal feature extraction module of Example 1. Detailed Implementation

[0042] To make the technical means, inventive features, objectives, and effects of the invention readily understandable, the invention is further described below with reference to specific illustrations. However, the invention is not limited to the embodiments described below.

[0043] It should be noted that the structures, proportions, sizes, etc., illustrated in the accompanying drawings of this specification are only used to complement the content disclosed in the specification for those skilled in the art to understand and read, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0044] Example 1:

[0045] like Figure 1 The lightweight cross-view gait recognition method based on a multilayer perceptron network, as shown, includes the following steps:

[0046] S1. Obtain and preprocess the gait dataset, which includes gait contour maps and gait skeleton maps;

[0047] Gait datasets containing multi-view walking sequences were acquired from publicly available gait recognition data resources to serve as the training and validation basis for a lightweight cross-view gait recognition model. The datasets used included representative multi-view gait databases such as CASIA-B, OU-MVLP, Gait3D, and GREW. These datasets provide gait video sequences of walkers from multiple camera perspectives, along with corresponding identity annotation information, effectively supporting cross-view feature learning needs. After acquiring the data, the original video frames or contour maps underwent formatting and preprocessing, including frame size normalization, invalid frame filtering, and contour quality checks, to meet the subsequent feature input requirements based on a multilayer perceptron network. Simultaneously, the processed data was divided into training, validation, and test sets according to identity level to ensure the reliability of the model training process and the comparability of evaluation results, laying the foundation for the construction of the lightweight gait recognition model.

[0048] In this embodiment, the single-channel gait contour map sample is resized to 1×64×44 using bilinear interpolation, while the gait skeleton map is resized to 2×64×44, where 1 and 2 both represent the number of image channels.

[0049] S2. Construct a gait recognition model including a spatial compression enhancement module and a spatiotemporal feature extraction module; input the gait contour map and gait skeleton map into the spatial compression enhancement module, and generate compressed and enhanced bimodal fusion features through multi-level convolutional compression, LIF neuron encoding of spiking neural networks and channel attention mechanism; input the bimodal fusion features into the spatiotemporal feature extraction module, and generate discriminative gait features through dynamic window spatiotemporal segmentation, frequency domain processing and multi-dimensional perceptron modeling; perform gait recognition based on discriminative gait features;

[0050] This model uses a multilayer perceptron (MLP) and convolution as its backbone, and takes gait contour maps and gait skeleton maps from the gait dataset as input. It adopts a two-stage architecture of "spatial compression enhancement - spatiotemporal feature extraction": In the spatial compression enhancement stage, the LIF neuron module from the spiking neural network is introduced, and convolution, pooling and channel attention mechanisms are combined to significantly reduce the number of parameters and floating-point operations while maintaining key edges and shape cues; In the spatiotemporal feature extraction stage, the temporal and spatial domain information is decoupled and modeled, and the frequency domain representation is explicitly introduced using Fourier transform. Gait features are further captured through multi-dimensional modeling with small parameters, thereby improving inference efficiency while ensuring discriminability.

[0051] S21. The gait contour map and gait skeleton map are sequentially passed through the first convolutional layer group, the first LIF neuron module, and the first channel attention module of the spatial compression enhancement module, where they undergo preliminary convolutional compression, pulse recoding, and channel dimension recalibration. Then, they are sequentially passed through the second convolutional layer group, the second LIF neuron module, and the second channel attention module for further spatial compression, pulse coding, and channel recalibration. Finally, after convolutional compression by the third convolutional layer group, the contour features and skeleton features after multi-level compression enhancement are adaptively weighted and fused by the fusion gating module to generate dual-modal fusion features.

[0052] The spatial compression enhancement module consists of seven convolutional layers, four LIF neuron modules, four channel attention modules, and one pooling layer. It is used for spatial compression enhancement of dual gait features from the contour map and skeleton map. The channel attention module comprises a max-pooling layer, an average-pooling layer, and a multilayer perceptron. The specific steps are as follows:

[0053] S211. Input the gait contour map and gait skeleton image into two-dimensional convolutional layers a and c of the first convolutional layer group, respectively, for feature extraction. The convolutional kernel size is 3×3, the stride is 1, the padding is 1, and the output channels are 8. Then, the ReLU activation function is used to obtain the first contour feature map and the first skeleton feature map, both with dimensions of 8×64×44.

[0054] S212. Input the first contour feature map and the first bone feature map into LIF neuron module a and LIF neuron module c of the first LIF neuron module respectively for pulse recoding to generate the first pulse contour enhancement feature and the first bone contour enhancement feature.

[0055] The first contour feature map and the first bone feature map are respectively input into channel attention module a and channel attention module c of the first channel attention module to recalibrate the channel dimension of the first pulse contour enhancement feature and the first bone contour enhancement feature, thereby generating the first contour calibration feature and the first bone calibration feature.

[0056] In channel attention modules a and c, firstly, global max pooling and global average pooling operations are performed on the input features, respectively. The resulting two feature vectors are then input into two identical multilayer perceptrons (MLPs). Each MLP contains: a 2D convolutional layer with a 3×3 kernel, a stride of 1, padding of 1, and an output channel count that is 1 / 4 of the input channel count (compression ratio set to 4); followed by a ReLU activation function; and then another 2D convolutional layer with the same 3×3 kernel, stride of 1, padding of 1, restoring the output channel count to 4 times the input channel count. The output features of the two MLPs are then element-wise summed, and the final channel attention weights are generated using a Sigmoid activation function. These channel attention weights are then multiplied channel-wise by the input feature map to achieve channel-level feature recalibration. The result is then passed through the Sigmoid activation function and multiplied element-wise with the first pulse contour enhancement feature and the first bone contour enhancement feature, respectively. The final output first contour calibration feature and first bone calibration feature are both 8×64×44 in dimension.

[0057] S213. Input the first contour calibration feature and the first bone calibration feature into the second convolutional layer group respectively to further compress the two-dimensional convolutional layer b and the two-dimensional convolutional layer d to generate the second contour feature map and the second bone feature map.

[0058] Two-dimensional convolutional layers b and d have kernel sizes of 3×5, strides of 2, padding of 1×0, and 16 output channels. Then, the ReLU activation function is applied to obtain features of dimensions 16×32×20.

[0059] S214. Input the second contour feature map and the second bone feature map into the LIF neuron module b and LIF neuron module d of the second LIF neuron module respectively for pulse coding to generate the second contour pulse enhancement feature and the second bone pulse enhancement feature.

[0060] The second contour feature map and the second bone feature map are respectively input into channel attention module b and channel attention module d of the second channel attention module. Channel recalibration is performed on the second contour pulse enhancement feature and the second bone pulse enhancement feature to generate the second contour calibration feature and the second bone calibration feature.

[0061] like Figure 2As shown, in channel attention modules b and d, firstly, global max pooling and global average pooling operations are performed on the input features (second contour feature map and second skeleton feature map), respectively. The resulting two feature vectors are then input into two structurally identical multilayer perceptrons (MLPs). Each MLP contains: a two-dimensional convolutional layer with a kernel size of 5×5, a stride of 1, padding of 2, and an output channel count that is 1 / 8 of the input channel count (compression ratio set to 8); followed by a ReLU activation function; then another two-dimensional convolutional layer with a kernel size of 3×3, a stride of 1, padding of 1, and an output channel count restored to 8 times the input channel count. The output features of the two MLPs are then element-wise summed, and the final channel attention weights are generated using a Sigmoid activation function. These weights are then multiplied channel-wise by the original input feature map to achieve channel-level feature recalibration. The result is then passed through the Sigmoid activation function and multiplied element-wise with the second contour pulse enhancement feature and the second bone pulse enhancement feature. The final output of the second contour calibration feature and the second bone calibration feature are both 16×32×20 in dimension.

[0062] S215. Input the second contour calibration features and the second bone calibration features into the two-dimensional convolutional layer e and the two-dimensional convolutional layer f of the third convolutional layer group, respectively. The convolutional kernel size is 3×5, the stride is 2×1, the padding is 1×0, and the output channel is 32 channels. Then, use the ReLU activation function to obtain the third contour feature map and the third bone feature map, both with dimensions of 32×16×16.

[0063] The third contour feature map and the third skeleton feature map are concatenated along the channel dimension and then input into the fusion gating module for adaptive weighted fusion. The convolutional layer g has a kernel size of 3×3, a stride of 1×1, padding of 1×1, and 32 output channels. It is then input into the pooling layer and global average pooling is performed in the spatial dimension to obtain the global description vector of each channel. Subsequently, it is input into the multilayer perceptron and outputs a vector of length 2C, which is then transformed into a 2×C shape, where C is 32 and 2 corresponds to the two modalities of contour and skeleton. Each channel has a pair of logits, and softmax is applied to the modal dimension, expanding the dimension to 32×1×1. The vector is then multiplied element-wise with the third contour feature map and the third skeleton feature map respectively, and finally added to obtain a dual-modal fusion feature with an output dimension of 32×16×16.

[0064] S22. Input the dual-modal fusion features into the spatiotemporal feature extraction module, and generate discriminative gait features through dynamic window spatiotemporal segmentation, frequency domain processing and multi-dimensional perceptron modeling;

[0065] like Figure 3As shown, the spatiotemporal feature extraction module consists of a dynamic window segmentation module, three convolutional layers, a pooling layer, an upsampling layer, a multilayer perceptron module, two frequency domain processing modules, a channel attention module, a high-dimensional processing module, a wide-dimensional processing module, and a channel dimension processing module. The high-dimensional, wide-dimensional, and channel dimension processing modules are all composed of a multilayer perceptron, a frequency domain processing module, and depthwise separable convolutional layers. The frequency domain processing module consists of Fast Fourier Transform, frequency domain attention, and Inverse Fast Fourier Transform. Specifically, it includes the following steps:

[0066] S221. Input the dual-modal fusion features into the dynamic window segmentation module of the spatiotemporal feature extraction module to perform dynamic window segmentation and generate window segmentation features, specifically including:

[0067] The window length is adaptively determined based on the number of frames in the original gait sequence corresponding to the dual-modal fusion features. After zero-padding the time dimension according to the window length, the input is fed into the convolutional layer to generate window segmentation features.

[0068] The window segmentation is based on prior experience to reasonably divide the time dimension. If the number of input frames is less than 10, the window length is 4 frames; if the number of input frames is greater than or equal to 10 but less than 20, the window length is 6 frames; if the number of input frames is greater than or equal to 20 but less than 30, the window length is 8 frames; and if the number of input frames is greater than or equal to 30, the window length is 10 frames. If the segmentation is insufficient, zeros are padded at the beginning and end of the time dimension. The image is then input into a two-dimensional convolutional layer h with a kernel size of 3×3, a stride of 2, padding of 1, and 128 output channels. Batch normalization and ReLU activation function are then used to obtain a window segmentation feature with a dimension of 64×8×8.

[0069] S222. The window segmentation features are passed through pooling layer a with a pooling window size of 2×2, then through convolutional layer i with a kernel size of 1×1, stride of 1, padding of 0, and 64 output channels. Batch normalization and ReLU activation functions are applied, and finally, the pooled upsampled features are obtained through upsampling layer a with a dimension of 64×8×8.

[0070] S223. The pooled upsampled features are processed through the first frequency domain processing module: first, they undergo Fast Fourier Transform (FFT), then frequency domain attention is obtained, then inverse FFT is performed, and then they are processed through a multilayer perceptron with the same input and output dimensions. After batch normalization and sigmoid activation function, the result is multiplied element-wise with the output features obtained in b) to obtain frequency domain enhanced features with an output dimension of 256×8×8.

[0071] S224. Input the frequency domain enhancement feature into the first branch. For example... Figure 4As shown, the first branch employs a high-dimensional processing module: one branch is first divided into 64 sub-feature maps in the channel dimension and 4 sub-feature maps in the long dimension, resulting in 64 sub-feature maps, each with a size of 64×4×1. Each sub-feature map is flattened into a 256×1 vector and then fed into a multilayer perceptron. This multilayer perceptron contains two layers of two-dimensional convolution and activation: the first convolutional layer has a kernel size of 3×3, a stride of 1, padding of 1, and the number of output channels is 1 / 4 of the number of input channels; ReLU activation is then applied; the second convolutional layer has a kernel size of 3×3, a stride of 1, padding of 1, and the number of output channels is restored to 4 times the number of input channels. Subsequently, depthwise separable convolutions are introduced for feature reshaping: the depthwise convolution branch has a kernel size of 3×1, a stride of 1×1, padding of 1×0, 64 groups, and 64 output channels; the pointwise convolution branch has a kernel size of 1×1, a stride of 1, and 64 output channels. The other branch first undergoes a Fast Fourier Transform (FFT), then frequency domain attention, followed by an Inverse FFT. Finally, the outputs of the two branches are summed element-wise. The 64 processed sub-feature maps are then reassembled as is, resulting in a high-dimensional reconstructed feature map with a size of 256×8×8.

[0072] S225. The frequency domain enhancement features are fed into the second branch. The second branch uses a wide-dimensional processing module: one branch is first divided into 64 channels and 4 wide channels, resulting in 64 sub-feature maps, each with a size of 64×1×4. Each sub-feature map is flattened into a 256×1 vector and then fed into a multilayer perceptron. This multilayer perceptron contains two layers of two-dimensional convolution and activation: the first convolutional layer has a kernel size of 5×5, a stride of 1, padding of 2, and the number of output channels is 1 / 8 of the number of input channels; then ReLU activation is applied; the second convolutional layer has a kernel size of 3×3, a stride of 1, padding of 1, and the number of output channels is restored to 8 times the number of input channels. Subsequently, depthwise separable convolutions are introduced for feature reshaping: the depthwise convolution branch has a kernel size of 1×3, a stride of 1×1, padding of 0×1, 64 groups, and 64 output channels; the pointwise convolution branch has a kernel size of 1×1, a stride of 1, and 64 output channels. The other branch first undergoes a Fast Fourier Transform (FFT), then frequency domain attention, followed by an Inverse FFT. Finally, the outputs of the two branches are summed element-wise. The 64 processed sub-feature maps are then reassembled as is, resulting in a wide-dimensional reconstructed feature map with dimensions of 256×8×8.

[0073] S226. The frequency domain enhancement features are fed into the third branch. The third branch uses a channel dimension processing module: one branch is not divided, resulting in 64 sub-feature maps, each with a size of 256×1×1. Each sub-feature map is flattened into a 256×1 vector and then fed into a multilayer perceptron; this multilayer perceptron contains two layers of two-dimensional convolution and activation: the first convolutional layer has a kernel size of 3×3, a stride of 1, padding of 1, and the number of output channels is 1 / 4 of the number of input channels; then ReLU activation is applied; the second convolutional layer has a kernel size of 5×5, a stride of 1, padding of 2, and the number of output channels is restored to 4 times the number of input channels. Subsequently, depthwise separable convolutions are introduced for feature reshaping: the depthwise convolution branch has a 1×1 kernel size, a 1×1 stride, zero padding, 256 groups, and 64 output channels; the pointwise convolution branch also has a 1×1 kernel size, a 1×1 stride, zero padding, and 64 output channels. The other branch first undergoes a Fast Fourier Transform (FFT), then frequency domain attention, followed by an Inverse FFT. Finally, the outputs of the two branches are summed element-wise. The 64 processed sub-feature maps are then reassembled as is, resulting in a channel-dimensional reconstructed feature map with a size of 256×8×8.

[0074] S227. The output features of the three branches above are summed element-wise and input into the channel attention module e. First, global max pooling and global average pooling operations are performed on the input features respectively. The resulting two feature vectors are then input into two multilayer perceptrons (MLPs) with identical structures. Each MLP contains: a two-dimensional convolutional layer with a kernel size of 3×3, a stride of 1, padding of 1, and an output channel count that is 1 / 8 of the input channel count (compression ratio set to 8); followed by a ReLU activation function; then another two-dimensional convolutional layer with a kernel size of 3×3, a stride of 1, padding of 1, and the output channel count restored to 8 times the input channel count. The output features of the two MLPs are summed element-wise, and the final channel attention weights are generated using the Sigmoid activation function. These weights are multiplied channel-wise with the original input feature map to achieve channel-level feature recalibration, resulting in a multi-dimensional fused attention feature with a final output dimension of 256×8×8.

[0075] S228. The multi-dimensional fused attention feature, obtained by adding the three branches, is processed through the second frequency domain processing module: first, it undergoes a Fast Fourier Transform (FFT) to obtain the frequency domain attention, followed by an Inverse Fast Fourier Transform (IFFT). Then, it passes through convolutional layer j with a kernel size of 1×1, a stride of 1, padding of 0, and 256 output channels. After that, a sigmoid activation function is used, and the result is multiplied element-wise with the multi-dimensional fused attention feature, finally outputting a discriminative gait feature with a dimension of 256×8×8.

[0076] S23. Input the discriminative gait features into two fully connected layers respectively to obtain the metric feature vector y1 and the classification feature vector y2. The dimension of y1 is 1024; the dimension of y2 is N, where N represents the number of sample identities in all training data.

[0077] S3. Train the gait recognition model by jointly optimizing triplet loss and cross-entropy loss;

[0078] During the model training phase, gait contour maps from existing gait datasets and skeleton maps proposed in Skeletongait are used as model training data. This invention uses triplet loss and cross-entropy loss together to optimize the model.

[0079] The triplet loss takes a triplet sample as input, consisting of a reference sample (Anchor), a positive sample belonging to the same identity as the reference sample (Positive), and a negative sample belonging to a different identity (Negative). This loss function minimizes the feature Euclidean distance between the reference sample and the positive sample, while maximizing the feature distance between the reference sample and the negative sample, so that samples of the same identity are clustered as much as possible in the feature space, and samples of different identities are separated as much as possible.

[0080] Cross-entropy loss is used to supervise the model's identity classification ability. This loss function models the classification probability of the output after feature extraction and enhances the discriminative power of the features by minimizing the cross-entropy between the predicted identity label and the true label. The network model structure of this technical solution is shown in the figure below:

[0081] The pedestrian identity classification loss L1 is calculated using Softmax Loss based on the classification feature vector, and the loss L2 is calculated using Triplet Loss on the metric feature vector y1. The total network loss function is L = L1 + L2.

[0082] The backpropagation algorithm is used to update the parameters of the entire network. The gait map and skeleton map are repeatedly input into the spatial compression enhancement module and the spatiotemporal feature extraction module for propagation until the total loss function is minimized, at which point training stops and the network training is complete.

[0083] S4. Test the trained gait recognition model.

[0084] The testing steps are as follows:

[0085] For a pedestrian profile to be tested, the size is adjusted to 1×64×44 using bilinear interpolation. At the same time, based on the key point coordinates and confidence level, the skeleton map creation method proposed in Skeletonait is used to adjust the size of the skeleton map to be tested to 2×64×44, where 1 and 2 both represent the number of channels.

[0086] The gait contour map and skeletal map to be tested are input into the trained network model to obtain discriminative gait features, thus yielding a metric feature vector y1 with a dimension of 1024. For all gait contour maps and skeletal maps in the registration set, the aforementioned steps are repeated to obtain the metric feature vector y1 for each image.

[0087] The features of the sample to be tested are compared one by one with the Euclidean distances of the image features of all samples in the registration set, and the registration set sample with the smallest distance is selected. The identity of the selected registration set sample can then be determined as the identity of the target to be tested.

[0088] The spatial compression enhancement module of this invention effectively suppresses redundant spatial information and adaptively enhances key discriminative features by introducing a LIF neuron module and channel attention mechanism from a spiking neural network into the multi-level convolutional compression process. Specifically, each convolutional layer first performs spatial dimensionality reduction on the input contour map and skeleton map features, and then the LIF neurons perform pulse encoding on the features, reducing the repetitive computation caused by continuous activation and highlighting discriminative structural changes from the perspective of dynamic response. At the same time, the channel attention module selectively enhances effective features based on the importance distribution of feature channels, further reducing unnecessary channels participating in computation, and significantly reducing the floating-point operation and inference cost of the model overall. Finally, in the fusion stage of the dual-modal branch, a softmax-gated weighted summation is used instead of concatenation and stacked convolution or cross-attention fusion, thereby achieving adaptive modality selection with minimal parameters and computational overhead.

[0089] The spatiotemporal feature extraction module of this invention effectively reduces the parameter size and computational cost of the gait recognition model through the synergistic effects of multi-level feature compression, domain-specific processing, and channel-selective enhancement. First, the dynamic window segmentation module adaptively divides the input gait sequence into spatiotemporal dimensions, reducing the time dimension. Based on this, the network employs a frequency domain processing module and a multilayer perceptron for parallel modeling. Fourier transform and frequency domain attention are used to represent some features in the frequency domain in a lower-dimensional, more periodic form, thereby reducing the need for large convolutional kernels in the spatial domain and lowering the number of parameters. Next, the multi-dimensional processing module decomposes the features into three sub-branches: high-dimensional, wide-dimensional, and channel-dimensional, each undergoing lightweight processing. This module breaks down the complex operations originally performed on a unified tensor into multiple low-computational-cost paths, reducing the computational pressure of a single path, and maintaining overall expressive power through final feature fusion.

Claims

1. A lightweight cross-view gait recognition method based on a multilayer perceptron network, characterized in that, Includes the following steps: S1. Obtain and preprocess the gait dataset, which includes gait contour maps and gait skeleton maps; S2. Construct a gait recognition model that includes a spatial compression enhancement module and a spatiotemporal feature extraction module; input the gait contour map and gait skeleton map into the spatial compression enhancement module, and generate compressed and enhanced dual-modal fusion features through multi-level convolution compression, LIF neuron encoding of spiking neural network and channel attention mechanism; The dual-modal fusion features are input into the spatiotemporal feature extraction module. Discriminative gait features are generated through dynamic window spatiotemporal segmentation, frequency domain processing, and multi-dimensional perceptron modeling. Gait recognition is then performed based on the discriminative gait features. S3. Train the gait recognition model by jointly optimizing triplet loss and cross-entropy loss; S4. Test the trained gait recognition model.

2. The lightweight cross-view gait recognition method based on a multilayer perceptron network according to claim 1, characterized in that, S2 include: S21. The gait contour map and gait skeleton map are sequentially passed through the first convolutional layer group, the first LIF neuron module, and the first channel attention module of the spatial compression enhancement module for preliminary convolutional compression, pulse recoding, and channel dimension recalibration, respectively. Then, they are sequentially passed through the second convolutional layer group, the second LIF neuron module, and the second channel attention module for further spatial compression, pulse coding, and channel recalibration. After convolutional compression through the third convolutional layer group, the contour features and skeleton features after multi-level compression enhancement are adaptively weighted and fused by the fusion gating module to generate dual-modal fusion features.

3. The lightweight cross-view gait recognition method based on a multilayer perceptron network according to claim 2, characterized in that, The implementation of the space compression enhancement module includes: S211. Input the gait contour map and gait skeleton image into two-dimensional convolutional layer a and two-dimensional convolutional layer c respectively to extract features and generate the first contour feature map and the first skeleton feature map; S212. Input the first contour feature map and the first bone feature map into LIF neuron module a and LIF neuron module c respectively for pulse recoding to generate the first pulse contour enhancement feature and the first bone contour enhancement feature; input the first contour feature map and the first bone feature map into channel attention module a and channel attention module c respectively for channel dimension recalibration, and multiply them element-wise with the first pulse contour enhancement feature and the first bone contour enhancement feature respectively to generate the first contour calibration feature and the first bone calibration feature; S213. Input the first contour calibration feature and the first bone calibration feature into the two-dimensional convolutional layer b and the two-dimensional convolutional layer d respectively for further spatial compression to generate the second contour feature map and the second bone feature map. S214. Input the second contour feature map and the second bone feature map into LIF neuron module b and LIF neuron module d respectively for pulse coding to generate the second contour pulse enhancement feature and the second bone pulse enhancement feature; input the second contour feature map and the second bone feature map into channel attention module b and channel attention module d respectively to perform channel recalibration on the second contour pulse enhancement feature and the second bone pulse enhancement feature to generate the second contour calibration feature and the second bone calibration feature. S215. Input the second contour calibration feature and the second bone calibration feature into the two-dimensional convolutional layer e and the two-dimensional convolutional layer f respectively to generate the third contour feature map and the third bone feature map; concatenate the third contour feature map and the third bone feature map along the channel dimension, and then input them into the fusion gating module for adaptive weighted fusion to obtain the dual-modal fusion feature.

4. The lightweight cross-view gait recognition method based on a multilayer perceptron network according to claim 1, characterized in that, S2 also includes: S22. The dual-modal fusion features are input into the dynamic window segmentation module of the spatiotemporal feature extraction module for dynamic window segmentation to generate window segmentation features. The window segmentation features are then processed by pooling, convolution, and upsampling layers and input into the first frequency domain processing module to obtain frequency domain enhancement features. The frequency domain enhancement features are input into the multi-branch processing module, and the results are fused to generate multi-dimensional fusion attention features. The multi-dimensional fusion attention features are then processed by the second frequency domain processing module to generate discriminative gait features.

5. A lightweight cross-view gait recognition method based on a multilayer perceptron network according to claim 1, characterized in that, S2 also includes: S23. Input the discriminative gait features into two fully connected layers respectively to obtain the metric feature vector and the classification feature vector.

6. A lightweight cross-view gait recognition method based on a multilayer perceptron network according to claim 4, characterized in that, The S22 mid-frequency domain enhanced feature input multi-branch processing module includes: The frequency domain enhancement features are fed into the high-dimensional processing module of the first branch to obtain high-dimensional reconstructed features; the frequency domain enhancement features are fed into the wide-dimensional processing module of the second branch to obtain wide-dimensional reconstructed features; the frequency domain enhancement features are fed into the channel dimension processing module of the third branch to obtain channel dimension reconstructed features; the high-dimensional reconstructed features, wide-dimensional reconstructed features and channel dimension reconstructed features are added element-wise and then input into the channel attention module e to generate multi-dimensional fused attention features.

7. A lightweight cross-view gait recognition method based on a multilayer perceptron network according to claim 6, characterized in that, The high-dimensional processing module, the wide-dimensional processing module, and the channel-dimensional processing module all contain two branches: One branch is used to flatten the extracted sub-feature maps and feed them into a multilayer perceptron containing two layers of 2D convolutions and activations; the output of the multilayer perceptron is then used for feature reshaping through depthwise separable convolutions. Another branch is used to pass the frequency domain enhanced features through Fast Fourier Transform, frequency domain attention, and Inverse Fast Fourier Transform in sequence; the outputs of the two branches are added element by element, and the processed sub-feature maps are then pieced back together as is to obtain the reconstructed features of each branch.

8. A lightweight cross-view gait recognition method based on a multilayer perceptron network according to claim 3 or 6, characterized in that, The channel attention modules a, b, c, d, and e are implemented as follows: Global max pooling and global average pooling operations are performed on the input features respectively, and the resulting two feature vectors are input into two multilayer perceptrons with the same structure. The output features of the two multilayer perceptrons are added element by element, and channel attention weights are generated by the sigmoid activation function. The channel attention weights are multiplied with the input feature map channel by channel to achieve feature recalibration at the channel level.

9. A lightweight cross-view gait recognition method based on a multilayer perceptron network according to claim 5, characterized in that, S3 include: The loss function is constructed as follows: L = L1 + L2 Where L1 is the identity classification loss calculated using Softmax Loss based on the classification feature vector, and L2 is the metric feature vector loss calculated using Triplet Loss.

Images