End-to-end gait recognition method and system based on skeleton guided differentiable silhouettes

Abstract

This invention discloses an end-to-end gait recognition method and system based on skeleton-guided differentiable silhouettes. The method first acquires pedestrian video sequences and preprocesses them to obtain a skeleton confidence map and a real silhouette. Next, a dual-branch split decoding network is constructed to generate a foreground probability map and an adaptive threshold map based on the skeleton confidence map and the real silhouette, performing linear interval calibration of the foreground probability map and adaptive alignment of the differentiable binary map. Then, a structural prior is constructed based on the skeleton confidence map, the real silhouette, and the adaptive threshold map, and the spatial structural constraint region is obtained, calculating the spatially weighted threshold loss. Finally, selective gradient propagation is performed based on the spatial structural constraint region, and joint training and gait recognition are completed based on a two-stage optimization strategy. This invention can flexibly adapt to various gait recognition models, enabling existing silhouette-based gait recognition methods to achieve significant performance improvements on large-scale multi-covariate datasets and exhibiting good robustness.

Description

Technical Field

[0001] This invention relates to the field of gait recognition technology, specifically to a gait recognition method and system that achieves end-to-end optimization through differentiable silhouette bridging, applicable to gait recognition based on video image sequences in complex scenarios. Background Technology

[0002] Gait recognition is a biometric identification technology that uses changes in human walking posture to distinguish identities. Existing gait recognition methods mostly employ a two-step approach: first, extracting intermediate representations such as silhouettes or skeletons from RGB (Red-Green-Blue) images using semantic segmentation or pose estimation algorithms; then, training a deep network based on these intermediate representations for recognition. This approach is prone to upstream segmentation errors in complex scenarios such as occlusion and insufficient lighting at night, and these errors accumulate and propagate to the downstream recognition model, significantly reducing the model's accuracy and robustness in real-world environments. To address these issues, end-to-end gait recognition solutions have emerged in recent years, integrating silhouette generation and feature extraction into a unified framework for joint optimization, mitigating the inherent error accumulation problem of the two-step method to some extent. However, this also introduces new challenges—when the recognition gradient is directly fed back to the silhouette generation process, identity-irrelevant information such as clothing textures can easily be simultaneously introduced into the silhouette, causing texture leakage. Existing end-to-end solutions have explored many ways to suppress texture, but generally do not impose reasonable structural constraints on intermediate silhouette representations: methods lacking explicit constraints are prone to introducing identity-independent textures to aid recognition under the influence of recognition gradients; methods using rigid constraints are difficult to flexibly cover complex structurally controversial regions, making it impossible to fully leverage the advantages of end-to-end joint optimization.

[0003] Differentiable binarization is a mathematical method that transforms a continuous probability map into an approximate binary map by introducing a learnable threshold map, making the binarization process differentiable. This method inherently possesses the property of suppressing texture fluctuations in continuous probability responses and has been successfully applied in fields such as scene text detection. However, in end-to-end gait recognition tasks, how to construct a reasonable learnable threshold map that effectively suppresses the transmission of texture information to downstream models while maintaining silhouette trainability remains a systematic solution in current technologies. Especially when balancing texture suppression and recognition task requirements on large-scale multivariate datasets, building a general end-to-end gait recognition framework remains a significant challenge. Summary of the Invention

[0004] The purpose of this invention is to provide a universal end-to-end gait recognition method and system to solve the problems of segmentation error accumulation and appearance texture leakage in the prior art, and to achieve flexible adaptation to various downstream gait recognition models.

[0005] To achieve the above objectives, the present invention provides an end-to-end gait recognition method based on a skeleton-guided differentiable silhouette bridge, comprising the following steps:

[0006] Step 1: Obtain pedestrian video sequences and preprocess them to obtain skeleton confidence maps and real silhouettes.

[0007] Preferably, for the input pedestrian video sequence, a pre-trained human pose estimation algorithm is used to extract human key points and their skeletal topological connections from each frame of the image, generating a dual-channel skeleton map containing joint channels and a single-channel skeleton confidence map, and then synthesizing the single-channel skeleton confidence map by taking the maximum value of the dual-channel skeleton map; at the same time, the foreground mask corresponding to each frame is obtained as the real silhouette using a pre-trained segmentation algorithm or labeled data; the skeleton map, the real silhouette and the corresponding pedestrian video sequence are geometrically aligned and sized in the spatial dimension, and used as the input prior and structural supervision signal for subsequent feature extraction.

[0008] Step 2: Construct a dual-branch separation decoding network to generate a foreground probability map and an adaptive threshold map based on the skeleton confidence map and the real silhouette.

[0009] Preferably, for the input pedestrian video sequence, the encoder part of the U-Net segmentation network (a common lightweight convolutional neural network for image segmentation) is used as a shared encoder to extract frame-level feature maps for each frame. These frame-level feature maps are then input into independent foreground decoders and threshold decoders. The foreground decoder uses the real silhouette obtained in step 1 as a supervision signal to generate a foreground probability map for providing basic human shape estimation. The threshold decoder constructs a threshold supervision target using the real silhouette and skeleton confidence map, generating an adaptive threshold map for adjusting the binarization boundary. The dual-branch independent decoder structure here is consistent with the U-Net segmentation decoder; the separate architecture is only used to decouple foreground estimation and adaptive threshold prediction, preventing gradient interference introduced by shared decoding parameters.

[0010] Step 3: Based on the foreground probability map and the adaptive threshold map, perform linear interval calibration of the foreground probability map and adaptive alignment of the differentiable binary map.

[0011] Preferably, the foreground probability map is first calibrated linearly according to preset upper and lower bound parameters to obtain a calibrated foreground probability map with compressed difference range. The difference between the calibrated foreground probability map and the adaptive threshold map is then subjected to differentiable binarization operation to generate an approximate binary silhouette.

[0012] For example, differentiable binarization operations use the Sigmoid function with a steepness factor:

[0013]

[0014] The sigmoid function is a mapping function that monotonically maps the input to the interval (0,1). As a steepness control factor, To calibrate the foreground probability map, This is an adaptive threshold image. Subsequently, the minimum bounding rectangle of the foreground pixel distribution of the human body is extracted based on this approximate binary silhouette. Before region sampling, a reliability quality assessment of the human region is performed: the number of valid image rows in the foreground pixel distribution and the proportion of the foreground box height to the total image height are counted. When all the above image metrics meet the preset reliability threshold, the foreground box is resampled and aligned to match the input requirements of the downstream gait recognition model. When any proportion falls below the preset reliability threshold, a conservative degradation fallback mechanism is triggered, resetting the cropped approximate binary silhouette using preset fixed upper and lower boundary ranges to avoid over-cropping caused by sample quality.

[0015] Step 4: Construct a structural prior based on the skeleton confidence map, the real silhouette, and the adaptive threshold map, obtain the spatial structural constraint region, and calculate the spatial weighted threshold loss.

[0016] Preferably, this step uses the skeleton confidence map obtained from the preprocessing in step 1, the real silhouette, and the adaptive threshold map generated in step 2 as inputs to construct a structural prior and constrain the learning of the threshold decoder:

[0017] Based on the high-confidence region of the skeleton determined by the real silhouette and the preset skeleton confidence threshold, the union of the real silhouette and the high-confidence region of the skeleton is taken as the spatial structure constraint region. This region is used to limit the spatial range in the threshold map that allows the receiving of recognition gradient updates.

[0018] A structural supervision weight map is generated by combining the real silhouette, skeleton confidence map, and spatial structural constraint regions. Larger weights are assigned to locations where the real silhouette and skeleton confidence map are highly consistent, while smaller weights are assigned to structurally controversial regions where they are inconsistent. This weight map is then used to perform position-weighted calculations on the loss function of the adaptive threshold map, resulting in a spatially weighted threshold loss.

[0019] Step 5: Selectively identify gradient propagation based on spatial structural constraints.

[0020] Preferably, the selective recognition gradient propagation includes two parts of gradient truncation operation: First, gradient truncation is applied to the calibrated foreground probability map to block the downstream recognition gradient from being transmitted back to the shared encoder and foreground decoder via the foreground decoder, so that the basic foreground representation is not interfered with by the identity supervision signal; Second, in the gradient transmission path of the threshold decoder, only the downstream recognition gradient is allowed to enter within the spatial structure constraint region constructed in step 4, and the gradient outside this region is truncated to avoid background information contaminating the threshold decoder.

[0021] After the selective gradient propagation described above, the recognition loss gradient only updates the parameters of the threshold decoder within the spatial structural constraint region, thereby shifting the differentiable binarized decision boundary. This causes a controlled adjustment of the local contour shape of the near-binary silhouette along the direction of the decreasing recognition loss. This adjustment only occurs within the structural constraint region, allowing the recognition task to correct the binarization boundary. This enables the near-binary silhouette to gradually move towards a region that enhances effective identity discrimination while maintaining texture suppression, achieving adaptive alignment between representation and recognition tasks.

[0022] Step 6: After selectively identifying gradient propagation, complete joint training and gait recognition based on a two-stage optimization strategy.

[0023] Preferably, the network training employs a two-stage optimization strategy: The first stage is the basic representation learning stage, which simultaneously optimizes the shared encoder, foreground decoder, and threshold decoder using segmentation loss and spatially weighted threshold loss. In this stage, the recognition gradient pointing to the silhouette generation end is truncated to ensure a stable basic foreground representation is formed without interference from downstream models. The second stage is the joint optimization stage, which freezes the shared encoder and foreground decoder, keeping the foreground representation base fixed. Only the downstream gait recognition model and threshold decoder are jointly optimized, and the recognition gradient updates the threshold decoder only within the spatial structural constraint region, achieving adaptive adaptation of the silhouette representation to the recognition task.

[0024] After model training, the inference phase begins: This phase only requires a pedestrian video sequence as input. The network adaptively generates a foreground probability map and an adaptive threshold map, which are then binarized and adaptively aligned to obtain an approximate binary silhouette, which is then input into the trained downstream gait recognition model. The recognition model extracts the gait embedding vector of the query sample, calculates the distance between it and each embedding vector in the candidate sample library, sorts them, and outputs the identity corresponding to the candidate sample with the smallest distance as the recognition result, ultimately completing identity recognition.

[0025] In another aspect, the present invention also provides an end-to-end gait recognition system based on skeleton-guided differential silhouettes, comprising the following modules:

[0026] The pedestrian video preprocessing module is used to acquire pedestrian video sequences and preprocess them to obtain skeleton confidence maps and real silhouettes.

[0027] The separation decoding module is used to construct a dual-branch separation decoding network, which generates a foreground probability map and an adaptive threshold map based on the skeleton confidence map and the real silhouette.

[0028] The calibration and alignment module performs linear interval calibration of the foreground probability map and adaptive alignment of the differentiable binary map based on the foreground probability map and the adaptive threshold map.

[0029] The spatial weighted threshold loss module constructs structural priors based on skeleton confidence maps, real silhouettes, and adaptive threshold maps, obtains spatial structural constraint regions, and calculates spatial weighted threshold loss.

[0030] The gait recognition module selectively identifies gradient propagation based on spatial structural constraints, and then completes joint training and gait recognition through a two-stage optimization strategy.

[0031] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: By constructing a skeleton-guided differentiable silhouette bridge, this invention integrates silhouette generation and gait recognition into a unified end-to-end framework, effectively alleviating the problem of segmentation error accumulation propagating to downstream recognition models in two-step methods. Compared with existing end-to-end methods, this invention combines a differentiable binarization mechanism with structural priors, precisely limiting the recognition-driven adaptive adjustment within the structural constraint region. This leverages the advantages of end-to-end joint optimization while avoiding erroneous enhancement of the background region, achieving a unity between explicit morphological preservation and recognition task adaptation. It can more flexibly handle structural changes in complex scenarios such as clothing changes and occlusions. Experiments demonstrate that this invention can flexibly adapt to various downstream gait recognition models, enabling existing silhouette-based gait recognition methods to achieve significant performance improvements on large-scale multi-covariate datasets, demonstrating good robustness and practical application value. Starting from skeleton-guided differentiable silhouette generation, this invention provides a new technical approach for end-to-end gait recognition that balances accuracy and generalization ability. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the overall framework of the end-to-end gait recognition method based on skeleton guidance and differentiable binarization collaboration provided in the embodiments of the present invention;

[0033] Figure 2 This is a visualization diagram of heatmaps of the region of interest of the model with different end-to-end configurations under the same model, provided in an embodiment of the present invention.

[0034] Figure 3 This is a visual comparison of intermediate characterizations under different model configurations and ablation experiments provided in the embodiments of the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Combination Figures 1 to 3As shown, in order to effectively suppress the leakage of identity-irrelevant information such as clothing texture while maintaining the explicit shape of the silhouette, and to achieve upstream and downstream collaborative optimization of silhouette generation and gait recognition, this invention constructs a unified end-to-end network structure including a shared encoder, a foreground decoder, a threshold decoder, differentiable binarization operation, a skeleton-guided structure prior, and selective recognition gradient propagation. That is, this invention provides an end-to-end gait recognition method based on a skeleton-guided differentiable silhouette bridge, including the following steps:

[0037] Step 1: Obtain pedestrian video sequences and preprocess them to obtain skeleton confidence maps and real silhouettes.

[0038] Specifically, the process of generating the skeleton confidence map includes: extracting the coordinates and confidence scores of human key points in each frame image using a pre-trained human pose estimation algorithm; then generating a joint heatmap corresponding to each key point using the coordinates of the key point as the center and the confidence score as the peak value through a two-dimensional Gaussian distribution; and generating a corresponding skeleton heatmap by calculating the projection distance from pixels to each bone segment according to the preset human skeleton connection relationship; and taking the maximum value along the channel dimension of the generated joint heatmap and skeleton heatmap to obtain the single-channel skeleton confidence map. Simultaneously, a pre-trained segmentation algorithm is used to obtain the foreground mask corresponding to each frame as the real silhouette. The skeleton confidence map, the real silhouette, and the corresponding RGB image sequence are geometrically aligned and sized in the spatial dimension, serving as the input prior and structural supervision signal for subsequent feature extraction.

[0039] Step 2: Construct a dual-branch separation decoding network to generate a foreground probability map and an adaptive threshold map based on the skeleton confidence map and the real silhouette.

[0040] Subsequently, the encoder portion of the lightweight segmentation network U-Net was used as a shared encoder E for each frame of the image. Feature extraction is performed to obtain frame-level feature maps. .Will Input to independent foreground decoders and threshold decoder Predict the foreground probability map respectively With adaptive threshold map Both the foreground decoder and threshold decoder adopt a lightweight structure consistent with the U-Net decoding branch, consisting of progressively upsampling layers and skip connections. After each upsampling stage, the data undergoes 3×3 convolution and batch normalization, ultimately outputting a single-channel probability map or an adaptive threshold map via a 1×1 convolution. The foreground probability map uses the real silhouette as the supervision signal and employs L1 loss for supervision. The adaptive threshold map uses the skeleton confidence map. The threshold supervision target constructed together with the real silhouette is the supervision signal. Spatial weighted threshold loss is used for supervision. The specific supervision and loss forms are shown in steps 4 and 6.

[0041] Step 3: Based on the foreground probability map and the adaptive threshold map, perform linear interval calibration of the foreground probability map and adaptive alignment of the differentiable binary map.

[0042] Linear interval calibration: foreground probability map A linear mapping is performed from the original interval to a preset calibration interval to limit the difference between the foreground probability and the threshold, preventing the differentiable binarization function from entering the saturation region due to excessive difference, thereby weakening the effective gradient propagation, and thus obtaining a calibrated foreground probability map. The calculation process is expressed as follows:

[0043]

[0044] in, and These represent the lower and upper limits of the calibration interval, respectively. In this embodiment, take , .

[0045] Differentiable binarization: calibrating the foreground probability map With adaptive threshold map Calculate the difference pixel by pixel to obtain the foreground-threshold difference map. Then, through mapping with a sigmoid function and a steepness factor, an approximate binary silhouette is generated. The calculation process is as follows:

[0046]

[0047] in, This is the binarization steepness factor, which controls the intensity of binarization. When it is a positive value A value approaching 1 corresponds to the foreground region; a negative value approaching 0 corresponds to the background region. In this embodiment, the binarization steepness factor... During training, the value is linearly increased from 30 to 50, taking into account early adaptation by smoothing first and then sharpening.

[0048] Adaptive alignment: This embodiment of the invention introduces a reliability-aware backoff mechanism, first starting from an approximate binary silhouette. The minimum bounding rectangle of the human foreground pixel distribution is extracted: pixels with values ​​not less than a threshold of 0.5 are identified as foreground pixels, and pixels with values ​​less than the threshold are identified as background pixels, resulting in a binary foreground mask; image rows containing at least two foreground pixels are identified as valid rows, and the proportion of the image height covered by the valid rows and the proportion of the foreground bounding box height to the image height are calculated respectively: when the proportion of valid rows reaches at least 30% of the image height and the height of the foreground bounding box reaches at least 60% of the image height, the estimated body region is considered reliable, and the human body bounding box is scaled to the uniform input size of the downstream gait recognition model. When any of the above ratios falls below the corresponding threshold, the estimated body region is deemed unreliable, and a preset safe cropping ratio will be used for scaling. This fallback mechanism aims to avoid over-cropping and maintain the consistent relative positions of different body parts in the image, preventing spatial misalignment from disrupting the part distribution patterns upon which the gait recognition model relies.

[0049] Step 4: Construct a structural prior based on the skeleton confidence map, the real silhouette, and the adaptive threshold map, obtain the spatial structural constraint region, and calculate the spatial weighted threshold loss.

[0050] Skeleton Prior Processing: Skeleton Confidence Map A larger value indicates a higher confidence level in the pose at the corresponding location.

[0051] Threshold-supervised target construction: using the skeleton confidence map obtained from the preprocessing in step 1. With real silhouette Constructing threshold supervision targets The calculation process is expressed as follows:

[0052]

[0053] in, This represents the background filtering threshold assigned to the background region. Within the true silhouette, the skeleton confidence is inversely mapped to the interval. The higher the skeleton confidence, the lower the threshold, and the easier it is for the corresponding body region to pass the binarization decision; outside the real silhouette, a higher background filtering threshold is used. .

[0054] Spatial structural constraint region construction: based on skeleton confidence map With real silhouette Constructing spatial structural constraint regions The calculation process is expressed as follows:

[0055]

[0056] in This indicates an element-wise logical OR. This represents the skeleton confidence threshold. The update also includes blurred areas where the silhouette is missed but the skeleton is strongly activated. In this embodiment, we take... .

[0057] Construction of the Structure-Supervised Weight Graph: Weight Graph This is used to perform position-by-position local weighting on the spatially weighted threshold loss, where To balance the coefficients, determine the realistic silhouette. With skeleton confidence map The relative proportion in weight estimation This indicates element-wise multiplication. By adjusting the local intensity of threshold supervision, the relative roles of recognition supervision and threshold supervision are indirectly controlled: at locations with high skeleton confidence, the structural prior is more reliable, and the intervention of the recognition gradient should be reduced; therefore, in this embodiment, we take... The calculation process of applying the weighted graph to the threshold loss is expressed as follows:

[0058]

[0059] in, Let (u,v) represent the smoothed L1 loss, where i represents the frame number in the sequence; and (u,v) represents the pixel spatial coordinates within a single frame of the image. Represents the structural supervision weight graph of the i-th frame. Similarly, the values ​​at spatial locations (u, v) are determined by... Let S, H, and W represent the values ​​of the adaptive threshold map and the threshold-supervised target at spatial location (u,v) in the i-th frame, respectively; and let S, H, and W represent the number of frames in the sequence and the height and width of a single frame image, respectively. This represents the total number of pixel spatial locations in the sequence.

[0060] Step 5: Selectively identify gradient propagation based on spatial structural constraints.

[0061] In the joint optimization phase, selective gradient propagation involves two gradient truncation operations: first, calibrating the foreground probability map... Gradient separation is performed to block the downstream recognition gradient from being transmitted back to the shared encoder and foreground decoder via the foreground decoder, ensuring that the basic foreground representation is not interfered with by the identity supervision signal. Secondly, on the gradient transmission path of the threshold decoder, the spatial structure constrains the region. Gradient separation threshold map used in constructing the recognition path The construction method is as follows: The location retains the gradient connectivity between the location threshold and the recognition loss, i.e. ;exist The gradient separation operator is applied to the position threshold, i.e. The above construction does not change the numerical value and shape of the approximate binary silhouette, but only implements pixel-by-pixel spatial gating at the gradient level—allowing the recognition gradient to update the threshold within the structurally constrained region. Based on this, a position-by-position weighting of the threshold supervision is achieved through a spatially weighted threshold loss, forming a collaborative adjustment mechanism: the spatial structural constraint region determines "where the recognition gradient can update the threshold," while the spatially weighted threshold loss adjusts "the relative strength of threshold supervision and recognition supervision at each position" through the structural supervision weight map. Thus, while maintaining the explicit silhouette shape, local adaptive adjustment of the recognition-driven cropping generation end is achieved.

[0062] Step 6: After selectively identifying gradient propagation, complete joint training and gait recognition based on a two-stage optimization strategy.

[0063] The total loss during training is: ,in, The recognition loss is composed of triplet loss and cross-entropy loss. The segmentation loss is represented by L1 loss, which is used to calculate the absolute deviation of the foreground probability map from the real silhouette pixel by pixel; alternatively, other pixel-by-pixel loss forms such as binary cross-entropy loss can also be used. This represents the threshold loss, the specific form of which has been given in step 4; This represents the threshold loss weighting coefficient.

[0064] The first stage is basic representation learning: simultaneously optimizing the shared encoder using segmentation loss and weighted threshold loss. Foreground Decoder and threshold decoder This stage truncates the recognition gradient pointing to the silhouette generation end, causing the downstream recognition model to first adapt to the generated approximate binary silhouette. This avoids identity supervision distorting early feature extraction.

[0065] Phase 2 Joint Optimization: Freezing the Shared Encoder With foreground decoder Keeping the foreground representation basis fixed, the downstream gait recognition model is combined with the threshold decoder. Joint optimization is employed. The gait recognition model takes a generated approximate binary silhouette as input, maps the silhouette sequence to a gait embedding vector via a feature extraction network, and performs identity discrimination training using triplet loss and cross-entropy loss. Its recognition loss gradient only exists within the spatially constrained region. The threshold decoder is updated to enable silhouette generation and gait recognition to be optimized collaboratively within a unified framework, achieving adaptive adaptation of silhouette representation to the recognition task.

[0066] Inference phase: Only a pedestrian RGB video sequence needs to be input. The network will adaptively generate a foreground probability map and an adaptive threshold map, which will be differentiable and binarized to generate an approximate binary silhouette. After alignment, the silhouette is input into the gait recognition model. The recognition model will extract the gait embedding vector of the query sample, calculate the distance between it and each embedding vector in the candidate sample library, sort them, and take the identity corresponding to the candidate sample with the smallest distance as the recognition result output, thus completing the identity recognition.

[0067] In another aspect, the present invention provides an end-to-end gait recognition system based on skeleton-guided micro-silhouettes, for implementing the aforementioned end-to-end gait recognition method, including a pedestrian video preprocessing module, a separation decoding module, and a calibration and alignment module.

[0068] The pedestrian video preprocessing module is used to acquire pedestrian video sequences, preprocess them to obtain skeleton confidence maps and real silhouettes, a spatial weighted threshold loss module, and a gait recognition module.

[0069] The separation decoding module is used to construct a dual-branch separation decoding network, which generates a foreground probability map and an adaptive threshold map based on the skeleton confidence map and the real silhouette.

[0070] The calibration and alignment module performs linear interval calibration of the foreground probability map and adaptive alignment of the differentiable binary map based on the foreground probability map and the adaptive threshold map.

[0071] The spatial weighted threshold loss module constructs a structural prior based on the skeleton confidence map, the real silhouette, and the adaptive threshold map, and obtains the spatial structural constraint region to calculate the spatial weighted threshold loss.

[0072] The gait recognition module selectively identifies gradient propagation based on spatial structural constraints, and then completes joint training and gait recognition through a two-stage optimization strategy.

[0073] The effectiveness of this invention is demonstrated through experimental examples below. This embodiment conducted systematic experiments on two publicly available gait datasets, CASIA-B* and SUSTech1K. The results are illustrated in the diagram below. Figure 2 and Figure 3As shown in Tables 1-2, CASIA-B* is a classic indoor gait dataset constructed by the Institute of Automation, Chinese Academy of Sciences; SUSTech1K is an outdoor gait dataset constructed by the Southern University of Science and Technology, characterized by multiple covariate scenarios. To evaluate the generality of this invention, Tables 1-2 use various existing silhouette-based and multimodal gait recognition models as downstream recognition models: the silhouette-based single-modal models GaitSet, GaitPart, and GaitGL are early classic gait recognition methods (achieving cross-view recognition through set aggregation, part-level temporal modeling, and global-local feature extraction, respectively); GaitBase, DeepGaitV2, DyGait, and GLGait are recent strong baseline models with high recognition accuracy on standard datasets; multimodal models include XGait, SkeletonGait++, and MultiGait++, which improve recognition performance by fusing auxiliary modal information such as skeleton, human body analysis, and optical flow. For multimodal models, this invention only replaces the silhouette input with the generated near-binary silhouette, retaining its auxiliary modal branches unchanged, to evaluate the gain of the differentiable silhouette bridge on multimodal recognition. The higher the numerical value of the result, the higher the accuracy of gait recognition. As can be seen from the table, the present invention utilizes a skeleton-guided differentiable silhouette bridge, which can bring performance improvement to various silhouette types and multimodal gait recognition models, effectively improving the adaptability of intermediate modalities to gait tasks.

[0074] To demonstrate the effectiveness of each module, Tables 3-5 show ablation experiments conducted based on the multimodal gait recognition model designed in this invention. Table 3 explores the role of the recognition gradient propagation target and the structural constraint region, verifying the design of selectively routing the recognition gradient to the threshold decoder and limiting it to the structurally relevant region. Table 4 verifies the effectiveness of the "smoothing before sharpening" steepness factor scheduling strategy of this invention. Table 5 verifies the role of the linear calibration range of the foreground probability map, indicating that the calibration mechanism of this invention, which limits the difference between the foreground probability and the threshold to avoid premature saturation of differentiable binarization, is effective.

[0075] Table 1. Comparison of accuracy results of various recognition methods on the SUSTech1K gait dataset.

[0076]

[0077] Table 2. Comparison of accuracy results of various recognition methods on the CASIA-B* gait dataset.

[0078]

[0079] Table 3 Ablation experiments identifying gradient targets and structural constraints (using GaitSet as the downstream identification model)

[0080]

[0081] As can be seen from Table 3, the performance is lowest when the recognition gradient is directly applied to the foreground representation (traditional end-to-end); the performance is best when the recognition gradient is only directed to the threshold branch and the gradient is restricted within the structural constraint region, supplemented by spatially weighted threshold loss; while full-graph propagation or simultaneous updating of probability and threshold branch both significantly degrade.

[0082] Table 4. The impact of binarization steepness factor k on the performance of SUSTech1K (using GaitSet as the downstream recognition model).

[0083]

[0084] Table 4 verifies the effectiveness of the binarization steepness factor scheduling strategy. Larger k values ​​(k=50, 70) lead to performance degradation. The accompanying figures also show the visualization when the k value is small (k=10). While this achieves higher performance, it introduces more background texture information, contradicting the design intent of this invention to suppress texture information leakage. In contrast, the linearly increasing scheduling of "smoothing first, then sharpening" gradually forms a near-binary silhouette while maintaining early adaptability, balancing texture information suppression with task suitability.

[0085] Table 5. Impact of foreground calibration range on SUSTech1K performance (using GaitSet as the downstream recognition model)

[0086]

[0087] As shown in Table 5, the overall performance is lowest when the probability map is not calibrated. Reducing the mapping range of the probability map can improve performance, while the calibration range is constrained to... It performs best at that time.

Claims

1. An end-to-end gait recognition method based on skeleton-guided differentiable silhouettes, characterized in that, Includes the following steps: Step 1: Obtain pedestrian video sequences and preprocess them to obtain skeleton confidence maps and real silhouettes; Step 2: Construct a dual-branch separation decoding network to generate a foreground probability map and an adaptive threshold map based on the skeleton confidence map and the real silhouette; Step 3: Based on the foreground probability map and the adaptive threshold map, perform linear interval calibration of the foreground probability map and adaptive alignment of the differentiable binary map; Step 4: Construct a structural prior based on the skeleton confidence map, the real silhouette, and the adaptive threshold map, obtain the spatial structural constraint region, and calculate the spatial weighted threshold loss; Step 5: Selectively identify gradient propagation based on spatial structural constraints; Step 6: After selectively identifying gradient propagation, complete joint training and gait recognition based on a two-stage optimization strategy.

2. The end-to-end gait recognition method based on a skeleton-guided, differentiable silhouette bridge according to claim 1, characterized in that, Step 1 specifically includes: for the input pedestrian video sequence, using a pre-trained human pose estimation algorithm to extract human key points and their skeletal topological connections from each frame of the image, generating a dual-channel skeleton map containing joint channels and bone channels, and synthesizing a single-channel skeleton confidence map by taking the maximum value of the dual-channel skeleton map; simultaneously obtaining the foreground mask corresponding to each frame as the real silhouette; and performing geometric alignment and size normalization processing on the skeleton confidence map, the real silhouette, and the corresponding pedestrian video sequence in the spatial dimension.

3. The end-to-end gait recognition method based on a skeleton-guided, differentiable silhouette bridge according to claim 2, characterized in that, Step 2 specifically includes: for the input pedestrian video sequence, using the encoder part of the segmentation network as a shared encoder, extracting frame-level feature maps of each frame image; then inputting the frame-level feature maps into independent foreground decoders and threshold decoders respectively; The foreground decoder uses the real silhouette as a supervision signal to generate a foreground probability map for providing basic human shape estimation; the threshold decoder constructs a threshold supervision target using the real silhouette and skeleton confidence map to generate an adaptive threshold map for adjusting the binarization boundary; the segmentation loss is calculated using the generated foreground probability map and the adaptive threshold map.

4. The end-to-end gait recognition method based on skeleton-guided differentiable silhouette bridge according to claim 3, characterized in that, Step 3 specifically includes: First, the foreground probability map is linearly calibrated according to the preset upper and lower bound parameters to obtain the calibrated foreground probability map after compressing the difference range. Then, the difference between the calibrated foreground probability map and the adaptive threshold map is subjected to differentiable binarization operation to generate an approximate binary silhouette. The differentiable binarization operation is implemented using the Sigmoid function with a steepness factor; Subsequently, the minimum bounding rectangle of the human foreground pixel distribution is extracted based on the approximate binary silhouette, and the number of effective image rows of the foreground pixel distribution and the proportion of the foreground box height to the total image height are counted. When the number of effective image rows and the proportion of the foreground box height to the total image height both meet the preset reliability threshold, the foreground box is resampled and aligned to match the input requirements of the downstream gait recognition model. When any proportion falls below a preset reliability threshold, a conservative degradation fallback mechanism is triggered, resetting the approximate binary silhouette generated by cropping using a preset fixed upper and lower boundary range.

5. The end-to-end gait recognition method based on a skeleton-guided, differentiable silhouette bridge according to claim 4, characterized in that, Step 4 specifically includes: Using the preprocessed skeleton confidence map, the true silhouette, and the adaptive threshold map as input, a structural prior is constructed to constrain the learning of the threshold decoder: Based on the high-confidence region of the skeleton determined by the real silhouette and the preset skeleton confidence threshold, the union of the real silhouette and the high-confidence region of the skeleton is taken as the spatial structure constraint region. The weighted sum of the real silhouette and the skeleton confidence map is then multiplied element-wise with the spatial structure constraint region to generate a structural supervision weight map. Using skeleton confidence maps With real silhouette Constructing threshold supervision targets : in, This represents the background filtering threshold assigned to the background region; the adaptive threshold map is compared with the threshold supervision target. The loss function for the adaptive threshold map is calculated by smoothing the L1 loss. The spatially weighted threshold loss is obtained by weighting the loss function of the adaptive threshold map by position using the structure-supervised weighted graph.

6. The end-to-end gait recognition method based on a skeleton-guided differentiable silhouette bridge according to claim 5, characterized in that, Step 5 specifically includes: Selective recognition gradient propagation includes two parts of gradient truncation operation: first, applying gradient truncation to the calibrated foreground probability map; second, in the gradient backpropagation path of the threshold decoder, only downstream recognition gradients are allowed to enter within the spatial structure constraint region, and gradients outside this region are truncated. After selective recognition gradient propagation, the recognition loss gradient only updates the parameters of the threshold decoder within the spatial structure constraint region, moves the differentiable binarized decision boundary, and causes the local contour shape of the approximate binary silhouette to undergo controlled adjustment along the direction of recognition loss descent. This adjustment only occurs within the spatial structure constraint region, allowing the recognition task to correct the binarization boundary within the spatial structure constraint region.

7. The end-to-end gait recognition method based on a skeleton-guided differentiable silhouette bridge according to claim 6, characterized in that, Step 6 specifically includes: The training adopts a two-stage optimization strategy: the first stage is the basic representation learning stage, which uses segmentation loss and spatial weighted threshold loss to simultaneously optimize the shared encoder, foreground decoder and threshold decoder; the second stage is the joint optimization stage, which freezes the shared encoder and foreground decoder and only jointly optimizes the downstream gait recognition model and threshold decoder, and the recognition gradient only updates the threshold decoder within the spatial structure constraint region. After training, the inference phase begins: This phase only requires inputting a pedestrian video sequence, which adaptively generates a foreground probability map and an adaptive threshold map. After differentiable binarization and adaptive alignment, an approximate binary silhouette is obtained and input into the trained downstream gait recognition model. The recognition model extracts the gait embedding vector of the query sample, calculates the distance between it and each embedding vector in the candidate sample library, sorts them, and takes the identity corresponding to the candidate sample with the smallest distance as the recognition result output, thus completing the identity recognition.

8. An end-to-end gait recognition system based on skeleton-guided differential silhouette, used to implement the end-to-end gait recognition method according to any one of claims 1 to 7, characterized in that, Includes the following modules: The pedestrian video preprocessing module is used to acquire pedestrian video sequences and preprocess them to obtain skeleton confidence maps and real silhouettes; The separation decoding module is used to construct a dual-branch separation decoding network, which generates a foreground probability map and an adaptive threshold map based on the skeleton confidence map and the real silhouette; The calibration and alignment module performs linear interval calibration of the foreground probability map and adaptive alignment of the differentiable binary map based on the foreground probability map and the adaptive threshold map. The spatial weighted threshold loss module constructs structural priors based on skeleton confidence maps, real silhouettes, and adaptive threshold maps, obtains spatial structural constraint regions, and calculates spatial weighted threshold loss. The gait recognition module selectively identifies gradient propagation based on spatial structural constraints, and then completes joint training and gait recognition through a two-stage optimization strategy.

Images