A semantic guidance and gait enhancement clothes-changing pedestrian re-identification method, storage medium and equipment

Abstract

This application provides a semantically guided and gait-enhanced pedestrian re-identification method, storage medium, and device, relating to the field of computer vision technology. The method includes constructing a multimodal three-stream framework comprising a pedestrian re-identification stream, a semantic information enhancement stream, and a gait assistance stream to robustly extract pedestrian identity features in complex scenarios such as clothing changes, posture changes, and viewpoint changes. The model includes a feature extraction network, a dynamic adaptive multi-head attention module, and a semantic consistency constraint module. By jointly learning semantic features, gait features, and identity features, the robustness of the model in clothing change scenarios is improved. This method can effectively improve the accuracy of pedestrian re-identification, has strong generalization ability, and can be widely applied in fields such as video surveillance, intelligent security, and urban management.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a semantically guided and gait-enhanced method, storage medium, and device for re-identifying pedestrians changing clothes. Background Technology

[0002] Person re-identification (ReID) aims to match the same pedestrian under different camera perspectives and temporal conditions. Most existing ReID methods rely on visual appearance features, but in real-world scenes, pedestrians frequently change clothing, exhibit significant differences in posture and movement, and have complex perspectives, making appearance-based feature extraction inherently limited. Currently available datasets such as PRCC, LTCC, and VC-Clothes address clothing variation, but still suffer from the following shortcomings: 1) Visual features are significantly affected by clothing, making it difficult to reliably extract key identity features; 2) Existing semantic parsing is unstable and prone to missegmentation under occlusion or lighting changes; 3) While gait features are stable, they are insufficient to cover identity details when used alone; 4) There is a lack of a unified multimodal framework that can simultaneously integrate semantic, gait, and visual information. Therefore, developing a person re-identification method that maintains high robustness under clothing variation conditions and possesses multimodal fusion capabilities is of great significance. Summary of the Invention

[0003] The purpose of this invention is to address the problems of insufficient multimodal information fusion and weak cross-modal feature alignment in existing pedestrian re-identification methods for changing clothes, which result in low recognition accuracy and poor generalization ability in complex scenarios such as clothing changes and varied postures. This invention provides a semantically guided and gait-enhanced pedestrian re-identification method for changing clothes.

[0004] The above-mentioned objective of this application is achieved through the following technical solution: S1. Construct a dataset based on the acquired pedestrian images to be identified; S2. Construct a multimodal three-flow framework for pedestrian re-identification of clothing changes. The pedestrian re-identification model of clothing changes includes: a feature extraction network for extracting multi-scale visual features, a dynamic adaptive multi-head attention module for dynamically suppressing clothing interference and enhancing key identity features, and a semantic consistency constraint module for cross-modal alignment of semantic features and gait features. S3. Train the pedestrian re-identification model using the dataset; during training, jointly optimize the gait-assisted loss function, the semantic information enhancement loss function, and the semantic consistency constraint loss function. S4. Input the pedestrian image to be tested into the trained pedestrian re-identification model after changing clothes, and output the pedestrian re-identification result.

[0005] Optionally, step S2 includes: The feature extraction network is connected to a dynamically adaptive multi-head attention module; the dynamically adaptive multi-head attention module is connected to a semantic consistency constraint module. The feature extraction network includes: a backbone network and multiple branch modules. The backbone network adopts a ResNet-50 or OSNet structure and is used to extract identity-related basic features. The branch module is used to extract multi-scale feature maps from different stages of the backbone network, including feature maps with scales of 64×64, 32×32, 16×16 and 8×8.

[0006] Optionally, step S2 may further include: The dynamic adaptive multi-head attention module includes an adaptive parameter modulation unit and a multi-head self-attention fusion unit; The processing procedure of the adaptive parameter modulation unit is as follows: Multi-scale feature maps include: low-scale feature maps and high-scale feature maps; The low-scale feature maps are input into the branches used to generate Value and Key, respectively, and then passed through 1×1 convolution, dilated spatial pyramid pooling module and view transformation operation to obtain linear matrices γ and θ. The high-scale feature map is input into the branch used to generate the query, and then passed through 1×1 convolution and view transformation operations to obtain a linear matrix φ. Calculate the dot product of φ and θ, and obtain the spatial location feature attention weights after Softmax normalization; The attention weights are multiplied by the linear matrix γ, and a linear transformation is performed to output the globally associated feature map. The adaptive parameter modulation unit dynamically generates scaling factors and biases based on input features to adjust attention weights; The semantically enhanced feature map and gait feature map are obtained through scaling factors, biases, and multi-head self-attention fusion units, specifically including: Through the semantic information enhancement task, pedestrian images are processed by a semantic parser to generate semantic mask images. The semantic mask images are then fused with multi-scale feature maps to obtain semantically enhanced feature maps. The gait-assisted task inputs multi-scale feature maps into the gait sequence prediction module to generate gait feature maps.

[0007] Optionally, step S2 may further include: The semantic consistency constraint module includes: a semantic consistency loss function and a gait sequence prediction module; The input to the semantic consistency constraint module includes a semantic enhancement feature map and a gait feature map, and its processing procedure is as follows: The semantic enhancement feature map and the gait feature map are added together to obtain the fused feature map. The local attention weight and the global attention weight are calculated through the local feature branch and the global feature branch, respectively. The local feature branch is composed of two convolutional modules, two batch normalization modules, a ReLU activation function and a Sigmoid function concatenated together. The global feature branch consists of a pooling module, two convolutional modules, two batch normalization modules, a ReLU activation function, and a Sigmoid function concatenated together. The result of multiplying the semantically enhanced feature map with the global attention weights is added to the result of multiplying the gait feature map with the local attention weights to obtain the aligned output features.

[0008] Optionally, step S3 includes: The gait-assisted loss function We use a weighted binary cross-entropy loss, the expression of which is:

[0009] in, This represents the gait predictions made by the model, i.e., the gait feature map; Indicates the true gait label, These are the weighting coefficients. This represents the binary cross-entropy loss function.

[0010] Optionally, step S4 includes: The semantic information enhancement loss function Combining cross-entropy loss and KL divergence, its expression is:

[0011] in, Represents the actual semantic segmentation labels, This represents the semantic probability predicted by the model. Represents cross-entropy loss, Denotes KL divergence, These are the balancing weighting coefficients.

[0012] Optionally, step S4 includes: The semantic consistency constraint loss function The expression used to constrain the consistency between semantic features and gait features is: in, The loss function represents the semantic consistency constraint task. This represents semantic enhancement features. Indicates gait characteristics, Indicates fusion features, Indicates a real identity tag. It is a threshold. Indicates the number of pixels or features. This is the cross-entropy loss function.

[0013] An electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform a semantically guided and gait-enhanced pedestrian re-identification method for changing clothes.

[0014] A computer-readable storage medium storing instructions that, when executed, perform a semantically guided and gait-enhanced method for re-identifying pedestrians changing clothes.

[0015] The beneficial effects of the technical solution provided in this application are: This invention effectively suppresses interference from clothing regions through a dynamic attention mechanism, guides the model to focus on key human structural regions through semantic masks, and supplements cross-clothing structural features that visual information cannot stably express through gait features. Furthermore, it achieves deep fusion of visual, semantic, and gait information by combining cross-modal semantic consistency constraints. This invention significantly improves the stability and accuracy of pedestrian re-identification in complex real-world scenarios such as clothing changes, lighting changes, pose differences, partial occlusion, and rapid movement. Simultaneously, the method supports end-to-end training, has a clear structure, stable training, and good engineering deployment capabilities, and can be directly applied to intelligent surveillance, multi-camera security systems, and urban computing scenarios. Attached Figure Description

[0016] The present application will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a flowchart of a semantically guided and gait-enhanced pedestrian re-identification method according to an embodiment of the present invention; Figure 2 This is a diagram of the dynamic adaptive multi-head attention network structure according to an embodiment of the present invention; Figure 3 This is a diagram of the semantic information enhancement network structure according to an embodiment of the present invention; Figure 4 This is a structural diagram of the multimodal three-flow framework pedestrian re-identification model according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the electronic device structure in an embodiment of the present invention; Figure 6 This is a reference diagram showing the specific recognition effect of an embodiment of the present invention. Detailed Implementation

[0017] To provide a clearer understanding of the technical features, objectives, and effects of this application, the specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0018] Embodiments of this application provide a semantically guided and gait-enhanced method for re-identifying pedestrians changing clothes.

[0019] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating the steps of a semantically guided and gait-enhanced pedestrian re-identification method for changing clothes, as described in an embodiment of this application, including: S1. Construct a dataset based on the acquired pedestrian images to be identified; S2. Construct a multimodal three-flow framework for pedestrian re-identification of clothing changes. The pedestrian re-identification model of clothing changes includes: a feature extraction network for extracting multi-scale visual features, a dynamic adaptive multi-head attention module for dynamically suppressing clothing interference and enhancing key identity features, and a semantic consistency constraint module for cross-modal alignment of semantic features and gait features. S3. Train the pedestrian re-identification model using the dataset; during training, jointly optimize the gait-assisted loss function, the semantic information enhancement loss function, and the semantic consistency constraint loss function. S4. Input the pedestrian image to be tested into the trained pedestrian re-identification model after changing clothes, and output the pedestrian re-identification result.

[0020] This application provides an embodiment as follows: An image of a pedestrian to be identified is input into a feature extraction network to obtain a multi-scale feature map; the multi-scale feature map is processed by a dynamic adaptive multi-head attention module to obtain enhanced identity features; a semantic information enhancement task generates a semantic mask and fuses it with visual features; a gait assistance task uses a gait sequence prediction module to generate gait features; finally, a semantic consistency constraint module performs cross-modal fusion and alignment of the three types of features.

[0021] Step S2 includes: The feature extraction network is connected to a dynamically adaptive multi-head attention module; the dynamically adaptive multi-head attention module is connected to a semantic consistency constraint module. The feature extraction network includes: a backbone network and multiple branch modules. The backbone network adopts a ResNet-50 or OSNet structure and is used to extract identity-related basic features. The branch module is used to extract multi-scale feature maps from different stages of the backbone network, including feature maps with scales of 64×64, 32×32, 16×16 and 8×8.

[0022] As one example, identity-related basic features are used in subsequent semantic enhancement streams, gait-assisted streams, and multi-stream fusion modules.

[0023] As one embodiment, based on a multimodal learning strategy, a pedestrian re-identification task, a semantic information enhancement task, and a gait assistance task are constructed, and a multimodal three-flow framework pedestrian re-identification model for changing clothes is built, performing joint learning of the multimodal approaches. A feature extraction network is used to extract multi-scale feature maps from the image input to the model; a dynamic adaptive multi-head attention module is used to dynamically suppress clothing-related regions and amplify key identity features; based on the image input to the model, the multi-scale feature maps, and the fused feature maps, a semantic consistency constraint module generates semantically consistent feature maps and gait feature maps, and aligns the semantically consistent feature maps and gait feature maps.

[0024] As one embodiment, the pedestrian image to be identified is acquired, and the image is divided into a training set, a validation set, and a test set according to a preset ratio. The data may come from public pedestrian re-identification datasets (such as PRCC, LTCC, VC-Clothes) or multi-view video frames in actual monitoring environments. The training set is used for model parameter learning, the validation set is used for parameter tuning, and the test set is used for performance evaluation.

[0025] Step S2 also includes: The dynamic adaptive multi-head attention module includes an adaptive parameter modulation unit and a multi-head self-attention fusion unit; The processing procedure of the adaptive parameter modulation unit is as follows: Multi-scale feature maps include: low-scale feature maps and high-scale feature maps; The low-scale feature maps are input into the branches used to generate Value and Key, respectively, and then passed through 1×1 convolution, dilated spatial pyramid pooling module and view transformation operation to obtain linear matrices γ and θ. The high-scale feature map is input into the branch used to generate the query, and then passed through 1×1 convolution and view transformation operations to obtain a linear matrix φ. Calculate the dot product of φ and θ, and obtain the spatial location feature attention weights after Softmax normalization; The attention weights are multiplied by the linear matrix γ, and a linear transformation is performed to output the globally associated feature map. The adaptive parameter modulation unit dynamically generates scaling factors and biases based on input features to adjust attention weights; The semantically enhanced feature map and gait feature map are obtained through scaling factors, biases, and multi-head self-attention fusion units, specifically including: Through the semantic information enhancement task, pedestrian images are processed by a semantic parser to generate semantic mask images. The semantic mask images are then fused with multi-scale feature maps to obtain semantically enhanced feature maps. The gait-assisted task inputs multi-scale feature maps into the gait sequence prediction module to generate gait feature maps.

[0026] As one embodiment, the multi-scale feature map is input to the adaptive parameter modulation unit of the dynamic adaptive multi-head attention module to generate input-dependent scaling factors and biases; The scaling factor and bias are adaptively generated based on the input features, thereby dynamically adjusting the attention intensity and enabling the model to stably focus on identity-sensitive areas in environments with changing clothing.

[0027] The scaling factor and bias are combined with the multi-head self-attention fusion unit to obtain the enhanced identity feature map. Through the semantic information enhancement task, the pedestrian image is processed by the semantic parser to generate a semantic mask image. The semantic mask image is fused with the multi-scale feature map to obtain the semantically enhanced feature map.

[0028] The model also includes a gait-assisted flow to extract temporal structural features of the human body from gait sequences or continuous frame variations. Gait features are extracted using GaitSet-type operators and upsampled through convolutional mapping to achieve the same spatial size as visual features, enabling pixel-wise alignment between gait features and visual features. The features learned by the gait-assisted flow exhibit strong cross-clothing stability, effectively compensating for the interference caused by clothing deformation on visual features.

[0029] The gait-assisted task inputs multi-scale feature maps into the gait sequence prediction module to generate gait feature maps; after fusing the semantic enhancement feature maps and gait feature maps, a semantic alignment association is established through the semantic consistency constraint module.

[0030] Step S2 also includes: The semantic consistency constraint module includes: a semantic consistency loss function and a gait sequence prediction module; The input to the semantic consistency constraint module includes a semantic enhancement feature map and a gait feature map, and its processing procedure is as follows: The semantic enhancement feature map and the gait feature map are added together to obtain the fused feature map. The local attention weight and the global attention weight are calculated through the local feature branch and the global feature branch, respectively. The local feature branch is composed of two convolutional modules, two batch normalization modules, a ReLU activation function and a Sigmoid function concatenated together. The global feature branch consists of a pooling module, two convolutional modules, two batch normalization modules, a ReLU activation function, and a Sigmoid function concatenated together. The result of multiplying the semantically enhanced feature map with the global attention weights is added to the result of multiplying the gait feature map with the local attention weights to obtain the aligned output features.

[0031] As one implementation, the semantic information enhancement stream generates a semantic mask from the input image through a semantic parser, including identity-related regions such as the head and legs. After the semantic mask is fused with the visual feature map, fine-grained semantic cues and global consistency information are extracted through local semantic branches and global semantic branches, respectively. Two types of attention work together on feature fusion, enabling the model to accurately highlight stable parts of the human body even in complex backgrounds and with significant variations in clothing texture.

[0032] Step S3 includes: The gait-assisted loss function We use a weighted binary cross-entropy loss, the expression of which is:

[0033] in, This represents the gait predictions made by the model, i.e., the gait feature map; Indicates the true gait label, These are the weighting coefficients. This represents the binary cross-entropy loss function.

[0034] Step S4 includes: The semantic information enhancement loss function Combining cross-entropy loss and KL divergence, its expression is:

[0035] in, Represents the actual semantic segmentation labels, This represents the semantic probability predicted by the model. Represents cross-entropy loss, Denotes KL divergence, These are the balancing weighting coefficients.

[0036] As one example, the semantic information enhancement loss function strengthens the semantic weights of identity-related regions and suppresses clothing-related regions, enabling the model to focus on stable identity features.

[0037] Step S4 includes: The semantic consistency constraint loss function The expression used to constrain the consistency between semantic features and gait features is: in, The loss function represents the semantic consistency constraint task. This represents semantic enhancement features. Indicates gait characteristics, Indicates fusion features, Indicates a real identity tag. It is a threshold. Indicates the number of pixels or features. This is the cross-entropy loss function.

[0038] As one embodiment, the multimodal joint training framework proposed in this invention employs three types of loss functions to jointly optimize the model, including a gait-aided loss for supervising gait prediction. Semantic augmentation loss used to improve semantic region segmentation and focusing capabilities and semantic consistency loss for feature alignment. During training, the three components work together through backpropagation, enabling visual flow, semantic flow, and gait flow to learn collaboratively from different spatial and modal dimensions, thereby obtaining a more robust identity representation of clothing changes.

[0039] As one embodiment, the model training process is as follows: select an optimizer, set the number of training iterations, and train a multimodal three-flow framework pedestrian re-identification model using the loss functions of the gait assistance task, the semantic information enhancement task, and the semantic consistency constraint task, along with the training set; calculate and update the network parameters of the model using the optimizer, and adjust the values ​​of the three loss functions according to the network parameters; guide the training process using a weight decay strategy and a learning rate reduction method based on cosine annealing, and determine whether training has ended based on the number of training iterations and the values ​​of the loss functions.

[0040] This embodiment uses Figure 1 The process shown is for reference and includes four stages: data preparation, model building, training, and evaluation.

[0041] Datasets and Preprocessing: (1) Data source: Public datasets PRCC, LTCC and VC-Clothes were used, and supplementary data (color video frames and corresponding gait sequences) were collected in actual deployment scenarios.

[0042] (2) Splitting ratio: Each dataset is split into training set, validation set and test set in a ratio of 6:2:2. The splitting is performed at the sample level to ensure that video frames of the same identity do not appear in different subsets at the same time.

[0043] (3) Image preprocessing: All input images are first scaled by 256 pixels on the short side, and then cropped to 256×128 (width×height) in the center; normalized to [0,1], and standardized using ImageNet mean / variance; random horizontal flipping (probability 0.5), random color jitter and random erasure are used as data augmentation.

[0044] (4) Gait sequence: Cut out pedestrian detection boxes from the video and align them according to the time sequence to form a sequence of length T (e.g., T=8) frames. If it is shorter than T, use frame copying or interpolation to complete it.

[0045] Input and output formats: (1) Input: A single RGB image I (H×W×3) and the corresponding gait sequence G (T×H'×W'×3 or gait feature map); (2) Output: Identity feature vector z (dimension D), used for retrieval or classification (matching based on Euclidean distance or cosine similarity).

[0046] Detailed Model Structure: Based on a multimodal learning strategy, multimodal learning is a machine learning method. In the field of machine learning, multimodal learning is a joint learning approach where multiple modalities learn in parallel, and the results influence each other. This model constructs tasks for pedestrian re-identification, semantic information enhancement, and gait assistance, and builds a multimodal three-stream framework for pedestrian re-identification under clothing changes.

[0047] The multimodal three-flow framework for pedestrian re-identification after changing clothes includes: a feature extraction network, a dynamic adaptive multi-head attention module, and a semantic consistency constraint module.

[0048] Feature extraction network (Backbone + multi-scale branches): (1) Backbone: ResNet-50 (pre-trained on ImageNet), constructing multi-scale branches on the outputs of conv2~conv5.

[0049] (2) Branch configuration: Connect a 1x1→3x3→1x1 Bottleneck branch to the output of conv2, conv3, conv4 and conv5 respectively, and output scales (taking the input 256×128 as an example) 64×64×256, 32×32×512, 16×16×1024 and 8×4×2048 respectively.

[0050] (3) Each branch is followed by BatchNorm and ReLU, and then L2 normalization is performed for subsequent attention calculation.

[0051] The feature extraction network in this embodiment of the invention adopts the ResNet-50 network structure. The first branch module of the feature extraction network is configured with 64 1×1 convolutional kernels, 64 3×3 convolutional kernels, and 256 1×1 convolutional kernels, outputting data with features of 64×64×256. The second branch module of the feature extraction network is configured with 128 1×1 convolutional kernels, 128 3×3 convolutional kernels, and 512 1×1 convolutional kernels, outputting data with features of 32×32×512. The third branch module of the feature extraction network is configured with 256 1×1 convolutional kernels, 256 3×3 convolutional kernels, and 1024 1×1 convolutional kernels, outputting data with features of 16×16×1024. The fourth branch module of the feature extraction network is configured with 512 1×1 convolutional kernels, 512 3×3 convolutional kernels and 2048 1×1 convolutional kernels, and outputs data with features of 16×16×2048.

[0052] Dynamic Adaptive Multi-Head Attention Module (AdaMHSA): The Dynamic Adaptive Multi-Head Attention Module includes an adaptive parameter modulation unit and a multi-head self-attention fusion unit.

[0053] The present invention implements a dynamic adaptive multi-head attention network reference. Figure 2 In the attention mechanism, Query is used to measure the relevance to each Key, Key is used to construct the computational basis for attention scores, and Value is used to perform a weighted sum based on the attention scores to generate the final attention output. The Dynamic Adaptive Multi-Head Attention Module (AdaMHSA) takes two feature maps of different scales as input, with sizes H×W×C. The lower-scale feature map in the two different scales calculates Value and Key respectively. Value is sequentially passed through a 1×1 convolution, ASPP, and view function to obtain a linear matrix γ, and Key is sequentially passed through a 1×1 convolution, ASPP, and view function to obtain a linear matrix θ. The higher-scale feature map in the two different scales calculates Query. Query is sequentially passed through a 1×1 convolution and view function to obtain a linear matrix φ. The original mapping of spatial location feature attention weights is calculated using a matrix dot product, and softmax is used to normalize the original mapping to obtain the spatial location feature attention weights. The spatial location feature attention weights are then multiplied by a matrix dot product to obtain a feature matrix with long-distance dependencies. The feature matrix is ​​then linearly transformed by the view function into a globally associated output feature map of size H×W×C.

[0054] Semantic Information Enhancement Stream (SIE-Stream): A reference network for semantic information enhancement in this invention. Figure 3The Semantic Information Enhancement Network (SIE-Stream) takes two feature maps of the same scale as input, x and y, and adds them together to obtain feature map x. Local and global attention weights are calculated using local and global feature branches, respectively. The sum of these weights is then multiplied by the input and output features to obtain the final output feature. The local feature branch consists of a Conv module, a BN module, a ReLU module, and a cascaded Sigmoid function; the global feature branch consists of a Pooling module, a Conv module, a BN module, a ReLU module, and a cascaded Sigmoid function. The dynamic adaptive multi-head attention network of the first semantic fusion subnetwork of the semantic information enhancement network globally correlates and fuses the feature maps output by the third and fourth branch modules of the feature extraction network into 16×16×256 data. The semantic information enhancement network of the first semantic fusion subnetwork then fuses the 16×16×256 data output by the dynamic adaptive multi-head attention network of the first semantic fusion subnetwork with the 16×16×256 data output by the third branch module of the feature extraction network to obtain the 16×16×256 feature map output by the first semantic fusion subnetwork. The dynamic adaptive multi-head attention network of the second semantic fusion sub-network of the semantic information enhancement network globally associates and fuses the feature map output by the second branch module of the feature extraction network with the feature map output by the first semantic fusion sub-network to obtain a 32×32×256 data. The semantic information enhancement network of the second semantic fusion sub-network then fuses the 32×32×256 data output by the dynamic adaptive multi-head attention network of the second semantic fusion sub-network with the 32×32×256 data output by the second branch module of the feature extraction network to obtain the 32×32×256 feature map output by the second semantic fusion sub-network. The dynamic adaptive multi-head attention network of the third semantic fusion sub-network of the semantic information enhancement network globally associates and fuses the feature map output by the first branch module of the feature extraction network with the feature map output by the second semantic fusion sub-network into a 64×64×256 data. The 64×64×256 data output by the dynamic adaptive multi-head attention network of the third semantic fusion sub-network is then fused with the 64×64×256 data output by the first branch module of the feature extraction network to obtain the 64×64×256 feature map output by the third semantic fusion sub-network.

[0055] The semantic information enhancement network acquires information from different spatial locations in the feature map, suppresses unimportant information, and highlights effective information. The gait sequence prediction module of the semantic consistency constraint module is configured with 256 1×1 convolutional kernels, 256 3×3 convolutional kernels with a dilation coefficient of 12, 256 3×3 convolutional kernels with a dilation coefficient of 24, and 256 3×3 convolutional kernels with a dilation coefficient of 36. It is then serially connected with 1×1 adaptive average pooling and bilinear interpolation upsampling, outputting data with a feature size of 16×16×256.

[0056] Gait Assist Flow: In the gait assistance task, the gait features extracted by the GaitSet operator are downsampled by a factor of four and merged with the output of the downsampling module of the feature extraction network. This data is then processed by a Conv module with 256 convolutional kernels of size 1×1 and a stride of 1, resulting in a gait feature map of size 128×128×256 rich in gait information. Through upsampling and convolutional mapping operations, the gait features are adjusted to a spatial scale consistent with the semantic enhancement flow, resulting in a gait feature map used for subsequent semantic consistency constraints and multimodal fusion. In the semantic information enhancement task, the output of the gait sequence prediction module is upsampled by a factor of eight, resulting in data with features of size 128×128×256. This data is then fused with the output of the third semantic fusion sub-network of the semantic information enhancement network to obtain the enhanced semantic feature map.

[0057] The semantic feature map is then fused with the gait features generated by the gait assistance stream at the feature level. Upsampling and feature mapping operations are then performed to obtain the final semantically enhanced features, which are used to enhance the model's focus on key identity regions. In the semantic consistency constraint task, the cross-modal relationship between the semantically enhanced features and the gait assistance features is used to construct a semantic consistency loss, ensuring consistency between the two at the identity level, thereby achieving alignment optimization of multimodal features. Corresponding loss functions are constructed for the gait assistance task, the semantic information enhancement task, and the semantic consistency constraint task to complete multimodal joint training and optimization.

[0058] Based on gait prediction results, a loss function for the gait-assisted task during training of a multimodal three-flow framework pedestrian re-identification model is constructed.

[0059] Fusion and Classification Header: Concatenation: f_vis (visual features), f_sem (semantic enhancement features), and f_gait are concatenated and then dimensionality reduced by 1x1 convolution to obtain the fusion feature f_fusion; Attention fusion: Apply local and global attention to f_fusion to obtain the final feature f_final; Output: Perform global average pooling (GAP) on f_final and connect it to an FC layer to obtain embeddingz; during training, the identity cross-entropy head and the triple / contrast loss head can be concatenated.

[0060] Loss Function and Training Configuration: In the multimodal three-stream framework of this invention, the loss function system plays a crucial role in optimizing overall performance. Unlike single visual stream models that mainly rely on identity classification loss or contrastive learning loss, this invention constructs a multi-dimensional composite loss system, including gait assistance loss, semantic enhancement loss, semantic consistency loss, and identity discrimination loss. This enables the model to simultaneously optimize feature representation capabilities across multiple information dimensions, thereby achieving highly robust recognition performance in complex scenarios such as clothing changes.

[0061] During training, the gait-assisted loss is primarily used to supervise gait sequence prediction or gait category discrimination. Gait, as a behavioral pattern independent of clothing, possesses strong structural stability. Supervising gait features through binary cross-entropy or multi-class cross-entropy guides the network to focus on capturing dynamic features weakly correlated with clothing changes, such as human movement rhythm, stride structure, and body posture. This loss term effectively enhances the discriminative ability of the gait-assisted flow, enabling gait features to provide a stable reference for the visual flow in subsequent fusion stages.

[0062] The semantic enhancement loss primarily serves to optimize the semantic segmentation module. In real-world surveillance scenarios, the semantic boundaries of different parts of the human body are often blurred or misclassified due to the influence of lighting, occlusion, or pose changes. Therefore, this invention introduces two components into the semantic enhancement loss: cross-entropy and KL divergence. Cross-entropy is used to supervise the semantic category boundaries, while KL divergence is used to constrain the probability-level closeness between the predicted distribution and the true distribution. This combination not only improves the stability of semantic parsing but also enables the semantic mask to more accurately locate key identity regions during multi-scale feature fusion, thereby enhancing the overall recognition performance.

[0063] Semantic consistency loss is used to constrain the consistency between the semantic enhancement stream and the gait assistance stream in the identity feature space. Since the features of the two modalities come from different representation channels, a lack of consistency constraints may lead to distributional biases in the fused features, affecting the robustness of the overall model. This invention calculates the mean squared error of the semantic enhancement features and gait features to gradually align the two modalities in spatial structure; simultaneously, it introduces cross-entropy loss for fusion classification to ensure that the final fused features maintain high discriminative power for identity categories. Through this cross-modal dual constraint, the model can establish a stable mapping relationship between different modalities, thereby further improving re-identification accuracy.

[0064] Building upon the aforementioned losses, this invention further introduces traditional identity cross-entropy and triplet loss. Identity cross-entropy directly improves the model's classification ability for different identity categories, while triplet loss optimizes the feature space by bringing positive samples closer and pushing negative samples further away, making the embedding representation more compact and separable. In summary, the total loss function of this invention is composed of multiple complementary modules, specifically in the following form: α and γ are important parameters for adjusting the weights of different tasks and can be flexibly adjusted according to the characteristics of the scenario.

[0065] During training, this invention employs a stochastic gradient descent optimizer with a driving term to ensure stability and generalization ability. The initial learning rate is set to 0.008, and a cosine annealing strategy is used to gradually reduce the learning rate, enabling the network to converge quickly in the early stages. In the later stages, fine-grained learning rate adjustment is used to avoid oscillations and improve final performance. The entire training process consists of 160 epochs, with a random seed of 1 to ensure the reproducibility of experimental results. For data processing, enhancement strategies such as random erasure, random flipping, and color perturbation are used to effectively improve the model's adaptability to complex environments such as lighting changes and occlusion.

[0066] Furthermore, in terms of engineering implementation, this invention employs mixed-precision training to improve computational efficiency and reduce GPU memory usage, and uses synchronous BatchNorm in multi-GPU training to maintain consistency of statistics across different devices. In the semantic parsing part, the semantic segmentation network can be pre-trained, and pseudo-labeling technology can be used to expand the training data, thereby further improving the stability of semantic features.

[0067] To facilitate understanding of the training and optimization process, the following provides a mathematical formula and technical explanation of the key loss function used in this invention: First, the gait-assisted loss is used to supervise the gait prediction results, and its mathematical form is:

[0068] in The predicted probability for the identity category. The loss statement represents the true label. This loss can directly improve the model's ability to classify identity categories.

[0069] To obtain a more compact and discriminative feature distribution, this invention further introduces a triplet loss, the formula of which is:

[0070] in, , , These represent anchor features, positive sample features, and negative sample features for the same identity, respectively. is the margin constant. This loss enhances the discriminative power of the feature space, making clusters of the same identity more compact and clusters of different identities more dispersed.

[0071] The aforementioned losses collectively constitute the final training objective of this invention, and its overall expression is:

[0072] in and The weights are hyperparameters used to adjust the contribution ratio of different tasks in the overall optimization process. Through the above-mentioned multi-loss collaborative optimization mechanism, the multimodal three-flow model constructed in this invention can obtain more stable, more accurate, and more generalized identity feature representations.

[0073] Optional implementation variant: Input the pedestrian image to be tested into the final pedestrian re-identification model to obtain the pedestrian re-identification result. In this invention, the pedestrian images to be tested are selected from the PRCC dataset; specific recognition results can be found by referring to... Figure 6 .

[0074] This application also discloses an electronic device. (See reference...) Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0075] The communication bus 502 is used to enable communication between these components.

[0076] The user interface 503 may include a display screen, and optionally, the user interface 503 may also include a standard wired interface or a wireless interface.

[0077] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0078] This application also discloses a computer-readable storage medium storing multiple instructions adapted for loading by a processor to execute the aforementioned semantically guided and gait-enhanced pedestrian re-identification method for changing clothes.

[0079] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure.

[0080] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for semantic-guided and gait-enhanced pedestrian re-identification for changing clothes, characterized in that, The method includes the following steps: S1. Construct a dataset based on the acquired pedestrian images to be identified; S2. Construct a multimodal three-flow framework for pedestrian re-identification of clothing changes. The pedestrian re-identification model of clothing changes includes: a feature extraction network for extracting multi-scale visual features, a dynamic adaptive multi-head attention module for dynamically suppressing clothing interference and enhancing key identity features, and a semantic consistency constraint module for cross-modal alignment of semantic features and gait features. S3. Train the pedestrian re-identification model using the dataset; during training, jointly optimize the gait-assisted loss function, the semantic information enhancement loss function, and the semantic consistency constraint loss function. S4. Input the pedestrian image to be tested into the trained pedestrian re-identification model after changing clothes, and output the pedestrian re-identification result.

2. The method of claim 1, wherein the semantic guidance and gait enhancement for a person re-identification in a dressing process is characterized by, Step S2 includes: The feature extraction network is connected to a dynamically adaptive multi-head attention module; the dynamically adaptive multi-head attention module is connected to a semantic consistency constraint module. The feature extraction network includes: a backbone network and multiple branch modules. The backbone network adopts a ResNet-50 or OSNet structure and is used to extract identity-related basic features. The branch module is used to extract multi-scale feature maps from different stages of the backbone network, including feature maps with scales of 64×64, 32×32, 16×16 and 8×8.

3. The method of claim 2, wherein the semantic guidance and gait enhancement for a person re-identification in a dressing process is characterized by, Step S2 also includes: The dynamic adaptive multi-head attention module includes an adaptive parameter modulation unit and a multi-head self-attention fusion unit; The processing procedure of the adaptive parameter modulation unit is as follows: Multi-scale feature maps include: low-scale feature maps and high-scale feature maps; The low-scale feature maps are input into the branches used to generate Value and Key, respectively, and then passed through 1×1 convolution, dilated spatial pyramid pooling module and view transformation operation to obtain linear matrices γ and θ. The high-scale feature map is input into the branch used to generate the query, and then passed through 1×1 convolution and view transformation operations to obtain a linear matrix φ. Calculate the dot product of φ and θ, and obtain the spatial location feature attention weights after Softmax normalization; The attention weights are multiplied by the linear matrix γ, and a linear transformation is performed to output the globally associated feature map. The adaptive parameter modulation unit dynamically generates scaling factors and biases based on input features to adjust attention weights; The semantically enhanced feature map and gait feature map are obtained through scaling factors, biases, and multi-head self-attention fusion units, specifically including: Through the semantic information enhancement task, pedestrian images are processed by a semantic parser to generate semantic mask images. The semantic mask images are then fused with multi-scale feature maps to obtain semantically enhanced feature maps. The gait-assisted task inputs multi-scale feature maps into the gait sequence prediction module to generate gait feature maps.

4. The method of claim 3, wherein the semantic guidance and gait enhancement for a person re-identification in a dressing process is characterized by, Step S2 also includes: The semantic consistency constraint module includes: a semantic consistency loss function and a gait sequence prediction module; The input to the semantic consistency constraint module includes a semantic enhancement feature map and a gait feature map, and its processing procedure is as follows: The semantic enhancement feature map and the gait feature map are added together to obtain the fused feature map. The local attention weight and the global attention weight are calculated through the local feature branch and the global feature branch, respectively. The local feature branch is composed of two convolutional modules, two batch normalization modules, a ReLU activation function and a Sigmoid function concatenated together. The global feature branch consists of a pooling module, two convolutional modules, two batch normalization modules, a ReLU activation function, and a Sigmoid function concatenated together. The result of multiplying the semantically enhanced feature map with the global attention weights is added to the result of multiplying the gait feature map with the local attention weights to obtain the aligned output features.

5. The method of claim 3, wherein the semantic guidance and gait enhancement for a person re-identification in a dressing process is characterized by, Step S3 includes: The gait-assisted loss function We use a weighted binary cross-entropy loss, the expression of which is: in, This represents the gait predictions made by the model, i.e., the gait feature map; Indicates the true gait label, These are the weighting coefficients. This represents the binary cross-entropy loss function.

6. The semantically guided and gait-enhanced method for re-identifying pedestrians changing clothes as described in claim 1, characterized in that, Step S4 includes: The semantic information enhancement loss function Combining cross-entropy loss and KL divergence, its expression is: in, Represents the actual semantic segmentation labels, This represents the semantic probability predicted by the model. Represents cross-entropy loss, Denotes KL divergence, These are the balancing weighting coefficients.

7. The semantically guided and gait-enhanced method for re-identifying pedestrians changing clothes as described in claim 1, characterized in that, Step S4 includes: The semantic consistency constraint loss function The expression used to constrain the consistency between semantic features and gait features is: in, The loss function represents the semantic consistency constraint task. This represents semantic enhancement features. Indicates gait characteristics, Indicates fusion characteristics, Indicates a real identity tag. It is a threshold. Indicates the number of pixels or features. This is the cross-entropy loss function.

8. An electronic device, characterized in that, The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the semantic guidance and gait enhancement pedestrian re-identification method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed by a computer, perform the semantic guidance and gait enhancement pedestrian re-identification method as described in any one of claims 1-7.

Images