A method for re-identifying a person changing clothes based on appearance analysis and guidance

Abstract

The present application belongs to the field of computer vision and pedestrian re-identification, and discloses a kind of based on human appearance analysis guide's change dress pedestrian re-identification method. The human body part semantic analysis is carried out to the training image, the original visible light branch and the human body analysis branch are constructed, and the original feature and the structural feature are extracted;Local difference estimation generates a dense gating matrix, and the weighted structural feature is injected into the original feature in a residual manner to obtain local reconstruction features and global fusion features, and training is carried out in combination with hierarchical contrastive loss, identity classification loss and triplet loss. In the inference stage, only the original visible light branch is retained, and pedestrian retrieval is carried out according to the cosine similarity of the image features to be tested. The present application uses human structure prior to adaptively weaken the interference of clothing change, improves the robustness and discriminability of the change dress pedestrian re-identification.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and pedestrian re-identification, and particularly relates to a method for re-identifying pedestrians in different outfits based on human appearance analysis. Background Technology

[0002] Pedestrian re-identification aims to retrieve target pedestrians with the same identity from images or videos captured by non-overlapping cameras. Traditional pedestrian re-identification methods usually assume that the clothing appearance of the same pedestrian remains relatively stable under different camera views. Therefore, the model tends to use appearance cues such as color, texture, and clothing style for identity matching.

[0003] In long-term monitoring and cross-time period retrieval scenarios, the clothing of the same pedestrian may change significantly, making the original cues of color, texture, and style unreliable. Pedestrian re-identification with clothing changes aims to extract stable identity-discriminating features from pedestrian images with significant clothing changes. Compared with traditional pedestrian re-identification tasks, it faces stronger intra-class discriminative differences and more complex appearance changes.

[0004] Existing methods for re-identifying pedestrians in different outfits mainly fall into the following categories: one type of method implicitly weakens the influence of clothing features through data augmentation, adversarial learning, causal inference, or clothing-irrelevant constraints; another type of method introduces auxiliary priors such as human posture, human contour, human analytical map, or attribute information to explicitly enhance the model's ability to perceive human structure and stable identity regions; and yet another type of method constrains the feature space through metric learning or contrastive learning loss functions to bring samples of the same identity closer together and samples of different identities further apart.

[0005] Existing methods for re-identifying people in different clothing based on human body analysis or semantic priors typically use the analysis results as masks, attention maps, or auxiliary inputs to suppress clothing areas or enhance human body structure areas. However, relying solely on the analyzed image can easily lead to the loss of fine-grained identity clues in the original image, such as head features, body contours, local accessories, and pose details. On the other hand, directly concatenating or adding the original visible light features with the analyzed features makes it difficult to differentiate and fuse them according to the degree of clothing changes in different local areas of the human body.

[0006] Furthermore, clothing changes typically do not affect the entire pedestrian image uniformly, but rather are concentrated in localized areas such as upper garments, lower garments, and outerwear. Different areas are affected to varying degrees by clothing, occlusion, background, and pose changes. If only global features are used for modeling, the model struggles to identify which localized areas require more structural priors and which areas should retain their original appearance details, thus impacting the ability to identify individuals in clothing-changing scenarios.

[0007] In terms of feature optimization, existing cross-stream or cross-modal constraints often only focus on strict alignment of the same image instance, that is, only the original image features and human body parsing features corresponding to the same image are used as positive samples, while ignoring the identity consistency between different clothing samples of the same identity. Pedestrian re-identification with clothing change not only needs to achieve cross-stream alignment of the same image, but also needs to establish stable identity-level consistency constraints between different clothing samples of the same identity.

[0008] Therefore, how to fully utilize the structural priors provided by the human body analysis map during the training phase, adaptively fuse the original appearance features and human body structural features at the local region level, establish instance-level and identity-level hierarchical consistency constraints in the feature space, and not increase additional computational overhead during the inference phase are the technical problems that urgently need to be solved in the field of pedestrian re-identification after changing clothes. Summary of the Invention

[0009] This invention addresses the shortcomings of existing technologies by providing a pedestrian re-identification method based on human appearance analysis. It introduces semantic parsing information from human body parts as structural priors and utilizes a local difference gating fusion mechanism to adaptively adjust the information fusion strength between human structural features and appearance features. This allows the model to enhance its representational ability of human structural layout while preserving pedestrian appearance discrimination information, thereby reducing the impact of changes in clothing color, texture, and style on identity feature learning and improving the accuracy of pedestrian re-identification in clothing-changing scenarios.

[0010] To achieve the above objectives, the present invention adopts the following technical solution: a method for re-identifying pedestrians in disguise based on human appearance analysis, comprising:

[0011] S1. Collect raw visible light images of several target pedestrians respectively, construct a training image set and a test image set with identity labels, and perform image enhancement on the raw visible light images in the training image set.

[0012] S2. Perform semantic parsing of human body parts on the original visible light images in the training image set to obtain a human body parsing image set that corresponds one-to-one with the original visible light images.

[0013] S3. Construct an asymmetric dual-path feature extraction network that includes an original visible light branch and a human body analysis branch. Input the enhanced original visible light image into the original visible light branch to obtain the original feature map, and input the human body analysis image into the human body analysis branch to obtain the structural feature map.

[0014] Among them, the original visible light branch is the main branch that is retained in both the training and inference phases, while the human body analysis branch is an auxiliary branch that introduces human body analysis priors during the training phase.

[0015] S4. Divide the original feature map and the structural feature map into multiple local regions according to the same region division method, and calculate the difference between the original features and structural features in the corresponding local regions to obtain the local difference map;

[0016] S5. Generate a dense gating matrix based on the local difference map, use the dense gating matrix to weight the local structural features in the structural feature map, and fuse the weighted local structural features into the corresponding original features by residual injection to obtain the local fused features.

[0017] S6. Replace each local fusion feature back with the corresponding region of the original feature map to obtain multiple local reconstructed feature maps, and generate global fusion weights based on the dense gating matrix of each local region to obtain a global fusion feature map.

[0018] S7. Pooling and bottleneck layer mapping are performed on multiple local reconstructed feature maps and global fusion feature maps respectively to obtain local fusion descriptors and global fusion descriptors;

[0019] S8. Construct a hierarchical contrast loss based on the hierarchical relationship between strong positive samples across branches of the same image instance, weak positive samples across branches of different image instances of the same identity, and negative samples of different identities, and train the asymmetric dual-path feature extraction network by combining identity classification loss and triplet loss.

[0020] S9. Input the original visible light image to be tested from the set of images to be tested into the trained original visible light branch, close the human body analysis branch and the local difference gating fusion process, and extract the features of the image to be tested using only the original visible light branch.

[0021] S10. Sort the candidate pedestrian images according to the cosine similarity between the features of the images to be tested, and obtain the re-identification result of the target pedestrian.

[0022] Furthermore, in S2, each human body parsing image in the human body parsing image set represents different parts of the human body with different pixel values ​​or color codes, which is used to provide prior information on the layout, contour structure and spatial location of human body parts; the network used for semantic parsing of human body parts keeps its parameters frozen during the training of the asymmetric dual-path feature extraction network.

[0023] Furthermore, in S3, the original visible light branch and the human body analysis branch obtain the original feature map and structural feature map respectively through the corresponding feature extraction mapping. The original feature map and structural feature map are represented as follows:

[0024]

[0025]

[0026]

[0027] in, This represents the original visible light pedestrian image. Indicates and One-to-one correspondence of human body analysis diagrams; This represents the feature extraction mapping of the original visible light branch. This represents the feature extraction mapping of the human body parsing branch; This represents the original feature map extracted from the original visible light pedestrian image by the original visible light branch. This represents the structural feature map extracted by the human body analysis branch from the human body analysis map corresponding to the same pedestrian image. This represents the number of feature channels used for identity verification. and These represent the feature map height and feature map width corresponding to the spatial layout of a pedestrian's body, respectively.

[0028] In S3, the asymmetry of the asymmetric dual-path feature extraction network includes: asymmetric input information, where the original visible light branch receives the original visible light pedestrian image and extracts appearance features related to color, texture, and local visual patterns, while the human body parsing branch receives the human body parsing map corresponding to the same original visible light pedestrian image and extracts structural features guided by human body parsing priors; asymmetric fusion direction, where structural features are injected into appearance features in a residual manner rather than being bidirectionally fused with appearance features; and asymmetric training and inference lifecycle, where the original visible light branch, human body parsing branch, and local difference gating fusion process are used simultaneously during the training phase, while only the original visible light branch is retained during the inference phase.

[0029] Furthermore, in S4, the original feature map... and structural feature diagram Divided along the height direction corresponding to the longitudinal layout of the pedestrian's body into The nth horizontal local region is obtained. Original local features of a local region and structural local features ,in , , , In order to make Positive integers that are divisible by 0;

[0030] For the Spatial location in a local area The local difference value is calculated from the channel vector at a given location to form a local difference map. , i∈{1,…,H k},j∈{1,…,W},D (k) ∈RHk×W The formula is as follows:

[0031]

[0032] in, Indicates the first Spatial location in a horizontal local area The local difference value between the original features and structural features at a given location; Represents the inner product along the channel dimension. Represents the L2 norm, This represents the numerical stability constant to prevent division by zero.

[0033] The larger the local difference value, the greater the difference between the appearance features extracted by the original visible light branch and the structural features extracted by the human body analysis branch at that spatial location. In an embodiment determined by experiments with the number of partitions, Take 12.

[0034] Furthermore, in S5, a dense gating matrix is ​​generated based on the local difference map, and local residual fusion is completed based on the dense gating matrix, expressed by the formula:

[0035]

[0036]

[0037] in, Indicates the first Dense gated matrices for local regions Indicates the spatial location of this local area. Gating weights for analyzing the structural features of the human body; This represents the Sigmoid activation function. Indicates the gate temperature parameter. This represents the learnable variable used to generate the gated temperature parameters. This represents a constant that ensures the gated temperature parameter remains positive and its value is stable. This indicates element-wise multiplication and broadcasting along the channel dimension. Indicates the first Local fusion characteristics of a local region.

[0038] Furthermore, in S6, the first Local fusion features Replace with the original feature map The corresponding region is obtained by preserving other local regions as their original local features, thus obtaining the first... Local reconstruction feature map This can be expressed as a formula:

[0039]

[0040] in, This indicates a stitching operation along the height direction of the feature map. Indicates only in the first The residual information of the human body structure is introduced into each horizontal local region, while preserving the overall human body spatial layout. A local reconstructed feature map.

[0041] Furthermore, in S6 and S7, global fusion weights are calculated based on the dense gating matrices of each local region to obtain a global fusion feature map; and descriptors for supervised training are generated from the local reconstructed feature map, the global fusion feature map, the original feature map, and the structural feature map, respectively, as expressed by the formula:

[0042]

[0043] in, This represents the global fusion weight obtained from the mean of the dense gating matrices of each level's local region. Represents the global fused feature map. This represents the average value across all spatial locations of a dense gated matrix. Indicates adaptive pooling. Indicates the flattening operation. Indicates the first The bottleneck layer corresponding to each locally reconstructed feature map This represents the bottleneck layer corresponding to the global fusion feature map. and These represent the bottleneck layers corresponding to the original visible light branch and the human body analysis branch, respectively. Indicates the first A local fusion descriptor, Indicates the global fusion descriptor, This represents the original visible light branch descriptor obtained from the original feature map. This represents the human body parsing branch descriptor obtained from the structural feature map.

[0044] Furthermore, in S8, when constructing the hierarchical contrastive loss, training batches are sampled according to identity, and each training batch contains Each identity contains [number] identities, and each identity contains [number] identities. A variety of image instances, Training batch size Different image instances include raw visible light pedestrian images and their corresponding human body analysis diagrams captured under different clothing, different acquisition times, and / or different camera perspectives for the same identity;

[0045] Let the normalized descriptor of the original visible light branch output be... The normalized descriptor output by the human body parsing branch is , No. The identity label of each sample is Cross-branch similarity matrix Identity adjacency matrix Strong positive sample mask Weak positive sample mask and hierarchical soft target matrix They are expressed by the following formulas:

[0046]

[0047]

[0048] in, Indicates the first The original visible light branch descriptor and the first Cross-branch similarity between individual body parsing branch descriptors. Indicates the temperature coefficient for comparative learning; Represents the identity adjacency matrix. express An identity matrix of order 1. A cross-branch strong positive sample mask representing the same image instance. A cross-branch weak positive sample mask representing different image instances with the same identity; and Let represent the weights of strong positive samples and weak positive samples, respectively. This makes the cross-branch alignment constraint of the same image instance stronger than the consistency constraint of different image instances with the same identity; This represents the numerical stability constant in the normalization of the hierarchical soft target matrix; Let i,j∈{1,…,B} represent the normalized hierarchical soft target matrix.

[0049] For sample pairs with different identity labels for Samples with different identity labels are not assigned soft target weights as positive samples, but their similarity terms still participate in the Softmax normalization denominator of the similarity matrix, thus serving as negative samples of different identities to constrain the network to learn identity-distinguishing features; in an embodiment determined by parameter analysis experiments, Pick , Pick , Pick .

[0050] Furthermore, in S8, the layered contrast loss includes the loss from the original visible light branch to the human body resolution branch and the loss from the human body resolution branch to the original visible light branch, expressed by the formula:

[0051]

[0052]

[0053] in, This represents the contrast loss from the original visible light branch to the human body resolution branch. This represents the contrast loss from the human body resolution branch to the original visible light branch. The stratified comparison loss is represented by the Softmax normalized denominator in both directions, which includes cross-branch similarity terms for all identity samples in the same batch. Represents the normalized hierarchical soft target matrix The transpose of .

[0054] Furthermore, in S8, the total training loss of the asymmetric dual-path feature extraction network consists of identity classification loss, triplet loss, and hierarchical contrastive loss, expressed by the formula:

[0055]

[0056] in, This represents the original visible light branch descriptor, the human body parsing branch descriptor, the global fusion descriptor, and... A set of supervised features consisting of local fusion descriptors; Representation and Supervision Descriptors The corresponding identity category header, Indicates identity label;

[0057] This represents the identity classification loss used to constrain pedestrian identity category determination. This represents the triplet loss used to constrain triplets where the distance to the same identity feature is less than the distance to different identity features. Indicates the loss due to layered contrast; This indicates that the hierarchical comparison loss weight, identity classification loss weight, and triplet loss weight are set to 1.

[0058] Furthermore, in S9 and S10, the model inference stage only retains the original visible light branch after training. The query image and candidate library images are respectively input into the original visible light branch to obtain query features and candidate features, and the re-identification result is obtained based on cosine similarity, expressed by the formula:

[0059]

[0060] in, Indicates querying image features, Represents the image features of the candidate library. This represents the cosine similarity between the query image and candidate images in the database; it outputs candidate pedestrian images that match the identity of the query image, ranked from highest to lowest cosine similarity.

[0061] Compared with the prior art, the present invention has the following advantages.

[0062] This invention utilizes human body analytical maps to provide prior knowledge of human body part layout and structure. Through a local difference gating fusion mechanism, it adaptively adjusts the fusion intensity of human body structure prior knowledge in different local areas, thereby reducing the interference caused by changes in clothing color, texture, and style.

[0063] This invention preserves the fine-grained appearance information of the original visible light image while introducing structural information from the human body analysis image, and uses residual injection to perform local fusion, thus avoiding the loss of identity details caused by simply relying on the analysis image.

[0064] This invention supervises model training by jointly reconstructing local feature maps and fusing global feature maps, enabling the model to simultaneously acquire local region discrimination ability and overall identity representation ability, thereby improving feature robustness in dressing-up scenarios.

[0065] This invention employs a hierarchical contrast loss design, simultaneously considering the alignment of strong positive samples from the same image instance and the consistency of weak positive samples from different image instances of the same identity. This mitigates the intra-class feature dispersion problem caused by clothing changes and enhances the clustering of identity features. Furthermore, during the training phase, a human body parsing branch and a difference-gated local fusion module are used for assisted learning, while during the inference phase, only the original visible light branch is retained for feature extraction and retrieval. Therefore, this approach improves recognition accuracy while maintaining practical deployment efficiency. Attached Figure Description

[0066] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. The scope of protection of the present invention is not limited to the following description.

[0067] Figure 1 This is a flowchart of the pedestrian re-identification method based on human appearance analysis guided by the present invention.

[0068] Figure 2 This is a schematic diagram of the overall architecture of the asymmetric dual-path feature extraction network in the embodiment;

[0069] Figure 3 This is a schematic diagram of the structure of the difference-gated local fusion module in the embodiment;

[0070] Figure 4 This is a schematic diagram illustrating the construction of strong positive samples, weak positive samples, and negative samples for hierarchical contrast loss in the embodiment. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] The purpose of this invention is to provide a method for re-identifying pedestrians in disguise based on human appearance analysis.

[0073] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0074] This embodiment provides a method for re-identifying pedestrians in different clothing based on human appearance analysis guidance. The method constructs a human appearance analysis-guided differential learning framework. During the training phase, it introduces two inputs: the original visible light image and the human appearance analysis image. A differential gating local fusion module and hierarchical contrast loss guide the model to learn stable identity representations under clothing changes. During the inference phase, only the original visible light branch is used to extract features and perform retrieval.

[0075] Step S1: Data Acquisition and Image Enhancement.

[0076] Raw visible light images of several target pedestrians are acquired to construct a training image set and a test image set. Each raw visible light image in the training image set has a corresponding identity label. To enhance the model's robustness to viewpoint, pose, and local occlusion, the training images are randomly horizontally flipped, randomly cropped, randomly erased, and their size normalized. Preferably, the input image size is adjusted to... .

[0077] Step S2: Offline human body analysis and generation of human body analysis diagram.

[0078] Before training the asymmetric dual-path feature extraction network, a pre-trained human body parsing model with fixed parameters is invoked to perform offline semantic parsing of human body parts on each original visible light pedestrian image in the training image set. This yields semantic parsing results for human body parts that correspond one-to-one with the original visible light pedestrian image, and a human body parsing map is generated based on these results. The human body parsing model outputs semantic parsing results for human body parts such as the head, upper body, lower body, arms, legs, and shoes, and encodes different human body parts with different colors or pixel values.

[0079] In one embodiment, the human body parsing model is a pre-trained Uniparser model; in other embodiments, other human body parsing networks capable of outputting semantic parsing results for human body parts may also be used. The human body parsing model is used only as an offline preprocessing tool, and its parameters remain fixed when generating the human body parsing image; when training the asymmetric dual-path feature extraction network, the original visible light image and its corresponding human body parsing image are directly read as training sample inputs. The human body parsing model itself is not a component of the asymmetric dual-path feature extraction network, does not receive gradients from identity classification loss, triplet loss, or hierarchical contrast loss, and does not participate in backpropagation updates.

[0080] A human body analysis image is a semantically encoded image of human body parts generated offline from the same original visible light pedestrian image and maintaining a one-to-one correspondence with the original visible light pedestrian image and its identity label. A human body analysis image does not mean that the analysis image alone contains complete identity and appearance information, nor is it required that the analysis image alone can complete identity recognition. Rather, it represents the analysis image as a structural priori representation of the corresponding original visible light pedestrian image, used to characterize the layout, contour structure, and spatial relationships of human body parts.

[0081] Step S3: Asymmetric dual-path feature extraction.

[0082] An asymmetric dual-path feature extraction network is constructed, consisting of a raw visible light branch and a human body analysis branch. The raw visible light branch extracts appearance features related to color, texture, and local visual patterns; the human body analysis branch extracts structural features guided by human body analysis priors. The raw visible light branch is the main branch retained in both the training and inference phases, while the human body analysis branch is an auxiliary branch used in the training phase to introduce human body structural priors.

[0083] The asymmetry of the asymmetric dual-path feature extraction network includes: asymmetric input information, i.e., the two branches receive the original visible light pedestrian image and the human body analysis image respectively; asymmetric fusion direction, i.e., structural features are injected into appearance features in a residual manner, rather than being bidirectionally and equivalently fused with appearance features; and asymmetric training and inference lifecycle, i.e., the two branches and the local difference gating fusion process are used simultaneously during the training phase, while the human body analysis branch and the local difference gating fusion process are turned off during the inference phase.

[0084] In a preferred embodiment, both the original visible light branch and the human body resolution branch use ResNet-50 as the backbone network, initialized with ImageNet pre-trained weights. This backbone network outputs feature maps with spatial layout information, and branch descriptors are obtained through adaptive pooling and batch normalization bottleneck layers. In one embodiment using ResNet-50, the number of feature channels is 2048; alternatively, VGG, DenseNet, Visual Transformer, or other feature extraction networks capable of outputting spatial feature maps can be used as the backbone network.

[0085] The enhanced original visible light image is input into the original visible light branch to obtain the original feature map; the corresponding human body analysis image is input into the human body analysis branch to obtain the structural feature map. The original feature map and the structural feature map are represented as follows:

[0086] (1)

[0087] in, This represents the original visible light pedestrian image. Indicates and One-to-one correspondence of human body analysis diagrams; This represents the feature extraction mapping of the original visible light branch. This represents the feature extraction mapping of the human body parsing branch; This represents the original feature map extracted from the original visible light pedestrian image by the original visible light branch. This represents the structural feature map extracted by the human body analysis branch from the human body analysis map corresponding to the same pedestrian image. This represents the number of feature channels used for identity verification. and These represent the feature map height and feature map width corresponding to the spatial layout of a pedestrian's body, respectively.

[0088] Step S4: Local region division and local difference estimation.

[0089] Original feature map and structural feature diagram Divided along the height direction corresponding to the longitudinal layout of the pedestrian's body into The nth horizontal local region is obtained. Original local features and structural local features of a local region:

[0090] (2)

[0091] in, In order to make Positive integers that are divisible by , Indicates the first The feature map height of a horizontal local region. In an embodiment determined by experiments with the number of partitions, Take 12; those skilled in the art can also choose other divisible values ​​based on the height of the backbone network output feature map and the validation set performance. Positive integers.

[0092] For the Spatial location in a local area The local difference value is calculated from the channel vector at a given location to form a local difference map. :

[0093] (3)

[0094] in, Indicates the first Spatial location in a horizontal local area The local difference value between the original features and structural features at a given location; Represents the inner product along the channel dimension. Represents the L2 norm, This represents the numerical stability constant to prevent division by zero. A larger local difference value indicates a greater difference between the appearance features extracted from the original visible light branch and the structural features extracted from the human body analysis branch at that spatial location.

[0095] Step S5: Dense gated matrix generation and local residual fusion.

[0096] Based on the local difference diagram A dense gating matrix is ​​generated, and local residual fusion is performed based on the dense gating matrix. First, the local difference values ​​are mapped to gating weights using a Sigmoid function with a temperature parameter:

[0097] (4)

[0098] (5)

[0099] in, Indicates the first Dense gated matrices for local regions Indicates the spatial location of this local area. Gating weights for analyzing the structural features of the human body; This represents the Sigmoid activation function. Indicates the gate temperature parameter. This represents the learnable variable used to generate the gated temperature parameters. This represents a constant that ensures the gated temperature parameter is positive and remains numerically stable.

[0100] Using dense gating matrix to the first The structural features of each local region are weighted and then fused into the corresponding original features using residual injection to obtain the local fused features:

[0101] (6)

[0102] in, This indicates element-wise multiplication and broadcasting along the channel dimension. Indicates the first This method utilizes local fusion features in specific regions. The residual fusion primarily employs the original visible light features, injecting human body structural features as auxiliary residual information, thereby enhancing the stability of the human body structure while preserving the details of appearance discrimination.

[0103] Step S6: Local reconstruction and global fusion.

[0104] To avoid the loss of overall context in isolated local descriptors, this embodiment will... Local fusion features Replace with the original feature map The corresponding local region is obtained by preserving the original local features of other local regions, thus obtaining the first... One local reconstructed feature map:

[0105] (7)

[0106] in, This indicates a stitching operation along the height direction of the feature map. Indicates only in the first The residual information of the human body structure is introduced into each horizontal local region, while preserving the overall human body spatial layout. A local reconstructed feature map.

[0107] Simultaneously, global fusion weights are calculated based on the dense gating matrices of each local region, and a global fusion feature map is obtained based on these weights.

[0108] (8)

[0109] (9)

[0110] in, This represents the global fusion weight obtained from the mean of the dense gating matrices of each level's local region. This represents the mean operation. This represents the global fused feature map.

[0111] Step S7: Generate fusion descriptors.

[0112] For each locally reconstructed feature map, adaptive pooling, flattening, and independent bottleneck layer mapping are performed to obtain a local fusion descriptor; for the global fusion feature map, adaptive pooling, flattening, and bottleneck layer mapping are performed to obtain a global fusion descriptor.

[0113] (10)

[0114] (11)

[0115] in, Indicates adaptive pooling. Indicates the flattening operation. Indicates the first The bottleneck layer corresponding to each locally reconstructed feature map This represents the bottleneck layer corresponding to the global fusion feature map. Indicates the first A local fusion descriptor, This represents the global fusion descriptor.

[0116] Simultaneously, the original visible light branch descriptor and the human body analytical branch descriptor are generated from the original feature map and the structural feature map, respectively:

[0117] (12)

[0118] in, and These represent the bottleneck layers corresponding to the original visible light branch and the human body analysis branch, respectively. This represents the original visible light branch descriptor obtained from the original feature map. This represents the human body parsing branch descriptor obtained from the structural feature map. Therefore, the descriptors used for supervision during the training phase include the global fusion descriptor, A local fusion descriptor, a raw visible light branch descriptor, and a human body parsing branch descriptor.

[0119] Step S8: Construction of hierarchical contrastive loss and model training.

[0120] To constrain cross-branch consistency between the original visible light branch and the human body resolution branch, this embodiment constructs a hierarchical contrastive loss. Training batches are constructed using an identity-based sampling method, and each training batch contains... Each identity contains [number] identities, and each identity contains [number] identities. A variety of image instances, The training batch size is represented as:

[0121] (13)

[0122] Different image instances include raw visible light pedestrian images and their corresponding human anatomy diagrams captured under different clothing, acquisition times, and / or camera perspectives for the same identity. Sampling by identity ensures that both strong positive samples across branches for the same identity and weak positive samples across branches for different image instances of the same identity exist within the same training batch.

[0123] Let the normalized descriptor of the original visible light branch output be... The normalized descriptor output by the human body parsing branch is , No. The identity label of each sample is Where i,j∈{1,…,B}, and B represents the training batch size. Cross-branch similarity matrix. Represented as:

[0124] (14)

[0125] in, Indicates the first The original visible light branch descriptor and the first Cross-branch similarity between individual body parsing branch descriptors. This represents the temperature coefficient for comparative learning.

[0126] Construct an identity adjacency matrix Strong positive sample mask and weak positive sample mask :

[0127] (15)

[0128] (16)

[0129] in, Represents the identity adjacency matrix. express An identity matrix of order 1. A cross-branch strong positive sample mask representing the same image instance. A cross-branch weak positive sample mask representing different image instances with the same identity.

[0130] Constructing a hierarchical soft target matrix And normalize it to obtain :

[0131] (17)

[0132] (18)

[0133] in, and Let represent the weights of strong positive samples and weak positive samples, respectively. This makes the cross-branch alignment constraint of the same image instance stronger than the consistency constraint of different image instances with the same identity; This represents the numerical stability constant in the normalization of the hierarchical soft objective matrix. In an embodiment determined by parametric analysis experiments, Take 1.0, Take 0.3, Take 0.1.

[0134] For sample pairs with different identity labels If the value is 0, samples with different identity labels are not assigned soft target weights as positive samples, but their similarity terms still participate in the Softmax normalization denominator of the similarity matrix, thus constraining the network to learn identity-distinguishing features as negative samples of different identities.

[0135] Layered contrast loss includes contrast loss along the original visible light branch to the human body resolution branch and contrast loss along the human body resolution branch to the original visible light branch:

[0136] (19)

[0137] (20)

[0138] (twenty one)

[0139] in, This represents the contrastive loss using the original visible light branch descriptor as the query and the human body parsing branch descriptor as the matching object. This represents the contrast loss using the human body parsing branch descriptor as the query and the original visible light branch descriptor as the matching object. This represents the hierarchical contrast loss; the Softmax normalized denominators in both directions include cross-branch similarity terms for all identity samples within the same batch.

[0140] During training, identity classification loss and triplet loss are also applied to the original visible light branch descriptor, human parsing branch descriptor, global fusion descriptor, and local fusion descriptor. The supervised feature set and the total training loss are expressed as:

[0141] (twenty two)

[0142] (twenty three)

[0143] in, This represents the original visible light branch descriptor, the human body parsing branch descriptor, the global fusion descriptor, and... A set of supervised features consisting of local fusion descriptors; Representation and Supervision Descriptors The corresponding identity category header, Indicates identity label; This represents the identity classification loss used to constrain pedestrian identity category determination. This represents the triplet loss used to constrain triplets where the distance to the same identity feature is less than the distance to different identity features. , and These represent the loss weights for identity classification, triplet loss, and hierarchical comparison loss, respectively.

[0144] In a preferred embodiment, the model is trained using the Adam optimizer with weight decay of . The training rounds are 80. The weights for both identity classification loss and triplet loss are set to 1. A warm-up strategy can be used for the stratified comparison loss weights, setting smaller values ​​in the first few rounds and larger values ​​thereafter to stabilize cross-branch alignment training. The above training parameters are configurations determined in the experimental embodiment; those skilled in the art can make routine adjustments based on the training set size and validation set performance.

[0145] The parameters mentioned above, such as the number of partitions, contrastive learning temperature coefficient, gating temperature parameter, strong positive sample weight, weak positive sample weight, and weights of each training loss, can be determined through parameter analysis experiments combined with the training set size and validation set performance. The specific parameter values ​​mentioned above are used to illustrate an implementation configuration that can realize the technical solution of this embodiment. Those skilled in the art can adjust the corresponding parameters according to the actual dataset, backbone network structure, and training conditions.

[0146] Step S9: Single-branch feature extraction during the inference stage.

[0147] After model training is complete, the human body analysis model is not invoked during the inference phase, no human body analysis image is input, and the human body analysis branch and the difference-gated local fusion module are disabled, retaining only the original visible light branch after training. The query image and candidate library images are input into the original visible light branch respectively to obtain the query image features and candidate library image features.

[0148] Step S10: Similarity calculation and pedestrian re-identification.

[0149] Calculate the cosine similarity between the query image features and the candidate image features:

[0150] (twenty four)

[0151] in, Indicates querying image features, Represents the image features of the candidate library. This represents the cosine similarity between the query image and the candidate images in the database. The candidate images are sorted from highest to lowest cosine similarity, and the top-ranked candidate pedestrian images are output as the re-identification results for the target pedestrian.

[0152] As can be seen from the above implementation, the present invention introduces human body analysis map as structural prior during the training phase, uses difference-gated local fusion module to realize local adaptive fusion of original appearance features and human body structural features, and establishes instance-level and identity-level consistency constraints through hierarchical contrast loss. This can reduce the interference of clothing changes on pedestrian identity feature learning and improve the robustness and discriminability of pedestrian identity representation in clothing changing scenarios.

[0153] Meanwhile, this invention retains only the original visible light branch during the inference stage, without relying on additional human body analysis branches and fusion modules. It can complete pedestrian retrieval without adding structural analysis and dual-branch fusion calculations during the inference stage, and has good practical deployment applicability.

[0154] A computer-storable medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for re-identifying pedestrians in disguise based on human appearance analysis.

[0155] Embodiments of the present invention may be provided as methods, systems, or computer program products. The invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the invention may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0157] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0158] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0159] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described method for re-identifying pedestrians in disguise based on human appearance analysis.

[0160] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0161] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for re-identifying pedestrians in disguise based on human appearance analysis, characterized in that, include: S1. Collect raw visible light images of several target pedestrians respectively, construct a training image set and a test image set with identity labels, and perform image enhancement on the raw visible light images in the training image set. S2. Perform semantic parsing of human body parts on the original visible light images in the training image set to obtain a human body parsing image set that corresponds one-to-one with the original visible light images. S3. Construct an asymmetric dual-path feature extraction network that includes an original visible light branch and a human body analysis branch. Input the enhanced original visible light image into the original visible light branch to obtain the original feature map, and input the human body analysis image into the human body analysis branch to obtain the structural feature map. S4. Divide the original feature map and the structural feature map into multiple local regions according to the same region division method, and calculate the difference between the original features and structural features in the corresponding local regions to obtain the local difference map; S5. Generate a dense gating matrix based on the local difference map, use the dense gating matrix to weight the local structural features in the structural feature map, and fuse the weighted local structural features into the corresponding original features by residual injection to obtain the local fused features. S6. Replace each local fusion feature back with the corresponding region of the original feature map to obtain multiple local reconstructed feature maps, and generate global fusion weights based on the dense gating matrix of each local region to obtain a global fusion feature map. S7. Pooling and bottleneck layer mapping are performed on multiple local reconstructed feature maps and global fusion feature maps respectively to obtain local fusion descriptors and global fusion descriptors; S8. Construct a hierarchical contrast loss based on the hierarchical relationship between strong positive samples across branches of the same image instance, weak positive samples across branches of different image instances of the same identity, and negative samples of different identities, and train the asymmetric dual-path feature extraction network by combining identity classification loss and triplet loss. S9. Input the original visible light image to be tested from the set of images to be tested into the trained original visible light branch, close the human body analysis branch and the local difference gating fusion process, and extract the features of the image to be tested using only the original visible light branch. S10. Sort the candidate pedestrian images according to the cosine similarity between the features of the images to be tested, and obtain the re-identification result of the target pedestrian.

2. The method for re-identifying pedestrians in disguise based on human appearance analysis as described in claim 1, characterized in that, In S2, each human body image in the human body image set represents different parts of the human body with different pixel values ​​or color codes, which is used to provide prior information on the layout, contour structure and spatial location of human body parts; the network used for semantic parsing of human body parts keeps its parameters frozen during the training of the asymmetric dual-path feature extraction network.

3. The method for re-identifying pedestrians in disguise based on human appearance analysis as described in claim 1, characterized in that, In S3, the original visible light branch and the human body analysis branch obtain the original feature map and structural feature map respectively through the corresponding feature extraction mapping. The original feature map and structural feature map are represented as follows: in, This represents the original visible light pedestrian image. Indicates and One-to-one correspondence of human body analysis diagrams; This represents the feature extraction mapping of the original visible light branch. This represents the feature extraction mapping of the human body parsing branch; This represents the original feature map extracted from the original visible light pedestrian image by the original visible light branch. This represents the structural feature map extracted by the human body analysis branch from the human body analysis map corresponding to the same pedestrian image. This represents the number of feature channels used for identity verification. and These represent the feature map height and feature map width corresponding to the spatial layout of a pedestrian's body, respectively.

4. The method for re-identifying pedestrians in disguise based on human appearance analysis as described in claim 3, characterized in that, In S4, the original feature map and structural feature diagram Divided along the height direction corresponding to the longitudinal layout of the pedestrian's body into The nth horizontal local region is obtained. Original local features of a local region and structural local features ,in , , , In order to make Positive integers that are divisible by 0; For the Spatial location in a local area The local difference value is calculated from the channel vector at a given location to form a local difference map. , i∈{1,…,H k },j∈{1,…,W},D (k) ∈R Hk×W ; expressed as a formula: in, Indicates the first Spatial location in a horizontal local area The local difference value between the original features and structural features at a given location; Represents the inner product along the channel dimension. Represents the L2 norm, This represents the numerical stability constant to prevent division by zero.

5. The method for re-identifying pedestrians in disguise based on human appearance analysis as described in claim 4, characterized in that, In S5, a dense gating matrix is ​​generated based on the local difference map, and local residual fusion is performed based on the dense gating matrix, as expressed by the formula: in, Indicates the first Dense gated matrices for local regions Indicates the spatial location of this local area. Gating weights for analyzing the structural features of the human body; This represents the Sigmoid activation function. Indicates the gate temperature parameter. This represents the learnable variable used to generate the gated temperature parameters. This represents a constant that ensures the gated temperature parameter remains positive and its value is stable. This indicates element-wise multiplication and broadcasting along the channel dimension. Indicates the first Local fusion characteristics of a local region.

6. The method for re-identifying pedestrians in disguise based on human appearance analysis as described in claim 5, characterized in that, In S6, the first Local fusion features Replace with the original feature map The corresponding region is obtained by preserving other local regions as their original local features, thus obtaining the first... Local reconstruction feature map This can be expressed as a formula: in, This indicates a stitching operation along the height direction of the feature map. Indicates only in the first The residual information of the human body structure is introduced into each horizontal local region, while preserving the overall human body spatial layout. A local reconstructed feature map.

7. The method for re-identifying pedestrians in disguise based on human appearance analysis as described in claim 6, characterized in that, In S6 and S7, global fusion weights are calculated based on the dense gating matrices of each local region to obtain a global fusion feature map. Descriptors for supervised training are then generated from the local reconstructed feature map, the global fusion feature map, the original feature map, and the structural feature map, respectively, as expressed by the formula: in, This represents the global fusion weight obtained from the mean of the dense gating matrices of each level's local region. Represents the global fused feature map. This represents the average value across all spatial locations of a dense gated matrix. Indicates adaptive pooling. Indicates the flattening operation. Indicates the first The bottleneck layer corresponding to each locally reconstructed feature map This represents the bottleneck layer corresponding to the global fusion feature map. and These represent the bottleneck layers corresponding to the original visible light branch and the human body analysis branch, respectively. Indicates the first A local fusion descriptor, Represents the global fusion descriptor. This represents the original visible light branch descriptor obtained from the original feature map. This represents the human body parsing branch descriptor obtained from the structural feature map.

8. The method for re-identifying pedestrians in disguise based on human appearance analysis as described in claim 1, characterized in that, In S8, when constructing the hierarchical contrastive loss, training batches are sampled according to identity, and each training batch contains Each identity contains [number] identities, and each identity contains [number] identities. A variety of image instances, Training batch size ; Let the normalized descriptor of the original visible light branch output be... The normalized descriptor output by the human body parsing branch is , No. The identity label of each sample is Cross-branch similarity matrix Identity adjacency matrix Strong positive sample mask Weak positive sample mask and hierarchical soft target matrix Where i,j∈{1,…,B}, they are expressed by the following formulas: in, Indicates the first The original visible light branch descriptor and the first Cross-branch similarity between individual body parsing branch descriptors. Indicates the temperature coefficient for comparative learning; Represents the identity adjacency matrix. express An identity matrix of order 1. A cross-branch strong positive sample mask representing the same image instance. A cross-branch weak positive sample mask representing different image instances with the same identity; and Let represent the weights of strong positive samples and weak positive samples, respectively. , This represents the numerical stability constant in the normalization of the hierarchical soft target matrix; This represents the normalized hierarchical soft target matrix.

9. The method for re-identifying pedestrians in disguise based on human appearance analysis as described in claim 1, characterized in that, In S8, the layered contrast loss includes the loss from the original visible light branch to the human body resolution branch and the loss from the human body resolution branch to the original visible light branch, expressed by the formula: in, This represents the contrast loss from the original visible light branch to the human body resolution branch. This represents the contrast loss from the human body resolution branch to the original visible light branch. The stratified comparison loss is represented by the Softmax normalized denominator in both directions, which includes cross-branch similarity terms for all identity samples in the same batch. Represents the normalized hierarchical soft target matrix The transpose of .

10. The method for re-identifying pedestrians in disguise based on human appearance analysis as described in claim 1, characterized in that, In S8, the total training loss of the asymmetric dual-path feature extraction network consists of identity classification loss, triplet loss, and hierarchical contrastive loss, expressed by the formula: in, This represents the original visible light branch descriptor, the human body parsing branch descriptor, the global fusion descriptor, and... A set of supervised features consisting of local fusion descriptors; Representation and Supervision Descriptors The corresponding identity category header, Indicates identity label; This represents the identity classification loss used to constrain pedestrian identity category determination. This represents the triplet loss used to constrain triplets where the distance to the same identity feature is less than the distance to different identity features. Indicates the loss due to layered contrast; This indicates that the hierarchical comparison loss weight, identity classification loss weight, and triplet loss weight are set to 1.

Images