Pedestrian re-identification method, system and device based on identity and pose feature decoupling

Abstract

The application provides a pedestrian re-identification method, system and device based on identity and posture feature decoupling, comprising: performing fine-grained posture label allocation on the extended pedestrian posture image data to obtain a training set with posture labels; training an identity and posture feature decoupling model, calculating the deep feature similarity of a to-be-identified pedestrian image and a target pedestrian image, selecting the image with the maximum similarity as a re-identification result output, and realizing the re-identification of the pedestrian. The method can process and expand a large amount of pedestrian posture image data, provide rich training samples for the model, and thus show stronger recognition capability when facing new data or unknown scenes, and solve the problem of low pedestrian re-identification accuracy caused by the influence of pedestrian posture changes, that is, the pedestrian images with different postures from the same identity have great visual appearance differences, and finally result in low pedestrian re-identification accuracy.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology and relates to a pedestrian re-identification method, system and device based on the decoupling of identity and posture features. Background Technology

[0002] With the acceleration of urbanization and the widespread application of intelligent surveillance systems, pedestrian information in surveillance videos has become an important component of massive amounts of data. However, with the continuous increase in the number of surveillance cameras and the expansion of the monitoring range, the amount of personnel information involved in surveillance videos is growing exponentially. This explosive growth in personnel information poses a severe challenge to related technologies, leading to serious information overload problems. In surveillance videos, the identification and tracking of personnel is crucial for public safety and urban management. However, traditional surveillance systems often cannot effectively process massive amounts of personnel information, making it difficult to identify and track personnel in surveillance videos in a timely and accurate manner. Therefore, quickly and accurately extracting the identity information of target personnel from massive amounts of surveillance video has become an urgent problem to be solved, and pedestrian re-identification (Re-ID) technology has emerged as a result.

[0003] In real-world scenarios, Re-ID still faces numerous challenges. These challenges primarily stem from the complex environments in surveillance videos and the characteristics of pedestrians themselves: 1) Viewpoint variations: Different camera positions and angles cause the same pedestrian to appear differently in different camera images due to variations in the shooting angle and viewpoint; 2) Posture variations: Pedestrian postures may change during walking, such as standing, walking, or bending over. These posture variations lead to inconsistent posture characteristics of the same pedestrian in different images; 3) Low image resolution: Surveillance cameras may be limited by factors such as equipment quality and installation location, resulting in low-resolution pedestrian images. The challenges include: 4) Lighting variations: Illumination conditions in surveillance scenes may change over time and with the environment, resulting in varying brightness and contrast in pedestrian images under different lighting conditions, increasing the difficulty of pedestrian re-identification; 5) Occlusion: Sightings in surveillance videos may involve pedestrians being occluded by other objects or people, causing some pedestrian features to be invisible or blurred, thus affecting the accuracy of pedestrian identification and matching; 6) Heterogeneous modalities: Surveillance videos may simultaneously contain image data in different modalities, such as RGB and infrared images. The feature differences and transformations between different modalities increase the complexity of pedestrian re-identification tasks. These challenges introduce numerous variations and uncertainties. Addressing these challenges is crucial for improving the accuracy and robustness of pedestrian re-identification technology, and can provide more reliable and effective solutions for fields such as urban security and intelligent surveillance systems.

[0004] Depending on the research focus, Re-ID research can be divided into standard person re-identification and open-scene person re-identification. Standard person re-identification generally makes the following assumptions: 1) the image or video was captured by a single-mode visible camera; 2) each pedestrian is represented by a bounding box (i.e., the upstream object detection task is sufficiently accurate); 3) there is sufficient labeled training data for supervising the learning of the discriminative Re-ID model; 4) the labels are usually correct; 5) the person being queried must appear in the image set. Standard person re-identification usually focuses more on the feature matching of the pedestrians themselves and ignores many practical problems in open scenes (cross-modal, noise, unsupervised, etc.). Correspondingly, open-scene person re-identification, based on standard person re-identification research, further considers practical problems in specific scenarios, including cross-modal person re-identification, noise-robust person re-identification, and unsupervised person re-identification. Zheng et al. first attempted to apply generative adversarial networks to person re-identification, improving the supervised feature representation learning based on generated person images. Liu et al. added pose constraints to the model to improve the quality of generated person images and generate person images with new pose variables. Qian et al. designed a pose-normalized image generation method to enhance robustness to pose variations. Camera style information was also integrated into the image generation process to address cross-camera variations, and a joint discriminative and generative learning model was proposed to learn appearance codes and structure codes respectively to improve image generation quality.

[0005] In summary, current technologies suffer from the problem that pedestrian images of people with the same identity but different postures have significant visual differences due to changes in pedestrian posture, ultimately leading to low accuracy in pedestrian re-identification. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a pedestrian re-identification method, system, and device based on the decoupling of identity and posture features. This solves the problem that in real-world scenarios, the changes in pedestrian posture lead to significant visual differences in pedestrian images with different postures from the same identity, ultimately resulting in low accuracy of pedestrian re-identification.

[0007] This invention is achieved through the following technical solution:

[0008] A pedestrian re-identification method based on the decoupling of identity and pose features includes,

[0009] Obtain pedestrian pose image data and expand it. Then, perform fine-grained pose label assignment on the expanded pedestrian pose image data to obtain a training set with pose labels.

[0010] The identity and pose feature decoupling model is trained based on a training set with pose labels;

[0011] Based on the trained identity and pose feature decoupling model, the pose feature classification head is removed, the depth feature similarity between the pedestrian image to be identified and the target pedestrian image is calculated, and the image with the highest similarity is selected as the re-identification result output to achieve pedestrian re-identification.

[0012] Preferably, the pedestrian pose image data is expanded, specifically as follows:

[0013] The pedestrian pose image data is horizontally flipped, which doubles the size of the pedestrian pose image dataset.

[0014] Preferably, fine-grained pose label assignment is performed on the expanded pedestrian pose image data to obtain a training set, specifically as follows:

[0015] A 2D human pose estimation model is used to extract pose skeleton keypoints from each pedestrian image in the expanded pedestrian pose image data. The K-means clustering algorithm is used to perform cluster analysis on the pose skeleton keypoints, and pose skeleton keypoints with similar attributes are aggregated into the same cluster. Each cluster represents a pose label. By traversing the entire pedestrian pose image dataset, a corresponding pose label is assigned to each pedestrian pose image, thereby obtaining a training set with pose labels.

[0016] Preferably, a decoupled model for identity and pose features is constructed, specifically as follows:

[0017] The identity and pose feature decoupling model includes L feature decoupling blocks, a pose feature classification head, and an identity feature classification head. Each feature decoupling block consists of an identity feature extractor, a pose feature extractor, and a cross-attention module.

[0018] The identity feature extractor and pose feature extractor are used to learn shallow identity features and pose features to extract deep identity features and pose features, respectively. The cross attention module is used to decouple deep identity features and pose features and remove noise from them.

[0019] The identity feature extractor uses a deep learning network or a Re-ID network as its basic structure;

[0020] Preferably, the training process for the decoupled identity and pose feature model is as follows:

[0021] Based on a training set with pose labels, Bag of Tricks, AGW, and RGA are used as identity feature extractors and pose feature extractors for the identity and pose feature decoupling model, respectively. The performance of the identity and pose feature decoupling model is compared with the corresponding identity feature extractors and pose feature extractors. The identity and pose feature decoupling model is trained in an end-to-end manner using identity loss and pose loss to learn the identity and pose representation of pedestrians.

[0022] Specifically, for identity feature representation, labeled smooth cross-entropy loss, weighted regularized triplet loss, and center loss are used for identity recognition.

[0023] Preferably, during the training process, the input pedestrian pose image data undergoes data augmentation processing including normalization, random erasure, and random cropping.

[0024] Preferably, reasoning is performed based on a trained decoupled model of identity and pose features to achieve pedestrian re-identification, specifically as follows:

[0025] The pose feature classification head in the identity and pose feature decoupling model is removed. Using the identity feature, the depth features of the pedestrian image to be identified and the target pedestrian image are calculated separately. The similarity between the pedestrian image to be identified and the target pedestrian image is obtained based on the similarity between the distances of the depth features of the pedestrian images to be identified and the target pedestrian images. The pedestrian images to be identified are sorted in descending order according to the similarity between the pedestrian images to be identified and the target pedestrian images. The pedestrian image with the highest similarity is taken as the output result, thus completing the re-identification of the pedestrian.

[0026] A pedestrian re-identification system based on decoupling of identity and posture features includes,

[0027] Data acquisition module, model training module, and pedestrian re-identification module;

[0028] The data acquisition module is used to acquire pedestrian posture image data and expand it, and to perform fine-grained posture label allocation on the expanded pedestrian posture image data to obtain a training set with posture labels.

[0029] The model training module is used to train the identity and pose feature decoupling model based on a training set with pose labels;

[0030] The pedestrian re-identification module is used to remove the pose feature classification head based on the trained identity and pose feature decoupling model, calculate the depth feature similarity between the pedestrian image to be identified and the target pedestrian image, and select the image with the highest similarity as the re-identification result output, thereby realizing the re-identification of pedestrians.

[0031] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the pedestrian re-identification method based on identity and posture feature decoupling.

[0032] A computer-readable storage medium storing a computer program, characterized in that, when executed by a processor, the computer program implements the steps of the pedestrian re-identification method based on identity and posture feature decoupling.

[0033] Compared with the prior art, the present invention has the following beneficial technical effects:

[0034] This invention provides a pedestrian re-identification method, system, and device based on the decoupling of identity and pose features. Addressing the impact of pedestrian pose and viewpoint changes on pedestrian representation, the method acquires and expands pedestrian pose image data, then assigns fine-grained pose labels to the expanded image data, obtaining a training set with pose labels. This method provides detailed pose information for each pedestrian pose image through fine-grained pose label assignment. This allows the model to learn more detailed and accurate pose features during training, thus enabling more accurate matching of pedestrian poses during re-identification and improving re-identification accuracy. The decoupled model of identity and pose features is trained based on the pose-labeled training set; the decoupling of identity and pose features allows the model to focus on pedestrian identity information and pose information separately. This decoupling design enables the model to more stably extract pedestrian identity features when facing complex scenarios such as pose changes, occlusion, or viewpoint changes, thereby enhancing the model's robustness. Based on a pre-trained identity and pose feature decoupling model, the pose feature classification head is removed, and the depth feature similarity between the pedestrian image to be identified and the target pedestrian image is calculated. The image with the highest similarity is selected as the re-identification result, thus achieving pedestrian re-identification. In the re-identification stage, by removing the pose feature classification head, the model only needs to calculate the depth feature similarity between the pedestrian image to be identified and the target pedestrian image, without needing to consider pose feature matching. This greatly simplifies the calculation process and improves the computational efficiency of re-identification. This method can process and expand large amounts of pedestrian pose image data, providing rich training samples for the model. This allows the model to learn more generalized feature representations, thus exhibiting stronger recognition capabilities when facing new data or unknown scenarios. It solves the problem that in real-world scenarios, due to the influence of pedestrian pose variations, pedestrian images with different poses from the same identity have huge visual appearance differences, ultimately leading to low pedestrian re-identification accuracy.

[0035] Furthermore, the pedestrian re-identification method based on the decoupling of identity and posture features in this invention can not only improve the accuracy and robustness of re-identification, but also improve computational efficiency, support training on large-scale datasets, promote multi-task learning and application, and enhance user experience and security. The method proposed in this invention is of great significance for improving the accuracy and robustness of pedestrian re-identification technology, and can provide more reliable and effective solutions for fields such as urban security and intelligent monitoring systems.

[0036] Furthermore, this invention constructs a progressive feature decoupling model based on cross-attention, which combines the attention mechanism and pose guidance end-to-end.

[0037] Furthermore, the identity and pose feature decoupling module consists of an identity and pose feature extractor and a cross-attention module. The cross-attention module learns the similarity between identity features and pose features and removes these similar parts from the original features. Through the progressive decoupling of several feature decoupling modules, the influence between the final output identity features and pose features is minimized.

[0038] Furthermore, the identity feature extractor uses a deep learning network or a Re-ID network as its basic structure, which improves the robustness of identity features;

[0039] Furthermore, the cross-attention module removes pose noise in identity features and identity noise in pose features to a certain extent, allowing identity features and pose features to influence and promote each other during the learning process, thus becoming discriminative together. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart of a specific implementation of the pedestrian re-identification method based on the decoupling of identity and posture features according to the present invention;

[0042] Figure 2 This is a schematic diagram of the fine-grained posture tag allocation of the present invention;

[0043] Figure 3 This is a framework diagram of the identity and posture feature decoupling model of the present invention;

[0044] Figure 4 This is a network architecture diagram of the cross-attention module of the present invention;

[0045] Figure 5 Figure (a) shows the training results of the model in the example on the Market1501 dataset, and Figure (b) shows the loss result and the accuracy result. Detailed Implementation

[0046] The present invention will be further described in detail below with reference to specific embodiments. These descriptions are for explanation purposes only and are not intended to limit the scope of the invention.

[0047] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0048] A pedestrian re-identification method based on the decoupling of identity and pose features includes the following steps:

[0049] S1: Assign fine-grained pedestrian pose labels;

[0050] S2: Construct a decoupled model for identity and pose features;

[0051] S3: Train the identity and pose feature decoupling model;

[0052] S4: Pedestrian re-identification model inference.

[0053] A further improvement of the present invention is that:

[0054] In step S1, existing methods typically perform random horizontal flipping of input images with a certain probability during training for data augmentation. However, in this invention, since pose labels are given before training, if an image happens to be horizontally flipped, it will cause a mismatch between the pose label and the image, i.e., introduce additional noise. To eliminate this noise without discarding the horizontal flipping data augmentation method, this invention pre-flips all images horizontally before assigning pose labels, effectively doubling the size of the dataset. Fine-grained pose label assignment is then performed on this expanded dataset to obtain the training set.

[0055] In step S2, the model consists of L feature decoupling blocks and two classification heads. Each feature decoupling block comprises an identity feature extractor, a pose feature extractor, and a cross-attention module. The identity feature extractor and pose feature extractor are used to learn deeper and more abstract identity and pose features, respectively, while the cross-attention module focuses on decoupling identity and pose features without changing the depth (level of abstraction) of the features. To extract robust identity features, the identity feature extractor is based on a typical deep learning network (ResNet or DenseNet) or a mature Re-ID network, while the pose feature extractor uses the same or a similar network structure. This allows the model to be flexible and not limited to a specific feature extraction network, and future superior deep learning networks or Re-ID models can still be further extended based on the model of this invention.

[0056] Furthermore, the main idea of ​​the proposed cross-attention module is as follows: Given two intermediate feature tensors with width W, height H, and C channels, this invention treats the c-dimensional feature vector at each spatial location as a feature node. For each feature tensor, all spatial locations form a graph G with N = H × W nodes. This invention represents all feature nodes as... Where i, p ∈ {1, 2, L, N}. Pose feature node x p For identity feature node x i The effect can be defined as the dot product affinity in the embedding space, as follows:

[0057] a i,p =f ip (x i ,x p )=θ id (x i ) T γ po (x p )

[0058] Where θ id and γ po These are two embedding functions, x, implemented by a convolutional layer with a 1×1 kernel. i For x p The effect is expressed as

[0059] a p,i =f pi (x p ,x i )=θ po (x p ) T γ id (x i )

[0060] in and Based on the affinity matrix, attention weights can be defined as follows:

[0061]

[0062]

[0063] Among them, s i,p x represents p For x i Influence weights, s p,i x represents i For x p Influence weights, The cross-attention module removes pose noise in identity features and identity noise in pose features to a certain extent, allowing identity features and pose features to influence and promote each other during the learning process, thus becoming discriminative together.

[0064] In step S3, this invention utilizes identity loss and pose loss to learn pedestrian identity and pose representations in an end-to-end manner. For identity feature representation, this invention selects labeled smoothed cross-entropy loss, weighted regularized triplet loss, and center loss for identity recognition. To better demonstrate the model's performance, this invention selects Bag of Tricks, AGW, and RGA from recent papers published at top conferences in the field of pedestrian re-identification as identity feature extractors for the model, using the output f of five residual modules. id and f po The input images are used as input to the cross-attention module, and their output is then used as input to subsequent residual blocks. This invention horizontally flips all input images, doubling the size of the training set, and then assigns pose labels to all training set images using a fine-grained pose labeling method. During training, the input images are resized to 256×128 and then augmented using normalization, random erasure, and random cropping. Test images are resized to 256×128 and augmented only using normalization.

[0065] In step S4, during the model inference stage, the entire pose feature classification head is directly removed. That is, for the output f of the last cross-attention module... id ,f po Directly discard f po Only f id The data is input into the identity classification head to obtain identity features. The model has already learned identity and pose feature representations during the training phase, so no identity or pose labels are needed during the inference phase; the model can adaptively complete the feature representation.

[0066] A pedestrian re-identification system based on decoupling of identity and posture features includes,

[0067] Data acquisition module, model training module, and pedestrian re-identification module;

[0068] The data acquisition module is used to acquire pedestrian posture image data and expand it, and to perform fine-grained posture label allocation on the expanded pedestrian posture image data to obtain a training set with posture labels.

[0069] The model training module is used to train the identity and pose feature decoupling model based on a training set with pose labels;

[0070] The pedestrian re-identification module is used to remove the pose feature classification head based on the trained identity and pose feature decoupling model, calculate the depth feature similarity between the pedestrian image to be identified and the target pedestrian image, and select the image with the highest similarity as the re-identification result output, thereby realizing the re-identification of pedestrians.

[0071] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can operate based on a pedestrian re-identification method decoupled from identity and posture features.

[0072] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the pedestrian re-identification method based on identity and gesture feature decoupling in the above embodiments.

[0073] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0075] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0076] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0077] Example 2

[0078] See Figure 1 This invention discloses a pedestrian re-identification method based on the decoupling of identity and posture features. The specific implementation process mainly includes: assigning fine-grained pedestrian posture labels, constructing an identity and posture feature decoupling model, and training the identity and posture feature decoupling model and the pedestrian re-identification model for inference. Detailed steps are shown below:

[0079] Step 1: Assign fine-grained pedestrian pose tags

[0080] See Figure 2 As shown. Given an original dataset. Where I i Represents the i-th image, This represents the identity label of the image. The OpenPose 2D human pose estimation model is used to extract the pose skeleton keypoints p for each pedestrian image. i =Φ(I i ), where p i Let Φ be a 36-dimensional vector, representing a 2D human pose estimation model. K-means clustering is used to group all pose skeleton points. Aggregated into M clusters Each cluster represents a pose label. For any image I i Its posture label is Finally, the input training set is obtained.

[0081] Existing methods typically perform random horizontal flipping of input images with a certain probability during training for data augmentation. However, in the method of this invention, since pose labels are given before training, if an image happens to be horizontally flipped, it will cause a mismatch between the pose label and the image, i.e., introduce additional noise. To eliminate this noise without discarding the horizontal flipping data augmentation method, this invention pre-flips all images horizontally before pose label assignment, which directly doubles the size of the dataset. Fine-grained pose label assignment is then performed on this expanded dataset to obtain the training set.

[0082]

[0083] Step 2: Construct an identity and pose feature decoupling model

[0084] See Figure 3 As shown, the model consists of L feature decoupling blocks and two classification heads. Each feature decoupling block comprises an identity feature extractor, a pose feature extractor, and a cross-attention module. The identity feature extractor and pose feature extractor are used to learn deeper and more abstract identity and pose features, respectively, while the cross-attention module focuses on decoupling identity and pose features without changing the depth (level of abstraction) of the features. To extract robust identity features, the identity feature extractor is based on a typical deep learning network (ResNet or DenseNet) or a mature Re-ID network, while the pose feature extractor uses the same or a similar network structure. This allows the model to be flexible and not limited to a specific feature extraction network, and future superior deep learning networks or Re-ID models can still be further extended based on the model of this invention.

[0085] Assuming FDB l-1 This represents the l-th feature decoupling block, and FDB l-1 The output is The output of the l-th feature decoupling block can then be expressed as:

[0086]

[0087] Specifically, the identity and pose feature extractor learns from shallower features to extract deeper features, and the cross-attention module decouples the deeper features to remove noise from the features, in the form of...

[0088]

[0089]

[0090] in It is a CNN model used for feature extraction, x id ,x po The output features of the identity and pose feature extractor are the identity and pose features in the Lth layer before noise is eliminated. As feature decoupling progresses, pose-related information in the identity features is gradually suppressed, and identity-related information in the pose features is also gradually suppressed, resulting in increasingly cleaner identity and pose features. The output features of the last feature decoupling block are then passed through a pooling layer to obtain the final identity feature f. id and posture features f po .

[0091] See Figure 4 As shown, in the cross-attention module proposed in this invention, given two intermediate feature tensors with width W, height H, and C channels, the c-dimensional feature vector at each spatial location is treated as a feature node. For each feature tensor, all spatial locations form a graph G with N = H × W nodes. This invention represents all feature nodes as... Where i, p ∈ {1, 2, L, N}. Pose feature node x p For identity feature node x i The effect can be defined as the dot product affinity in the embedding space, as follows:

[0092] a i,p =f ip (x i ,x p )=θ id (x i ) T γ po (x p )

[0093] Where θ id and γ po These are two embedding functions implemented by a convolutional layer with a 1×1 kernel, i.e. in h is a hyperparameter set to make the features learned by the attention mechanism more diverse; generally, h ≥ 1. Similarly, x i For x p The effect is expressed as

[0094] a p,i =f pi (x p ,x i )=θ po (x p ) T γ id (x i )

[0095] in and Based on the affinity matrix, attention weights can be defined as follows:

[0096]

[0097]

[0098] Among them, s i,p x represents p For x i Influence weights, s p,i x represents i For xp Influence weights, Intuitively, this invention uses A ip and A pi To construct the relationship between pedestrian identity features and posture features, the resulting matrix S ip and S pi This study reveals pose-related regions in identity features and identity-related regions in pose features, respectively. Through cross-attention, the spatial dependencies between any two locations in identity and pose features can be captured.

[0099] The features at all locations are summed with weights, and the attention weights are adaptively determined based on the feature similarity between corresponding two locations. Noise in the features is represented as...

[0100] δ id ′=S ip ×V id ,δ po ′=S pi ×V po

[0101] in, × represents matrix multiplication, V id V po It is the mapping of input features in the embedding space, that is, Where, ψ id and ψ po It is similar to θ id The two embedded functions, namely, in All δ id ′,δ po After concatenation and splicing, the characteristic noise δ with dimensions C×H×W was obtained. id ,δ po The final output of the cross-attention module is represented as

[0102] x id =x id -δ id

[0103] x po =x po -δ po

[0104] The cross-attention module removes pose noise in identity features and identity noise in pose features to a certain extent, allowing identity features and pose features to influence and promote each other during the learning process, thus becoming discriminative together.

[0105] Step 3: Train the identity and pose feature decoupling model

[0106] This invention utilizes identity loss and pose loss to learn pedestrian identity and pose representations in an end-to-end manner. For identity feature representation, this invention selects labeled smoothed cross-entropy loss, weighted regularized triplet loss, and center loss for identity recognition.

[0107] Cross-entropy loss can be expressed as:

[0108]

[0109] Where N is the size of mini_batch, and M is the number of pedestrian identity categories in the training set. It is the probability that the model predicts that sample i belongs to identity category c. It can be represented as

[0110]

[0111] Here, ε is a small constant used to encourage the model to be less confident about the training set. In this study, ε is set to 0.1.

[0112] The weighted regularized triplet loss can be expressed as

[0113]

[0114]

[0115] Where (i,j,k) represents the hardest triplet in each training batch. For anchor i, P i It is the corresponding positive set, N i It is a negative set. This represents the pairwise distance between positive and negative sample pairs. The weighted regularization described above inherits the advantages of optimizing the relative distance between positive and negative pairs, but it avoids introducing any additional margin parameters. The center loss simultaneously learns a center for the deep features of each class and penalizes the distance between the deep features and their corresponding class centers, thus compensating for the shortcomings of the triple loss.

[0116] The central loss function is expressed as

[0117]

[0118] Where f i id This represents the identity features of the i-th image. Indicates the first Feature centers for each category. The training objective for identity recognition can be expressed as:

[0119]

[0120] in Represent The weight.

[0121] Similarly, the training objective for pose recognition can be stated as:

[0122]

[0123] in Represent The weights. For triple loss using hard sample mining, each mini-batch must be arranged in a certain label order, so only one of identity recognition and pose recognition can use triple loss. This invention chooses to use it in identity recognition. Therefore, the final optimization objective of end-to-end training can be expressed as:

[0124]

[0125] Where β id ,β po The weights represent the loss of identity and the loss of posture.

[0126] To better demonstrate the effectiveness of this model, this invention selects BagofTricks, AGW, and RGA from recent papers published at top conferences in the field of person re-identification as identity feature extractors for FDCA, and compares the performance of this model with the corresponding identity feature extractors. This invention uses ResNet, ResNet(NL), and ResNet(RGA) to represent the backbone networks used by BagofTricks, AGW, and RGA, respectively. Taking ResNet as an example, it is used as the identity and pose feature extractor in this model, using the output f of five residual modules. id and f po The CAM is used as input, and its output is then used as input for subsequent residual blocks. Without loss of generality, similar settings are used for ResNet(NL) and ResNet(RGA), respectively. The values ​​are 1, 1, 0.0005. 1, 0.0005, β id ,β po The values ​​are 1 and 0.6.

[0127] This invention uses two well-known datasets, Market-1501 and CUHK03, as the training set for the model. The person re-identification images in these datasets come from two or more cameras, satisfying the cross-camera requirement of standard person re-identification tasks. This invention horizontally flips all input images, doubling the size of the training set, and then assigns pose labels to all training set images using a fine-grained pose labeling method. During training, the input image size is adjusted to 256×128, and then data augmentation is performed through normalization, random erasure, and random cropping. Test images are adjusted to 256×128 and augmented only through normalization. The backbone network is pre-trained on ImageNet. This invention uses the Adam optimizer with a learning rate of 0.0004 and a weight decay of 0.0005 to train the decoupled model. For different baselines, a learning rate strategy similar to the original model is adopted. Specifically, for decoupled models using ResNet and ResNet(NL) baselines, a learning rate warm-up strategy is employed for the first 5 epochs, and then reduced to a factor of 0.2 at epochs 15, 40, and 70, for a total of 120 epochs. For decoupled models using the ResNet(RGA) baseline, a learning rate warm-up strategy is employed for the first 10 epochs, and then reduced to a factor of 0.5 at epochs 40, 60, 80, 100, 120, 140, 160, 180, and 230, for a total of 300 epochs. This invention is trained on two RTX3090 GPUs with a batch size of 64. Each batch contains 16 identities, and each identity has 4 instances. All models are implemented using the PyTorch framework.

[0128] Step 4: Pedestrian Re-identification Model Inference

[0129] During the model inference phase, the entire pose feature classification head is directly removed. That is, for the output f of the last cross-attention module... id ,f po Directly discard f po Only f id The input is fed into the identity classification head to obtain identity features. The model has already learned identity and pose feature representations during the training phase, so no identity or pose labels are needed during the inference phase. The model can adaptively complete feature representation. Furthermore, the depth features of the image to be identified and the target image are calculated separately. The similarity between the depth features of the image to be identified and the target image is obtained based on the similarity between the depth features of the image to be identified and the target image. The images to be identified are sorted in descending order according to the similarity between the images to be identified and the target image. The image to be identified with the highest similarity is taken as the output result to complete the re-identification of pedestrians.

[0130] Furthermore, ablation experiments for person re-identification were conducted on Market-1501 and CUHK03(L) / CUHK03(D). Referring to Table 1, FDCA is the name of the person re-identification model proposed in this invention. In FDCA(*-*), the first term in parentheses represents the identity feature extractor, and the second term represents the pose feature extractor. R, N, and RG in parentheses after FDCA represent ResNet50, ResNet50(NL), and ResNet50(RGA), respectively. To approximate the BagofTricks setup, FDCA(RR) and ResNet50 use Trilardloss instead of WRT.

[0131] Table 1 shows the name feature table of the pedestrian re-identification model proposed in this invention using FDCA.

[0132]

[0133] On Market1501, FDCA(NN) and FDCA(RR) significantly outperform the corresponding identity feature extractors, with improvements of 6.1% and 6.0% for mINP, 2.4% and 2.8% for mAP, and 1.6% and 1.0% for Rank-1, respectively. On CUHK03(L), ​​FDCA(NN), FDCA(RR), and FDCA(RGA-RGA) significantly outperform the corresponding identity feature extractors, outperforming by 13.4%, 16.1%, and 1.0% for mINP, 12.2%, 14.1%, and 2.6% for mAP, and 11.9%, 14.5%, and 1.3% for Rank-1, respectively. It can be observed that although ResNet50(RGA) has already achieved quite good performance on CUHK03(L) / CUHK03(D), the model in this invention can still achieve further performance improvements.

[0134] For the three existing models used, this invention employs them as feature extractors for FDCA without any modifications, and almost all three performance metrics (rank_1, mAP, and mINP) show improvement. This indicates that the model of this invention has the potential to achieve further performance improvements compared to the vast majority of existing models. Although many models have already achieved excellent performance in the field of person re-identification, FDCA primarily purifies features from the perspective of decoupling identity and pose features. For person re-identification models that do not consider pose differences, FDCA has the potential to achieve even higher performance.

[0135] Table 2 shows the performance comparison results of the feature decoupling model of this invention with existing methods on the Market1501 and CUHK03 datasets. For a fair comparison, the method of this invention does not use any post-processing methods such as re-ranking or multi-query fusion. The FDCA model of this invention significantly outperforms other methods that utilize human pose and learn attention from the input image itself. Specifically, on Market-1501, FDCA(NN) achieves a rank-1 accuracy of 96.7%, FDCA(NR) achieves an mAP accuracy of 90.5%, and mINP accuracy of 71.2%, significantly outperforming all existing methods. On CUHK03(L) / CUHK03(D), FDCA(RGA-N) achieves a top-1 accuracy of 82.9% / 80.3%, an mAP accuracy of 80.4% / 77.7%, and a mINP accuracy of 71.3% / 67.7%, again demonstrating significant advantages. This invention emphasizes the comparison between FDCA and existing pose-based methods. Compared to the closest previous pose-based work, DG-Net, FDCA achieves at least a 1.9% top-1 ranking improvement and a 4.5% mAP improvement on Market-1501. In summary, the model of this invention fully leverages the pose information of pedestrian images through efficient end-to-end guidance of pose information and feature decoupling based on cross-attention, achieving superior performance.

[0136] Table 2 shows the performance comparison results of the feature decoupling model of the present invention with existing methods on the Market1501 and CUHK03 datasets.

[0137]

[0138] Figure 5 The training process of the model of this invention on the Market1501 dataset is shown. (a) is the loss result graph, and (b) is the accuracy result graph. It can be seen that in the first 5 epochs, the identity loss decreases rapidly, leading to a decrease in the pose loss. From epochs 5-20, the decrease in pose loss further influences the decrease in identity loss. After epoch 20, the already low identity and pose losses continue to influence and reinforce each other, decreasing steadily with small amplitudes. Similarly, identity accuracy and pose accuracy change with an almost symmetrical trend. This demonstrates that the model can achieve performance gains by leveraging the decoupling of identity and pose features, validating the model's effectiveness.

[0139] This invention discloses a pedestrian re-identification method based on the decoupling of identity and pose features. Addressing the impact of changes in pedestrian pose and viewpoint on pedestrian representation, a progressive feature decoupling model based on cross-attention is constructed, combining attention mechanisms and pose guidance end-to-end. This model includes a fine-grained pose label assignment module, which assigns pose labels to each image in the training set, and several feature decoupling modules. These modules consist of identity and pose feature extractors and a cross-attention module. The cross-attention module learns the similarity between identity and pose features and removes these similar parts from the original features. Through the progressive decoupling of these feature decoupling modules, the influence between the final output identity and pose features is minimized. The method proposed in this invention is significant for improving the accuracy and robustness of pedestrian re-identification technology and can provide a more reliable and effective solution for fields such as urban security and intelligent monitoring systems.

[0140] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Those skilled in the art can readily implement the present invention based on the accompanying drawings and the above description. However, any modifications, alterations, or variations made by those skilled in the art without departing from the scope of the present invention, utilizing the disclosed technical content, are equivalent embodiments of the present invention. Furthermore, any modifications, alterations, or variations made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the present invention.

Claims

1. A pedestrian re-identification method based on identity and pose feature decoupling, characterized in that, The application comprises the following steps: obtaining pedestrian posture image data and expanding the same, assigning fine-grained posture labels to the expanded pedestrian posture image data, and obtaining a training set with posture labels; training an identity and posture feature decoupling model based on the training set with posture labels; calculating the deep feature similarity of the to-be-identified pedestrian image and the target pedestrian image based on the trained identity and posture feature decoupling model, selecting the image with the maximum similarity as the re-identification result output, and realizing the re-identification of the pedestrian; constructing an identity and posture feature decoupling model, specifically: the identity and posture feature decoupling model comprises L feature decoupling blocks, a posture feature classification head, and an identity feature classification head, each feature decoupling block is composed of an identity feature extractor, a posture feature extractor, and a cross-attention module; the identity feature extractor and the posture feature extractor are respectively used for learning the shallow identity feature and the posture feature to extract the deep identity feature and the posture feature, and the cross-attention module is used for decoupling the deep identity feature and the posture feature to remove the noise in the identity feature and the posture feature; the identity feature extractor adopts a deep learning network or a Re-ID network as a basic structure; based on the trained identity and posture feature decoupling model, the inference is carried out to realize the pedestrian re-identification, specifically: remove the posture feature classification head in the identity and posture feature decoupling model, calculate the deep features of the to-be-identified pedestrian image and the target pedestrian image respectively by using the identity feature, and obtain the similarity between the to-be-identified pedestrian image and the target pedestrian image according to the distance between the deep features of the to-be-identified pedestrian image and the target pedestrian image; sort the to-be-identified pedestrian image in descending order according to the similarity between the to-be-identified pedestrian image and the target pedestrian image, and take the to-be-identified pedestrian image with the maximum similarity as the output result to complete the re-identification of the pedestrian; the cross-attention module is used for decoupling the deep identity feature and the posture feature to remove the noise in the identity feature and the posture feature, specifically: based on the identity feature and the posture feature, an intermediate feature tensor with a width of W, a height of H, and C channels is given; Based on the spatial locations of each intermediate feature tensor of identity features and posture features, respectively, all spatial locations are formed... For a graph G with nodes, the C-dimensional feature vector at each spatial location is taken as a feature node; embed the graph G of the identity feature and the posture feature in a space, define the influence of the posture feature node on the identity feature node and the influence of the identity feature node on the posture feature node as the dot product affinity in the embedding space, construct an affinity matrix, and then generate normalized attention weights; based on the attention weights, weight and sum the features of all spatial positions of the identity feature and the posture feature, and adaptively determine the attention weight according to the feature similarity of any two positions, to obtain the noise in the identity feature and the posture feature, respectively represented as: wherein, noise in the identity feature; noise in the pose feature, are respectively a mapping of the input identity feature and the pose feature in the embedding space, and are respectively a pose-related region in the identity feature and an identity-related region in the pose feature; After concatenating all the identity features with the noise in the pose features, we get the identity feature noise set and the pose feature noise set , and then remove the noise set and from the identity features and the pose features, respectively. ​ 2.The pedestrian re-identification method based on identity and pose feature decoupling according to claim 1, characterized in that, the pedestrian posture image data is expanded, specifically: the pedestrian posture image data is horizontally flipped to expand the pedestrian posture image data set by one time. 3.The pedestrian re-identification method based on identity and pose feature decoupling according to claim 1, characterized in that, fine-grained posture labels are assigned to the expanded pedestrian posture image data to obtain a training set, specifically: The 2D human pose estimation model is used to extract the pose skeleton key points of each pedestrian image in the expanded pedestrian pose image data, the K_means clustering algorithm is used for clustering analysis of the pose skeleton key points, and the pose skeleton key points with similar attributes are aggregated into the same cluster, each cluster represents a posture label, and each pedestrian pose image is assigned with a corresponding posture label by traversing the entire pedestrian pose image dataset, so as to obtain a training set with posture labels. 4.The pedestrian re-identification method based on identity and pose feature decoupling according to claim 1, characterized in that, The specific process of training the identity and pose feature decoupling model is as follows: Based on the training set with posture labels, AGW and RGA are used as the identity feature extractor and the pose feature extractor of the identity and pose feature decoupling model respectively, the performance of the identity and pose feature decoupling model is compared with that of the corresponding identity feature extractor and pose feature extractor, and the identity and pose feature decoupling model is trained in an end-to-end manner by using identity loss and pose loss to learn the identity and pose representation of pedestrians. During the training process, the input pedestrian pose image data is subjected to normalization, random erasing and random cropping data enhancement processing.

5. The method of claim 4, wherein, It includes, 6. A pedestrian re-identification system based on identity and pose feature decoupling, based on the pedestrian re-identification method based on identity and pose feature decoupling of any one of claims 1-5, characterized in that, a data acquisition module, a model training module and a pedestrian re-identification module; The data acquisition module is used to acquire pedestrian pose image data and expand it, assign fine-grained posture labels to the expanded pedestrian pose image data, and obtain a training set with posture labels. The model training module is used to train the identity and pose feature decoupling model based on the training set with posture labels. The pedestrian re-identification module is used to remove the pose feature classification head based on the trained identity and pose feature decoupling model, calculate the similarity of the deep features of the to-be-identified pedestrian image and the target pedestrian image, select the image with the largest similarity as the re-identification result output, and realize the re-identification of pedestrians. The processor executes the computer program to realize the steps of the pedestrian re-identification method based on identity and pose feature decoupling according to any one of claims 1-5.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The computer program is executed by the processor to realize the steps of the pedestrian re-identification method based on identity and pose feature decoupling according to any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7. ​

Images