Pedestrian re-identification method, system and equipment based on identity and posture feature decoupling

Abstract

The invention provides a pedestrian re-identification method, system and device based on identity and posture feature decoupling, and the method comprises the steps: carrying out the fine-grained posture label distribution of expanded pedestrian posture image data, and obtaining a training set with posture labels; and training the identity and posture feature decoupling model, calculating the depth feature similarity between the to-be-recognized pedestrian image and the target pedestrian image, and selecting the image with the maximum similarity as a re-recognition result to output, thereby realizing re-recognition of the pedestrian. According to the method, a large amount of pedestrian posture image data can be processed and expanded, rich training samples are provided for the model, so that higher recognition capability is shown in the face of new data or unknown scenes, and the problem that in the actual scene, due to the influence of pedestrian posture changes, the recognition efficiency is greatly improved is solved. Therefore, the pedestrian images of different postures from the same identity have huge visual appearance differences, and finally, the pedestrian re-identification accuracy is low.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology and relates to a pedestrian re-identification method, system and device based on decoupling of identity and posture features. Background Art

[0002] With the acceleration of urbanization and the widespread application of intelligent monitoring systems, pedestrian information in surveillance videos has become an important part of massive data. However, with the continuous increase in the number of surveillance cameras and the expansion of monitoring scope, the personnel information involved in surveillance videos has shown an exponential growth trend. This explosive growth of personnel information has posed severe challenges to related technologies and led to serious information overload problems. In surveillance videos, the identification and tracking of personnel are crucial for public safety and urban management. However, traditional monitoring systems are often unable to effectively process massive amounts of personnel information, making it difficult to identify and track personnel information in surveillance videos in a timely and accurate manner. Therefore, it has become an urgent problem to quickly and accurately extract the identity information of the target person from massive surveillance videos, and pedestrian re-identification (Re-ID) technology has emerged.

[0003] In real-world scenarios, Re-ID still faces many challenges. These challenges mainly stem from the complex environment in surveillance videos and the characteristics of pedestrians themselves: 1) Viewpoint changes: The different positions and angles of surveillance cameras cause the shooting angles and viewpoints of the same pedestrian in different camera fields of view to change, making the appearance of pedestrians in different camera images different; 2) Posture differences: The posture of pedestrians may change during walking, such as standing, walking, bending, etc. This posture difference will cause the posture features of the same pedestrian in different images to be inconsistent; 3) Low image resolution: Surveillance cameras may be limited by factors such as equipment quality and installation location, resulting in low resolution of the pedestrian images captured, thus Affects the accuracy of pedestrian feature extraction and matching; 4) Lighting changes: The lighting conditions in the monitoring scene may change with time and environment. Pedestrian images under different lighting conditions will show different brightness and contrast, which increases the difficulty of pedestrian re-identification; 5) Occlusion: Pedestrians may be blocked by other objects or people in the monitoring video, resulting in some pedestrian features being invisible or blurred, thus affecting the accuracy of pedestrian recognition and matching; 6) Heterogeneous modalities: The monitoring video may contain image data of different modalities such as RGB images and infrared images at the same time. The feature differences and conversions between different modalities increase the complexity of the pedestrian re-identification task. These challenges have caused a lot of variability and uncertainty. Solving these challenges is of great significance to improving the accuracy and robustness of pedestrian re-identification technology, and can provide more reliable and effective solutions for urban security, intelligent monitoring systems and other fields.

[0004] According to different research entry points, the research in the field of Re-ID can be divided into standard person re-identification and open scene person re-identification. Standard person re-identification generally has the following assumptions: 1) the image or video is captured by a single-mode visible camera; 2) each pedestrian is represented by a bounding box (that is, the upstream object detection task is sufficiently accurate); 3) there is enough labeled training data for supervised discriminative Re-ID model learning; 4) the annotations are usually correct; 5) the query person must appear in the image set. Standard person re-identification usually focuses more on the feature matching of the pedestrian itself and ignores many practical problems in open scenes (cross-modal, noisy, unsupervised, etc.). Correspondingly, open scene person re-identification, based on the research of standard person re-identification, further considers practical problems in specific scenarios, including cross-modal person re-identification, noise-robust person re-identification, unsupervised person re-identification, etc. Zheng et al. first attempted to apply generative adversarial networks to person re-identification and improved the supervised feature representation learning based on generated person images. Liu et al. added posture constraints to the model to improve the quality of generated person images and generate person images with new posture variables. Qian et al. designed a pose-normalized image generation method to enhance robustness to pose changes. Camera style information is also integrated into the image generation process to address cross-camera changes, and a joint discriminative and generative learning model is proposed to learn appearance codes and structure codes respectively to improve image generation quality.

[0005] In summary, the existing technology has the problem that pedestrian images with different postures from the same identity have huge visual appearance differences due to the influence of pedestrian posture changes, which ultimately leads to low pedestrian re-identification accuracy. Summary of the invention

[0006] In response to the problems existing in the prior art, the present invention provides a pedestrian re-identification method, system and device based on the decoupling of identity and posture features, which solves the problem that in actual scenarios, due to the influence of pedestrian posture changes, pedestrian images from the same identity but different postures have huge visual appearance differences, which ultimately leads to low pedestrian re-identification accuracy.

[0007] The present invention is achieved through the following technical solutions:

[0008] A pedestrian re-identification method based on decoupling of identity and posture features, comprising:

[0009] Obtaining and expanding pedestrian posture image data, assigning fine-grained posture labels to the expanded pedestrian posture image data, and obtaining a training set with posture labels;

[0010] Train the identity and posture feature disentanglement model based on the training set with posture labels;

[0011] Based on the trained identity and posture feature decoupling model, the posture feature classification head is removed, the deep feature similarity between the pedestrian image to be identified and the target pedestrian image is calculated, and the image with the largest similarity is selected as the re-identification result output to achieve pedestrian re-identification.

[0012] Preferably, the pedestrian posture image data is expanded, specifically:

[0013] The pedestrian pose image data is horizontally flipped to double the size of the pedestrian pose image dataset.

[0014] Preferably, fine-grained posture label assignment is performed on the expanded pedestrian posture image data to obtain a training set, specifically as follows:

[0015] A 2D human pose estimation model is used to extract the pose skeleton key points of each pedestrian image in the expanded pedestrian pose image data. The pose skeleton key points are clustered using the K_means clustering algorithm, and the pose skeleton key points with similar attributes are aggregated into the same cluster. Each cluster represents a pose label. By traversing the entire pedestrian pose image dataset, a corresponding pose label is assigned to each pedestrian pose image, thereby obtaining a training set with pose labels.

[0016] Preferably, a decoupling model of identity and posture features is constructed, specifically:

[0017] The identity and posture feature decoupling model includes L feature decoupling blocks, a posture feature classification head and an identity feature classification head, each feature decoupling block is composed of an identity feature extractor, a posture feature extractor and a cross attention module;

[0018] The identity feature extractor and posture feature extractor are used to learn shallow identity features and posture features to extract deep identity features and posture features respectively. The cross attention module is used to decouple the deep identity features and posture features to remove noise in the identity features and posture features.

[0019] The identity feature extractor uses a deep learning network or a Re-ID network as the basic structure;

[0020] Preferably, the identity and posture feature decoupling model is trained, and the specific process is as follows:

[0021] Based on the training set with posture labels, Bag of Tricks, AGW and RGA are used as the identity feature extractor and posture feature extractor of the identity and posture feature decoupling model respectively. The performance of the identity and posture feature decoupling model is compared with the corresponding identity feature extractor and posture feature extractor. The identity loss and posture loss are used to train the identity and posture feature decoupling model in an end-to-end manner to learn the identity and posture representation of pedestrians.

[0022] Among them, for identity feature representation, label-smoothed cross entropy loss, weighted regularized triplet loss and center loss are used for identity recognition.

[0023] Preferably, during the training process, the input pedestrian posture image data is subjected to data enhancement processing such as normalization, random erasing and random cropping.

[0024] Preferably, reasoning is performed based on the trained identity and posture feature decoupling model to achieve pedestrian re-identification, specifically:

[0025] The posture feature classification head in the identity and posture feature decoupling model is removed, and the depth features of the pedestrian image to be identified and the target pedestrian image are calculated respectively using the identity features. The similarity between the pedestrian image to be identified and the target pedestrian image is obtained according to the similarity of the distance between the depth features of the pedestrian image to be identified and the target pedestrian image. The pedestrian images to be identified are sorted in descending order according to the similarity between the pedestrian image to be identified and the target pedestrian image, and the pedestrian image to be identified with the greatest similarity is taken as the output result to complete the re-identification of the pedestrian.

[0026] A person re-identification system based on decoupling of identity and posture features, comprising:

[0027] Data acquisition module, model training module and pedestrian re-identification module;

[0028] The data acquisition module is used to acquire and expand pedestrian posture image data, and to assign fine-grained posture labels to the expanded pedestrian posture image data to obtain a training set with posture labels;

[0029] The model training module is used to train the identity and posture feature decoupling model based on a training set with posture labels;

[0030] The pedestrian re-identification module is used to remove the posture feature classification head based on the trained identity and posture feature decoupling model, calculate the deep feature similarity between the pedestrian image to be identified and the target pedestrian image, select the image with the largest similarity as the re-identification result output, and realize the re-identification of the pedestrian.

[0031] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of a method for pedestrian re-identification based on decoupling of identity and posture features are implemented.

[0032] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of a pedestrian re-identification method based on decoupling of identity and posture features.

[0033] Compared with the prior art, the present invention has the following beneficial technical effects:

[0034] The present invention provides a pedestrian re-identification method, system and device based on the decoupling of identity and posture features. In view of the influence of pedestrian posture and viewpoint changes on pedestrian representation, pedestrian posture image data is obtained and expanded, and fine-grained posture labels are assigned to the expanded pedestrian posture image data to obtain a training set with posture labels; the method provides detailed posture information for each pedestrian posture image through fine-grained posture label assignment. This enables the model to learn more detailed and accurate posture features during the training process, so that the posture of the pedestrian can be matched more accurately during re-identification, thereby improving the accuracy of re-identification. The identity and posture feature decoupling model is trained based on the training set with posture labels; the decoupling of identity and posture features enables the model to focus on the identity information and posture information of the pedestrian respectively. This decoupling design enables the model to more stably extract the identity features of the pedestrian when facing complex scenes such as posture changes, occlusion or perspective changes, thereby enhancing the robustness of the model. Based on the trained identity and posture feature decoupling model, the posture feature classification head is removed, the similarity of the deep features of the pedestrian image to be identified and the target pedestrian image is calculated, and the image with the largest similarity is selected as the re-identification result output to achieve pedestrian re-identification. In the re-identification stage, by removing the posture feature classification head, the model only needs to calculate the similarity of the deep features of the pedestrian image to be identified and the target pedestrian image, without considering the matching of posture features. This greatly simplifies the calculation process and improves the calculation efficiency of re-identification. This method can process and expand a large amount of pedestrian posture image data, providing a wealth of training samples for the model. This enables the model to learn a more generalized feature representation, thereby showing stronger recognition ability when facing new data or unknown scenes, solving the problem that in actual scenes, due to the influence of pedestrian posture changes, pedestrian images with different postures from the same identity have huge visual appearance differences, which ultimately leads to low pedestrian re-identification accuracy.

[0035] Furthermore, the pedestrian re-identification method based on the decoupling of identity and posture features in the present invention can not only improve the accuracy and robustness of re-identification, but also improve computing efficiency, support large-scale data set training, promote multi-task learning and application, and improve user experience and safety. The method proposed in the present invention is of great significance for improving the accuracy and robustness of pedestrian re-identification technology, and can provide more reliable and effective solutions for urban security, intelligent monitoring systems and other fields.

[0036] Furthermore, the present invention constructs a cross-attention based progressive feature decoupling model, which combines the attention mechanism and posture guidance end-to-end.

[0037] Furthermore, the identity and posture feature decoupling module is composed of an identity and posture feature extractor and a cross-attention module. The cross-attention module learns the similarities between identity features and posture features and removes these similar parts from the original features. Through the progressive decoupling of several feature decoupling modules, the influence between the final output identity features and posture features is minimized.

[0038] Furthermore, the identity feature extractor uses a deep learning network or a Re-ID network as the basic structure, which improves the robustness of the identity feature;

[0039] Furthermore, the cross-attention module removes the posture noise in the identity feature and the identity noise in the posture feature to a certain extent, so that the identity feature and the posture feature influence and promote each other in the learning process and become discriminable together. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 A flowchart of a specific implementation of the pedestrian re-identification method based on identity and posture feature decoupling of the present invention;

[0042] Figure 2 A schematic diagram of fine-grained posture label assignment of the present invention;

[0043] Figure 3 This is a framework diagram of the identity and posture feature decoupling model of the present invention;

[0044] Figure 4 This is a network architecture diagram of the cross-attention module of the present invention;

[0045] Figure 5 Graphs showing the training results of the model in the embodiment on the Market1501 dataset, where (a) is a loss result graph and (b) is an accuracy result graph. DETAILED DESCRIPTION

[0046] The present invention is further described in detail below in conjunction with specific embodiments, which are intended to explain the present invention rather than to limit it.

[0047] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0048] A pedestrian re-identification method based on decoupling of identity and posture features includes the following steps:

[0049] S1: Assign fine-grained pedestrian pose labels;

[0050] S2: Build a model to decouple identity and posture features;

[0051] S3: training a model to disentangle identity and pose features;

[0052] S4: Person Re-ID model reasoning.

[0053] A further improvement of the present invention is:

[0054] In step S1, the existing methods usually perform random horizontal flipping of the input image with a certain probability during training for data enhancement. However, in the present invention, since the posture label is given before training, if an image happens to be horizontally flipped, this will cause the posture label to not match the image, that is, additional noise is added. In order to eliminate this noise and not discard the data enhancement method of horizontal flipping, the present invention pre-flips all images horizontally before assigning posture labels, which directly doubles the data set. Fine-grained posture label assignment is then performed on this expanded data set to obtain the training set.

[0055] In step S2, the model consists of L feature decoupling blocks and two classification heads. Each feature decoupling block consists of an identity feature extractor, a posture feature extractor and a cross-attention module. The identity feature extractor and the posture feature extractor are used to learn deeper and more abstract identity features and posture features, respectively, and the cross-attention module is dedicated to decoupling identity features and posture features without changing the depth (abstractness) of the features. In order to extract robust identity features, the identity feature extractor is based on a typical deep learning network (ResNet or DenseNet) or a mature Re-ID network, while the posture feature extractor uses the same or similar network structure. This makes the model not confined to a specific feature extraction network, and future better deep learning networks or Re-ID models can still be further expanded on the model of the present invention.

[0056] Furthermore, the main idea of ​​the proposed cross-attention module is as follows: given two intermediate feature tensors of width W, height H and C channels, the present invention treats the c-dimensional feature vector at each spatial position as a feature node. For each feature tensor, all spatial positions form a graph G with N = H × W nodes. The present invention represents all feature nodes as Where i,p∈{1,2,L,N}. Pose feature node x p For identity feature node x i The influence of can be defined as the dot product affinity in the embedding space as follows:

[0057] a i,p =f ip (x i ,x p )=θ id (x i ) T γ po (x p )

[0058] where θ id and γ po are two embedding functions implemented by a convolutional layer with a 1×1 kernel, x i x p The influence is expressed as

[0059] a p,i =f pi (x p ,x i )=θ po (x p ) T γ id (x i )

[0060] in and According to the affinity matrix, the attention weight can be defined as

[0061]

[0062]

[0063] Among them, s i,p Represents x p x i The influence weight of p,i Represents x i x p The influence weight of The cross-attention module removes the posture noise in the identity feature and the identity noise in the posture feature to a certain extent, so that the identity feature and the posture feature influence and promote each other in the learning process and become discriminable together.

[0064] In step S3, the present invention uses identity loss and posture loss to learn the identity and posture representation of pedestrians in an end-to-end manner. For identity feature representation, the present invention selects cross entropy loss with label smoothing, weighted regularized triple loss and center loss for identity recognition. In order to better demonstrate the effect of the model, the present invention selects Bag of Tricks, AGW and RGA from the papers in the field of pedestrian re-identification published at the top conferences in recent years as the identity feature extractors of the model, and uses the output f of the five residual modules id and f po As the input of the criss-cross attention module, the output of the criss-cross attention module is then used as the input of the subsequent residual block. All input images are horizontally flipped to double the size of the training set, and then pose labels are assigned to all training set images through a fine-grained pose label assignment method. During training, the input images are resized to 256×128, and then data augmentation is performed through normalization, random erasing, and random cropping. The test images are resized to 256×128 and augmented only by normalization.

[0065] In step S4, in the model inference stage, the entire posture feature classification head is directly removed. That is, for the output f of the last cross attention module id ,f po , directly discard f po , only f id Input into the identity classification head to obtain the identity feature. The model has learned the identity and posture feature representation in the training phase, so the model can adaptively complete the feature representation without any identity or posture labels in the inference phase.

[0066] A person re-identification system based on decoupling of identity and posture features, comprising:

[0067] Data acquisition module, model training module and pedestrian re-identification module;

[0068] The data acquisition module is used to acquire and expand pedestrian posture image data, and to assign fine-grained posture labels to the expanded pedestrian posture image data to obtain a training set with posture labels;

[0069] The model training module is used to train the identity and posture feature decoupling model based on a training set with posture labels;

[0070] The pedestrian re-identification module is used to remove the posture feature classification head based on the trained identity and posture feature decoupling model, calculate the deep feature similarity between the pedestrian image to be identified and the target pedestrian image, select the image with the largest similarity as the re-identification result output, and realize the re-identification of the pedestrian.

[0071] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be based on the operation of the pedestrian re-identification method decoupled from identity and posture features.

[0072] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It can be understood that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the pedestrian re-identification method based on identity and posture feature decoupling in the above embodiment.

[0073] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0074] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0075] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0077] Example 2

[0078] See also Figure 1 The present invention discloses a pedestrian re-identification method based on identity and posture feature decoupling. The specific implementation process is mainly as follows: assigning fine-grained pedestrian posture labels, building an identity and posture feature decoupling model, training the identity and posture feature decoupling model and pedestrian re-identification model inference. The detailed steps are as follows:

[0079] Step 1: Assign fine-grained pedestrian pose labels

[0080] See also Figure 2 As shown. Given an original data set Among them I i represents the i-th image, Represents the identity label of the image. Use the 2D human pose estimation model OpenPose to extract the pose skeleton key points p of each pedestrian image i =Φ(I i ), where p i is a 36-dimensional vector, Φ is a 2D human posture estimation model. K_means clustering algorithm is used to cluster all posture skeleton points Aggregate into M clusters Each cluster represents a pose label. For any image I i , whose pose label is Finally, we get the input training set

[0081] Existing methods usually perform random horizontal flipping of input images with a certain probability during training for data augmentation. However, in the method of the present invention, since the posture labels are given before training, if an image happens to be horizontally flipped, this will cause the posture label to not match the image, that is, additional noise is added. In order to eliminate this noise and not discard the data augmentation method of horizontal flipping, the present invention pre-flips all images horizontally before assigning posture labels, which directly doubles the data set. Fine-grained posture label assignment is then performed on this expanded data set to obtain the training set.

[0082]

[0083] Step 2: Build a model to decouple identity and pose features

[0084] See also Figure 3 As shown, the model consists of L feature decoupling blocks and two classification heads. Each feature decoupling block consists of an identity feature extractor, a posture feature extractor and a cross-attention module. The identity feature extractor and the posture feature extractor are used to learn deeper and more abstract identity features and posture features, respectively, and the cross-attention module is dedicated to decoupling identity features and posture features without changing the depth (abstractness) of the features. In order to extract robust identity features, the identity feature extractor is based on a typical deep learning network (ResNet or DenseNet) or a mature Re-ID network, while the posture feature extractor uses the same or similar network structure. This makes the model not confined to a specific feature extraction network, and future better deep learning networks or Re-ID models can still be further expanded on the model of the present invention.

[0085] Assuming FDB l-1 represents the lth feature decoupling block, and FDB l-1 The output is Then the output of the lth feature decoupling block can be expressed as

[0086]

[0087] Specifically, the identity and posture feature extractors learn shallower features to extract deeper features, and the cross-attention module decouples the deeper features to remove noise in the features, in the form of

[0088]

[0089]

[0090] in is a CNN model used to extract features, x id ,x po is the output feature of the identity and posture feature extractor, that is, the identity feature and posture feature of the Lth layer that have not yet eliminated noise. As the feature decoupling proceeds, the posture-related information in the identity feature is gradually suppressed, and the identity-related information in the posture feature is also gradually suppressed, and both the identity feature and the posture feature gradually tend to be pure. The output feature of the last feature decoupling block passes through the pooling layer to obtain the final identity feature f id and posture feature f po .

[0091] See also Figure 4 As shown, in the cross-attention module proposed in the present invention, given two intermediate feature tensors of width W, height H and C channels, the c-dimensional feature vector at each spatial position is taken as a feature node. For each feature tensor, all spatial positions form a graph G with N = H × W nodes. The present invention represents all feature nodes as Where i,p∈{1,2,L,N}. Pose feature node x p For identity feature node x i The influence of can be defined as the dot product affinity in the embedding space as follows:

[0092] a i,p =f ip (x i ,x p )=θ id (x i ) T γ po (x p )

[0093] where θ id and γ po are two embedding functions implemented by a convolutional layer with a convolution kernel of size 1×1, namely, in h is a hyperparameter set to make the features learned by the attention mechanism more diversified. Generally, h ≥ 1. Similarly, x i x p The influence is expressed as

[0094] a p,i =f pi (x p ,x i )=θ po (x p ) T γ id (x i )

[0095] in and According to the affinity matrix, the attention weight can be defined as

[0096]

[0097]

[0098] Among them, s i,p Represents x p x i The influence weight of p,i Represents x i xp The influence weight of Intuitively, the present invention uses A ip and A pi To construct the relationship between the pedestrian's identity features and posture features, the resulting matrix S ip and S pi The pose-related regions in identity features and the identity-related regions in pose features are revealed respectively. Through cross-attention, the spatial dependency between any two positions of identity and pose features can be captured.

[0099] The features of all positions are weighted and summed, and the attention weights are adaptively determined according to the feature similarity of the corresponding two positions. The noise in the feature is expressed as

[0100] δ id ′=S ip ×V id ,δ po ′=S pi ×V po

[0101] in, × represents matrix multiplication, V id ,V po is the mapping of the input features in the embedding space, that is, Among them, ψ id and ψ po is similar to θ id The two embedding functions of are, in All δ id ′,δ po After cascading, we get the feature noise δ with the dimension of C×H×W. id ,δ po , the final output of the cross-attention module is expressed as

[0102] x id =x id -δ id

[0103] x po =x po -δ po

[0104] The cross-attention module removes the posture noise in the identity feature and the identity noise in the posture feature to a certain extent, so that the identity feature and the posture feature influence and promote each other in the learning process and become discriminable together.

[0105] Step 3: Train a model to disentangle identity and pose features

[0106] The present invention uses identity loss and posture loss to learn pedestrian identity and posture representation in an end-to-end manner. For identity feature representation, the present invention chooses to use cross entropy loss with label smoothing, weighted regularized triplet loss and center loss for identity recognition.

[0107] The cross entropy loss can be expressed as:

[0108]

[0109] Where N is the size of mini_batch, M is the number of categories of pedestrian identities in the training set, is the probability that the model predicts that sample i belongs to identity category c, It can be expressed as

[0110]

[0111] Where ε is a small constant used to encourage the model to be less confident in the training set. In this study, ε is set to 0.1.

[0112] The weighted regularized triplet loss can be expressed as

[0113]

[0114]

[0115] where (i, j, k) represents the hardest triplet in each training batch. For anchor i, P i is the corresponding positive set, N i It is a negative set. Represents the pairwise distance of positive / negative sample pairs. The above weighted regularization inherits the advantage of optimizing the relative distance between positive and negative pairs, but it avoids introducing any additional residual parameters. The center loss simultaneously learns a center for the deep features of each class and penalizes the distance between the deep features and their corresponding class centers, making up for the shortcomings of the triple loss.

[0116] The center loss function is expressed as

[0117]

[0118] where f i id represents the identity feature of the i-th image, Indicates The feature center of each category. The training objective of identity recognition can be expressed as:

[0119]

[0120] in Respectively represent The weight of .

[0121] Similarly, the training objective for gesture recognition can be stated as:

[0122]

[0123] in Respectively represent For the triplet loss using difficult sample mining, each mini_batch must be arranged in a certain label order, so only one of identity recognition and posture recognition can use the triplet loss. The present invention chooses to use it in identity recognition. Therefore, the optimization goal of the final end-to-end training can be expressed as

[0124]

[0125] where β id ,β po Represents the weights of identity loss and pose loss.

[0126] In order to better demonstrate the effect of this model, the present invention selects BagofTricks, AGW and RGA from the papers in the field of person re-identification published in the top conferences in recent years as the identity feature extractors of FDCA, and compares the performance of this model with the corresponding identity feature extractors. The present invention uses ResNet, ResNet (NL), and ResNet (RGA) to represent the backbone networks used by BagofTricks, AGW and RGA. Taking ResNet as an example, it is used as the identity and posture feature extractor in this model, and the output f of the five residual modules is used. id and f po As the input of CAM, the output of CAM is then used as the input of the subsequent residual block. Without loss of generality, similar settings are used for ResNet (NL) and ResNet (RGA), setting is 1,1,0.0005, 1,0.0005,β id ,β po is 1,0.6.

[0127] The present invention uses two well-known datasets Market-1501 and CUHK03 as training sets for the model. The pedestrian re-identification images of these datasets are all from two or more cameras, which meet the cross-camera requirements of the standard pedestrian re-identification task. The present invention flips all input images horizontally to double the size of the training set, and then assigns posture labels to all training set images through a fine-grained posture label assignment method. During the training process, the input image size is resized to 256×128, and then data enhancement is performed through normalization, random erasing, and random cropping. The test image is resized to 256×128 and enhanced only by normalization. The backbone network is pre-trained on ImageNet. The present invention uses the Adam optimizer to train the decoupled model with a learning rate of 0.0004 and a weight decay of 0.0005. For different baselines, a learning rate strategy similar to the original model is adopted. Specifically, for the decoupled model using ResNet and ResNet (NL) baselines, a learning rate warm-up strategy is adopted in the first 5 epochs, and is reduced to 0.2 times at 15, 40, and 70 epochs, respectively, for a total of 120 epochs. For the decoupled model using the ResNet (RGA) baseline, a learning rate warm-up strategy is adopted in the first 10 epochs, and is reduced to 0.5 times at 40, 60, 80, 100, 120, 140, 160, 180, and 230 epochs, respectively, for a total of 300 epochs. The present invention is trained on two RTX3090 GPUs with a batch_size of 64. Each batch contains 16 identities, each identity has 4 instances, and all models are implemented using the PyTorch framework.

[0128] Step 4: Person Re-ID Model Inference

[0129] In the model inference stage, the entire posture feature classification head is directly removed. That is, for the output f of the last cross attention module id ,f po , directly discard f po , only f id Input into the identity classification head to obtain identity features. The model has learned identity and posture feature representations in the training phase, so in the inference phase, no identity or posture labels are required, and the model can adaptively complete feature representation. Further, the deep features of the image to be identified and the target image are calculated respectively, and the similarity between the image to be identified and the target image is obtained according to the similarity of the distance between the deep features of the image to be identified and the target image; the images to be identified are sorted in descending order according to the similarity between the image to be identified and the target image, and the image to be identified with the greatest similarity is used as the output result to complete the re-identification of the pedestrian.

[0130] Furthermore, the present invention conducts ablation experiments on person re-identification on Market-1501 and CUHK03(L) / CUHK03(D), respectively. As shown in Table 1, FDCA is the name of the person re-identification model proposed by the present invention. In FDCA(*-*), the former in the brackets represents the identity feature extractor, and the latter represents the posture feature extractor. R, N and RG in the brackets after FDCA represent ResNet50, ResNet50(NL) and ResNet50(RGA), respectively. In order to approximate the setting of BagofTricks, FDCA(RR) and ResNet50 use Trilardloss instead of WRT.

[0131] Table 1 is a table of the names and features of the pedestrian re-identification model proposed by FDCA in this invention.

[0132]

[0133] On Market1501, FDCA(NN) and FDCA(RR) significantly outperform the corresponding identity feature extractors by 13.4%, 16.1% and 1.0% in mINP, 12.2%, 14.1% and 2.6% in mAP, and 11.9%, 14.5% and 1.3% in Rank-1. It can be found that although ResNet50(RGA) has achieved quite good performance in CUHK03(L) / CUHK03(D), the model in the present invention can still achieve further performance improvement on this basis.

[0134] For the three existing models used, the present invention uses them as feature extractors for FDCA without any modification, and the three performance indicators of rank_1 / mAP / mINP are almost all improved, which means that the model of the present invention has the potential to obtain further performance improvement based on the vast majority of models. Although many models have achieved good performance in the field of pedestrian re-identification, FDCA mainly purifies features from the perspective of identity / posture feature decoupling. For pedestrian re-identification models that do not consider posture differences, FDCA has the potential to obtain higher performance based on them.

[0135] See Table 2, which shows the performance comparison results of the feature decoupling model of the present invention with existing methods on the Market1501 and CUHK03 datasets. For fair comparison, the method of the present invention does not use any post-processing methods such as re-ranking or multi-query fusion. Compared with other methods that use human posture and learn attention from the input image itself, the FDCA model of the present invention significantly outperforms them. Specifically, on Market-1501, the rank-1 accuracy of FDCA (NN) is 96.7%, the mAP accuracy of FDCA (NR) is 90.5%, and the mINP accuracy is 71.2%, which is significantly better than all existing methods. On CUHK03 (L) / CUHK03 (D), the top-1 accuracy of FDCA (RGA-N) is 82.9% / 80.3%, the mAP accuracy is 80.4% / 77.7%, and the mINP accuracy is 71.3% / 67.7%, which also achieves significant advantages. The present invention emphasizes the comparison between FDCA and existing posture-based methods. Compared with the closest pose-based previous work DG-Net, FDCA achieves at least 1.9% top-1 and 4.5% mAP improvements on Market-1501. In general, our model fully utilizes the pose information of pedestrian images through effective end-to-end guidance of pose information and cross-attention-based feature decoupling, and achieves superior performance.

[0136] Table 2 shows the performance comparison results of the feature decoupling model of the present invention with the existing methods on the Market1501 and CUHK03 datasets.

[0137]

[0138] Figure 5 The training process of the model of the present invention on the Market1501 dataset is shown, (a) is the loss result diagram, and (b) is the accuracy result diagram. It can be seen that in the first 5 epochs, the identity loss decreases rapidly and drives the posture loss to decrease. In epochs 5-20, the posture loss decreases and further affects the identity loss. After epoch 20, the already low identity loss and posture loss continue to influence and promote each other, and continue to decrease at a small and stable rate. Similarly, the identity accuracy and posture accuracy change in an almost symmetrical trend. This proves that the model can obtain performance gains by decoupling the identity and posture features, verifying the effectiveness of the model.

[0139] The present invention discloses a pedestrian re-identification method based on the decoupling of identity and posture features. Aiming at the influence of pedestrian posture and viewpoint changes on pedestrian representation, a progressive feature decoupling model based on cross-attention is constructed to combine the attention mechanism and posture guidance end-to-end. The model includes a fine-grained posture label assignment module, which assigns a posture label to each image in the training set, and several feature decoupling modules, which are composed of an identity and posture feature extractor and a cross-attention module. The cross-attention module learns the similarity between identity features and posture features and removes these similar parts from the original features. Through the progressive decoupling of several feature decoupling modules, the influence between the identity features and posture features of the final output is minimized. The method proposed by the present invention is of great significance for improving the accuracy and robustness of pedestrian re-identification technology, and can provide more reliable and effective solutions for fields such as urban security and intelligent monitoring systems.

[0140] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any ordinary technician in the industry can smoothly implement the present invention as shown in the drawings and described above. However, any equivalent changes, modifications and evolutions made by technicians familiar with the profession without departing from the scope of the technical solution of the present invention using the technical content disclosed above are all equivalent embodiments of the present invention. At the same time, any equivalent changes, modifications and evolutions made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the technical solution of the present invention.

Claims

1. A pedestrian re-identification method based on decoupling of identity and posture features, characterized in that: include, Obtaining and expanding pedestrian posture image data, assigning fine-grained posture labels to the expanded pedestrian posture image data, and obtaining a training set with posture labels; Train the identity and posture feature disentanglement model based on the training set with posture labels; Based on the trained identity and posture feature decoupling model, the deep feature similarity between the image of the pedestrian to be identified and the target pedestrian image is calculated, and the image with the largest similarity is selected as the re-identification result output to achieve re-identification of the pedestrian.

2. The pedestrian re-identification method based on decoupling of identity and posture features according to claim 1, characterized in that: Pedestrian posture image data is expanded, specifically: The pedestrian pose image data is horizontally flipped to double the size of the pedestrian pose image dataset.

3. The pedestrian re-identification method based on decoupling of identity and posture features according to claim 1, characterized in that: Fine-grained posture label assignment is performed on the expanded pedestrian posture image data to obtain a training set, specifically: A 2D human pose estimation model is used to extract the pose skeleton key points of each pedestrian image in the expanded pedestrian pose image data. The pose skeleton key points are clustered using the K_means clustering algorithm, and the pose skeleton key points with similar attributes are aggregated into the same cluster. Each cluster represents a pose label. By traversing the entire pedestrian pose image dataset, a corresponding pose label is assigned to each pedestrian pose image, thereby obtaining a training set with pose labels.

4. The pedestrian re-identification method based on decoupling of identity and posture features according to claim 1, characterized in that: Construct a decoupling model of identity and posture features, specifically: The identity and posture feature decoupling model includes L feature decoupling blocks, a posture feature classification head and an identity feature classification head, each feature decoupling block is composed of an identity feature extractor, a posture feature extractor and a cross attention module; The identity feature extractor and posture feature extractor are used to learn shallow identity features and posture features to extract deep identity features and posture features respectively. The cross attention module is used to decouple the deep identity features and posture features to remove noise in the identity features and posture features. The identity feature extractor uses a deep learning network or a Re-ID network as the basic structure.

5. The pedestrian re-identification method based on decoupling of identity and posture features according to claim 4, characterized in that: The specific process of training the identity and posture feature decoupling model is: Based on the training set with posture labels, Bag of Tricks, AGW and RGA are used as the identity feature extractor and posture feature extractor of the identity and posture feature decoupling model respectively. The performance of the identity and posture feature decoupling model is compared with the corresponding identity feature extractor and posture feature extractor. The identity loss and posture loss are used to train the identity and posture feature decoupling model in an end-to-end manner to learn the identity and posture representation of pedestrians. Among them, for identity feature representation, label-smoothed cross entropy loss, weighted regularized triplet loss and center loss are used for identity recognition.

6. The pedestrian re-identification method based on decoupling of identity and posture features according to claim 5, characterized in that: During the training process, the input pedestrian posture image data is normalized, randomly erased and randomly cropped for data enhancement.

7. The pedestrian re-identification method based on decoupling of identity and posture features according to claim 1, characterized in that: Based on the trained identity and posture feature decoupling model, reasoning is performed to achieve pedestrian re-identification, specifically: The posture feature classification head in the identity and posture feature decoupling model is removed, and the depth features of the pedestrian image to be identified and the target pedestrian image are calculated respectively using the identity features. The similarity between the pedestrian image to be identified and the target pedestrian image is obtained according to the similarity of the distance between the depth features of the pedestrian image to be identified and the target pedestrian image. According to the similarity between the pedestrian image to be identified and the target pedestrian image, the pedestrian images to be identified are sorted in descending order, and the pedestrian image to be identified with the greatest similarity is taken as the output result to complete the re-identification of the pedestrian.

8. A pedestrian re-identification system based on decoupling of identity and posture features according to claim 1, a pedestrian re-identification method based on decoupling of identity and posture features according to any one of claims 1-7, characterized in that: include, Data acquisition module, model training module and pedestrian re-identification module; The data acquisition module is used to acquire and expand pedestrian posture image data, and to assign fine-grained posture labels to the expanded pedestrian posture image data to obtain a training set with posture labels; The model training module is used to train the identity and posture feature decoupling model based on a training set with posture labels; The pedestrian re-identification module is used to remove the posture feature classification head based on the trained identity and posture feature decoupling model, calculate the deep feature similarity between the pedestrian image to be identified and the target pedestrian image, select the image with the largest similarity as the re-identification result output, and realize the re-identification of the pedestrian.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of a pedestrian re-identification method based on decoupling of identity and posture features as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of a pedestrian re-identification method based on decoupling of identity and posture features as described in any one of claims 1 to 7 are implemented.

Images