Personnel re-recognition system and method based on attitude guide generation and reasoning stage enhanced retrieval

Abstract

The invention discloses a personnel re-recognition system and method based on attitude guide generation and reasoning stage enhanced retrieval, and the method comprises the steps: carrying out the retrieval of a query image, selecting the first k results with the highest cosine similarity from a database image, and selecting m reference images based on the identity recognition accuracy; performing attitude extraction on the reference image; pixel features and attitude features are extracted and fused to obtain an attitude normalized generated image; identifying identity consistency of the generated image and the query image and attitude consistency of the generated image and the reference image; and in a reasoning stage, combining the query image with the generated image set, performing feature matching, and calculating weighted cosine similarity to obtain a retrieval-enhanced personnel re-identification task sorting result. According to the method, the target attitude is extracted by using the reference image in the reasoning stage, the generated image with the normalized attitude is guided to be obtained, and the query image and the generated image are combined for retrieval, so that the personnel re-recognition performance is improved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and artificial intelligence image recognition technology, specifically relating to a person re-identification system and method based on pose-guided generation and inference stage enhanced retrieval. Background Technology

[0002] With the rapid development of artificial intelligence methods based on deep neural networks, computer vision has been widely applied in various scenarios, solving numerous practical application needs. Among these, the person re-identification task, which aims to match images of the same person across camera perspectives, plays a crucial role in achieving intelligent security protection and the aggregation of long-form video surveillance data. For example, in intelligent security tasks, using data from multiple sets of cameras deployed in key areas, the person re-identification task can quickly locate suspicious or missing persons. In intelligent video surveillance tasks, the person re-identification task can be implemented quickly and efficiently, simultaneously classifying, continuously monitoring, and mapping the trajectories of a large number of people, significantly reducing the workload of manual judgment.

[0003] However, compared to the tremendous success of deep neural networks in classifying various types of objects, people re-identification requires classifying objects within the same "people" category, necessitating more detailed feature extraction from the images. Simultaneously, due to factors such as camera angle and variations in human pose, pedestrian images from different perspectives often exhibit significant background differences and pose misalignments, affecting the accuracy of people image matching. Therefore, robust and discriminative features are needed to aid neural networks in accurate identity classification.

[0004] Currently, methods for solving the task of person re-identification are mainly divided into two categories: methods based on convolutional neural networks and methods based on generative adversarial networks.

[0005] 1. Person Re-identification Based on Convolutional Neural Networks: Robust deep visual representations are extracted through classification or metric learning to obtain stable cross-viewpoint and intra-class discriminative features. Meanwhile, other works enhance feature representations through attribute classification, body part alignment, or human body parsing. While these techniques improve upon pose variations, they are sensitive to changes in data distribution and prone to fitting problems in real-world scenarios, leading to poor performance.

[0006] 2. Person Re-identification Based on Generative Adversarial Networks (GANs): This approach leverages the advantages of GANs in enhancing sample diversity by augmenting the model's training data to address sample distribution issues. This includes generating samples with different poses, angles, and styles, as well as generating cross-domain adaptive samples. However, while such methods can generate visually plausible samples, they often introduce redundant or low-quality data, hindering feature learning, slowing convergence, and leading to overfitting.

[0007] Therefore, based on the above considerations, it is necessary to propose a novel person re-identification system that can better handle changes in person posture and environment, while avoiding overfitting caused by noise and low-quality samples. By utilizing the posture features of reference images in the samples as guidance, new posture-normalized images are dynamically generated during the inference stage, thereby improving the accuracy and generalization ability of cross-view retrieval without altering the training process. Summary of the Invention

[0008] To address the shortcomings of the existing technologies, the present invention aims to provide a person re-identification system and method based on pose-guided generation and enhanced retrieval during the inference stage, thereby solving the problem of overfitting caused by the introduction of noise and low-quality samples due to retraining the original model in the existing technologies. The present invention improves the performance of person re-identification (Re-ID) by extracting the target pose using a reference image during the inference stage, guiding the generation of a pose-normalized image, and using the query image and the generated image together for retrieval.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] The present invention provides a person re-identification system based on posture-guided generation and inference-stage enhanced retrieval, comprising: a reference image selection module, a posture estimation module, a posture-guided image generation module, a dual discriminator constraint module, and an inference-stage enhanced retrieval module;

[0011] The reference image selection module retrieves the input query image, selects the top k results with the highest cosine similarity from the database images, and selects the m reference images with the highest accuracy based on the identity recognition accuracy.

[0012] The pose estimation module extracts the pose from the reference image, including 18 human pose key points.

[0013] The posture-guided image generation module includes an image encoder, a posture encoder, and an image generator. The image encoder is used to extract pixel features, the posture encoder is used to extract posture features, and the image generator is used to fuse pixel features and posture features to obtain a posture-normalized generated image.

[0014] The dual-discriminator constraint module includes an identity discriminator and a pose discriminator. The identity discriminator is used to identify the identity consistency between the generated image and the query image, and the pose discriminator is used to identify the pose consistency between the generated image and the reference image.

[0015] The enhanced retrieval module in the inference stage is used to guide the query image to generate images sequentially by using the poses of reference images in the reference image set during the inference stage, thereby obtaining the corresponding generated image set; the query image and the generated image set are combined and feature-matched with each database image in the database image set; and the ranking result of the enhanced retrieval person re-identification task is obtained by calculating the weighted cosine similarity.

[0016] Furthermore, the reference image selection module utilizes the Re-ID model (ResNet-50) to process the input query image set. Image search Perform a search from the database image collection. The top k results with the highest cosine similarity are selected, and the identity recognition accuracy is determined based on the results. The top m results are selected as reference images;

[0017] Calculate each query image in the query image set With each database image The cosine similarity between the images is calculated, and the k images with the highest cosine similarity are extracted from the database. The database image collection The function expression for cosine similarity is:

[0018] ;

[0019] in, For the Re-ID model, To query images pixel features, For database images pixel features, To query images Images in the database Similarity between them; calculate the query image The k database images with the highest cosine similarity The database image collection Identity recognition accuracy The corresponding expression is:

[0020] ;

[0021] in, To query images The corresponding identity recognition accuracy rate; For identity functions; An indicator function used to reflect and Whether they are the same, the corresponding expression is:

[0022] ;

[0023] The top m query images with the highest identity recognition accuracy Selected as reference image And store in a reference image set .

[0024] Furthermore, the pose estimation module utilizes a person pose recognition model (OpenPose) on the reference image. Pose extraction was performed to obtain the coordinates of 18 human pose key points. The human pose recognition model used a keypoint regression algorithm to predict the key points of the target pose. For key point j, the predicted coordinates were... , is represented as:

[0025] ;

[0026] in, H( is the true location of key point j) ) is the confidence mapping function for key points. Here is the diffusion coefficient; for the reference image The coordinates of the key points of the human body posture Represented as:

[0027] ;

[0028] in, and The first The x-coordinate and y-coordinate of key points in an individual's posture. This includes: nose, neck, shoulders, elbows, wrists, hips, knees, ankles, eyes, and ears.

[0029] Furthermore, the pose-guided image generation module extracts the query image through an image encoder. pixel features Reference image is extracted using an attitude encoder. Posture characteristics The image generator fuses pixel features with pose features to generate a pose-normalized image. The specific steps are as follows:

[0030] (21) Using a ResNet-50 backbone network pre-trained on ImageNet, the corresponding fully connected layers are removed, and it is used as the feature extraction network of the image encoder, including one input convolutional layer and four residual convolutional layers; the input query image is... Unified adjustment to The dimensions are then input into the image encoder to extract 2048-dimensional pixel features. The corresponding representation is:

[0031] ;

[0032] in, This is a function representation of the image encoder;

[0033] (22) Use the Gaussian spread algorithm to transform the reference image Human posture key point coordinates Expanded into the corresponding attitude heatmap The size of the attitude heatmap is consistent with the size of the query image, and is uniformly adjusted to... It has 18-channel attitude heatmap features corresponding to 18 key points. for:

[0034] ;

[0035] in, , For the first Two-dimensional horizontal and vertical coordinates corresponding to key points of an individual's posture; , These represent the distribution parameters of the attitude heatmap along the horizontal axis (X) and the vertical axis (Y), respectively; a ResNet-50 backbone network with fully connected layers removed is used as the attitude encoder. The feature extraction network extracts pose features from the input reference image pose heatmap, resulting in a set of 128-dimensional pose feature vectors, represented as follows:

[0036] ;

[0037] in, It is a 128-dimensional pose feature vector; This is a function representation of the attitude encoder;

[0038] (23) Upsampling is achieved by using deconvolution, batch normalization and activation functions. ( ), will query the image pixel features and reference image Posture characteristics and a 256-dimensional noise vector obtained by sampling from a Gaussian distribution. The images are fused and the details of the transformed images are restored layer by layer to generate a pose-normalized generated image. ,as follows:

[0039] ;

[0040] in, This is a function representation of the image generator.

[0041] Furthermore, the dual-discriminator constraint module uses an identity discriminator to authenticate the generated image. With query image Identity consistency is determined by a pose discriminator in the generated image. Compared with reference image To ensure pose consistency, the generated image has the same identity information as the query image and pose information consistent with the reference image. The specific steps are as follows:

[0042] (31) Using a classifier consisting of convolution, batch normalization, fully connected layers, and activation functions, the generated images generated by the image generator in this round of training are processed. Compared to the generated images produced by the image generator in the previous training round. Or query images The resulting sample pairs or To perform identity verification in order to distinguish... Belongs to the query image Or generate an image The training process is for the identity detector. With image generator Co-training enables the generation of images With query image Same identity information; co-trained binary cross-entropy identity loss for:

[0043] ;

[0044] in, Let be the mathematical expectation function. and They are respectively and Data distribution parameters;

[0045] (32) Using a classifier consisting of convolution, batch normalization, fully connected layers, and activation functions, for the reference image Posture characteristics , and the generated image Or reference image The resulting sample pairs or Pose discrimination is performed to distinguish between the reference image and the generated image; the training process is the pose discriminator. With image generator Co-training enables the generation of images With reference image Same pose features Binary cross-entropy pose loss during co-training for:

[0046] ;

[0047] in, Let be the mathematical expectation function. and They are respectively and Data distribution parameters.

[0048] Furthermore, the enhanced retrieval module in the inference phase, during the inference phase, uses a reference image set... Reference image in The posture guides the query image sequentially. Perform image generation to obtain the corresponding set of generated images. The query image and the generated image set are combined using a joint feature similarity score ranking algorithm, and then compared with the database image set. Each database image Cosine similarity is calculated, and the cosine similarities are weighted and sorted in descending order. Finally, the database images are ranked for the enhanced retrieval person re-identification task. The joint feature similarity score ranking algorithm SIMI is then used. The expression is:

[0049] ;

[0050] in, To ensure that the similarity scores of the query images are sparsely weighted, Weighted coefficients are used to calculate the similarity scores between the generated image set and the images in the database. The search results are sorted from highest to lowest based on the joint similarity score, and the mean precision (mAP) is used as the evaluation metric, as follows:

[0051] ;

[0052] in, Indicates the first The first image of the query The query accuracy of each search result. To query the number of images, For the first The query returns the number of correct samples of an image in the database image collection.

[0053] The present invention provides a person re-identification method based on pose-guided generation and inference stage enhanced retrieval, which, based on the above system, comprises the following steps:

[0054] 1) Calculate the similarity between the query image and the images in the database;

[0055] 2) From the k most similar images in the database, select m images as reference images;

[0056] 3) Extract pose key point information from the reference image;

[0057] 4) Extract pixel features from the query image;

[0058] 5) Convert the pose key points of the reference image into a heatmap and extract the corresponding pose features;

[0059] 6) Fuse pixel features and pose features to restore image details layer by layer and generate a pose-normalized generated image;

[0060] 7) Verify the identity of the generated image to ensure that the generated image has the same identity information as the queried image;

[0061] 8) Perform pose identification on the generated image to ensure that the generated image has the same pose information as the reference image;

[0062] 9) During the inference phase, the query image is sequentially guided to generate images by the poses of the reference images in the reference image set, thereby obtaining the corresponding generated image set; the obtained generated image set is combined with the query image and feature-matched with the database image to obtain the ranking result of the enhanced retrieval personnel re-identification task.

[0063] The beneficial effects of this invention are:

[0064] 1. This invention employs an enhanced framework for the pose-guided generation and inference stages. By utilizing the pose features of reference images in the samples to guide image generation, and using joint feature similarity scores for sorting and retrieval, it achieves plug-and-play functionality and improves the accuracy of cross-view retrieval without altering the original Re-ID model.

[0065] 2. This invention dynamically generates pose-normalized query images during the inference phase, effectively avoiding interference from data distribution changes caused by data generated during the training phase, avoiding the introduction of noise and redundancy into the Re-ID model, effectively reducing the risk of model overfitting, and improving recognition robustness.

[0066] 3. This invention uses an image and pose dual encoder structure to extract pixel and pose features respectively and fuse them into the image generator; and uses an identity and pose dual discriminator structure to supervise the identity information and pose information of the image generated by the image generator respectively, thereby achieving pose-normalized image generation. Attached Figure Description

[0067] Figure 1 This is a schematic diagram of the system of the present invention.

[0068] Figure 2 Schematic diagram of the module for selecting a reference image.

[0069] Figure 3 This is a schematic diagram of the pose-guided image generation module.

[0070] Figure 4 This is a schematic diagram of the dual discriminator constraint module.

[0071] Figure 5 A schematic diagram of the principle for enhancing the retrieval module during the reasoning stage. Detailed Implementation

[0072] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to embodiments and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0073] Reference Figures 1 to 5 As shown, the present invention provides a person re-identification system based on posture-guided generation and inference-stage enhanced retrieval, comprising: a reference image selection module, a posture estimation module, a posture-guided image generation module, a dual discriminator constraint module, and an inference-stage enhanced retrieval module;

[0074] The reference image selection module retrieves the input query image, selects the k results with the highest cosine similarity from the database images, and selects the m reference images with the highest accuracy based on the identity recognition accuracy; specifically as follows:

[0075] Using the Re-ID model (ResNet-50) on the input query image set Image search Perform a search from the database image collection. The top k results with the highest cosine similarity are selected, and the identity recognition accuracy is determined based on the results. The top m results are selected as reference images;

[0076] Calculate each query image in the query image set With each database image The cosine similarity between the images is calculated, and the k images with the highest cosine similarity are extracted from the database. The database image collection The function expression for cosine similarity is:

[0077] ;

[0078] in, For the Re-ID model, To query images pixel features, For database images pixel features, To query images Images in the database Similarity between them; calculate the query image The k database images with the highest cosine similarity The database image collection Identity recognition accuracy The corresponding expression is:

[0079] ;

[0080] in, To query images The corresponding identity recognition accuracy rate; For identity functions; An indicator function used to reflect and Whether they are the same, the corresponding expression is:

[0081] ;

[0082] The top m query images with the highest identity recognition accuracy Selected as reference image And store in a reference image set .

[0083] The pose estimation module extracts the pose from the reference image, including 18 human pose key points; specifically as follows:

[0084] Using the OpenPose human pose recognition model to analyze reference images Pose extraction was performed to obtain the coordinates of 18 human pose key points. The human pose recognition model used a keypoint regression algorithm to predict the key points of the target pose. For key point j, the predicted coordinates were... , is represented as:

[0085] ;

[0086] in, H( is the true location of key point j) ) is the confidence mapping function for key points. Here is the diffusion coefficient; for the reference image The coordinates of the key points of the human body posture Represented as:

[0087] ;

[0088] in, and The first The x-coordinate and y-coordinate of key points in an individual's posture. This includes: nose, neck, shoulders, elbows, wrists, hips, knees, ankles, eyes, and ears.

[0089] The pose-guided image generation module includes an image encoder, a pose encoder, and an image generator. The image encoder is used to extract pixel features, the pose encoder is used to extract pose features, and the image generator is used to fuse the pixel features and pose features to obtain a pose-normalized generated image. Specifically:

[0090] Extract the query image using an image encoder. pixel features Reference image is extracted using an attitude encoder. Posture characteristics The image generator fuses pixel features with pose features to generate a pose-normalized image. The specific steps are as follows:

[0091] (21) Using a ResNet-50 backbone network pre-trained on ImageNet, the corresponding fully connected layers are removed, and it is used as the feature extraction network of the image encoder, including one input convolutional layer and four residual convolutional layers; the input query image is... Unified adjustment to The dimensions are then input into the image encoder to extract 2048-dimensional pixel features. The corresponding representation is:

[0092] ;

[0093] in, This is a function representation of the image encoder;

[0094] (22) Use the Gaussian spread algorithm to transform the reference image Human posture key point coordinates Expanded into the corresponding attitude heatmap The size of the attitude heatmap is consistent with the size of the query image, and is uniformly adjusted to... It has 18-channel attitude heatmap features corresponding to 18 key points. for:

[0095] ;

[0096] in, , For the first Two-dimensional horizontal and vertical coordinates corresponding to key points of an individual's posture; , These represent the distribution parameters of the attitude heatmap along the horizontal axis (X) and the vertical axis (Y), respectively; a ResNet-50 backbone network with fully connected layers removed is used as the attitude encoder. The feature extraction network extracts pose features from the input reference image pose heatmap, resulting in a set of 128-dimensional pose feature vectors, represented as follows:

[0097] ;

[0098] in, It is a 128-dimensional pose feature vector; This is a function representation of the attitude encoder;

[0099] (23) Upsampling is achieved by using deconvolution, batch normalization and activation functions. ( ), will query the image pixel features and reference image Posture characteristics and a 256-dimensional noise vector obtained by sampling from a Gaussian distribution. The images are fused and the details of the transformed images are restored layer by layer to generate a pose-normalized generated image. :

[0100] ;

[0101] in, This is a function representation of the image generator.

[0102] The dual-discriminator constraint module includes an identity discriminator and a pose discriminator. The identity discriminator is used to verify the identity consistency between the generated image and the query image, and the pose discriminator is used to verify the pose consistency between the generated image and the reference image; specifically as follows:

[0103] Image generation is authenticated by an identity authenticator. With query image Identity consistency is determined by a pose discriminator in the generated image. Compared with reference image To ensure pose consistency, the generated image has the same identity information as the query image and pose information consistent with the reference image. The specific steps are as follows:

[0104] (31) Using a classifier consisting of convolution, batch normalization, fully connected layers, and activation functions, the generated images generated by the image generator in this round of training are processed. Compared to the generated images produced by the image generator in the previous training round. Or query images The resulting sample pairs or To perform identity verification in order to distinguish... Belongs to the query image Or generate an image The training process is for the identity detector. With image generator Co-training enables the generation of images With query image Same identity information; co-trained binary cross-entropy identity loss for:

[0105] ;

[0106] in, Let be the mathematical expectation function. and They are respectively and Data distribution parameters;

[0107] (32) Using a classifier consisting of convolution, batch normalization, fully connected layers, and activation functions, for the reference image Posture characteristics , and the generated image Or reference image The resulting sample pairs or Pose discrimination is performed to distinguish between the reference image and the generated image; the training process is the pose discriminator. With image generator Co-training enables the generation of images With reference image Same pose features Binary cross-entropy pose loss during co-training for:

[0108] ;

[0109] in, Let be the mathematical expectation function. and They are respectively and Data distribution parameters.

[0110] The enhanced retrieval module in the inference phase is used to guide the generation of query images sequentially by using the poses of reference images in the reference image set during the inference phase, thereby obtaining a corresponding generated image set. The query images and the generated image set are then combined and feature-matched with each database image in the database image set. By calculating the weighted cosine similarity, the ranking result of the enhanced retrieval person re-identification task is obtained, as detailed below:

[0111] During the reasoning phase, reference image sets are used. Reference image in The posture guides the query image sequentially. Perform image generation to obtain the corresponding set of generated images. The query image and the generated image set are combined using a joint feature similarity score ranking algorithm, and then compared with the database image set. Each database image Cosine similarity is calculated, and the cosine similarities are weighted and sorted in descending order. Finally, the database images are ranked for the enhanced retrieval person re-identification task. The joint feature similarity score ranking algorithm SIMI is then used. The expression is:

[0112] ;

[0113] in, To ensure that the similarity scores of the query images are sparsely weighted, Weighted coefficients are used to calculate the similarity scores between the generated image set and the images in the database. The search results are sorted from highest to lowest based on the joint similarity score, and the mean precision (mAP) is used as the evaluation metric, as follows:

[0114] ;

[0115] in, Indicates the first The first image of the query The query accuracy of each search result. To query the number of images, For the first The query returns the number of correct samples of an image in the database image collection.

[0116] The present invention provides a person re-identification method based on pose-guided generation and inference stage enhanced retrieval, which, based on the above system, comprises the following steps:

[0117] 1) Calculate the similarity between the query image and the images in the database;

[0118] 2) From the k most similar images in the database, select m images as reference images;

[0119] 3) Extract pose key point information from the reference image;

[0120] 4) Extract pixel features from the query image;

[0121] 5) Convert the pose key points of the reference image into a heatmap and extract the corresponding pose features;

[0122] 6) Fuse pixel features and pose features to restore image details layer by layer and generate a pose-normalized generated image;

[0123] 7) Verify the identity of the generated image to ensure that the generated image has the same identity information as the queried image;

[0124] 8) Perform pose identification on the generated image to ensure that the generated image has the same pose information as the reference image;

[0125] 9) During the inference phase, the query image is sequentially guided to generate images by the poses of the reference images in the reference image set, thereby obtaining the corresponding generated image set; the obtained generated image set is combined with the query image and feature-matched with the database image to obtain the ranking result of the enhanced retrieval personnel re-identification task.

[0126] This invention has many specific applications. The above description is only a preferred embodiment of this invention. It should be noted that for those skilled in the art, several improvements can be made without departing from the principle of this invention, and these improvements should also be considered within the scope of protection of this invention.

Claims

1. A person re-identification system based on gesture-guided generation and inference stage enhanced retrieval, characterized in that, include: Reference image selection module, pose estimation module, pose-guided image generation module, dual discriminator constraint module, and inference stage enhanced retrieval module; The reference image selection module retrieves the input query image, selects the top k results with the highest cosine similarity from the database images, and selects the m reference images with the highest accuracy based on the identity recognition accuracy. The pose estimation module extracts the pose from the reference image, including 18 human pose key points. The posture-guided image generation module includes an image encoder, a posture encoder, and an image generator. The image encoder is used to extract pixel features, the posture encoder is used to extract posture features, and the image generator is used to fuse pixel features and posture features to obtain a posture-normalized generated image. The dual-discriminator constraint module includes an identity discriminator and a pose discriminator. The identity discriminator is used to identify the identity consistency between the generated image and the query image, and the pose discriminator is used to identify the pose consistency between the generated image and the reference image. The enhanced retrieval module in the inference stage is used to guide the query images to generate images sequentially by using the poses of reference images in the reference image set during the inference stage, thereby obtaining the corresponding generated image set. The query image is combined with the generated image set and then matched with each image in the database image set. By calculating the weighted cosine similarity, the ranking result of the enhanced retrieval person re-identification task is obtained.

2. The personnel re-identification system based on gesture-guided generation and inference stage enhanced retrieval according to claim 1, characterized in that, The reference image selection module uses the Re-ID model to select the input query image set. Image search Perform a search from the database image collection. The top k results with the highest cosine similarity are selected, and the identity recognition accuracy is determined based on the results. The top m results are selected as reference images; Calculate each query image in the query image set With each database image The cosine similarity between the images is calculated, and the k images with the highest cosine similarity are extracted from the database. The database image collection The function expression for cosine similarity is: ； in, For the Re-ID model, To query images pixel features, For database images pixel features, To query images Images in the database Similarity between them; calculate the query image The k database images with the highest cosine similarity The database image collection Identity recognition accuracy The corresponding expression is: ； in, To query images The corresponding identity recognition accuracy rate; For identity functions; An indicator function used to reflect and Whether they are the same, the corresponding expression is: ； The top m query images with the highest identity recognition accuracy Selected as reference image And store in a reference image set .

3. The personnel re-identification system based on gesture-guided generation and inference stage enhanced retrieval according to claim 2, characterized in that, The pose estimation module uses a human pose recognition model to analyze a reference image. Pose extraction was performed to obtain the coordinates of 18 human pose key points. The human pose recognition model used a keypoint regression algorithm to predict the key points of the target pose. For key point j, the predicted coordinates were... , is represented as: ； in, H( is the true location of key point j) ) is the confidence mapping function for key points. Here is the diffusion coefficient; for the reference image The coordinates of the key points of the human body posture Represented as: ； in, and The first The x-coordinate and y-coordinate of key points in an individual's posture. This includes: nose, neck, shoulders, elbows, wrists, hips, knees, ankles, eyes, and ears.

4. The person re-identification system based on gesture-guided generation and inference stage enhanced retrieval according to claim 3, characterized in that, The posture-guided image generation module extracts the query image using an image encoder. pixel features The reference image is extracted using an attitude encoder. Posture characteristics The image generator fuses pixel features with pose features to generate a pose-normalized image. The specific steps are as follows: (21) Using a ResNet-50 backbone network pre-trained on ImageNet, the corresponding fully connected layers are removed, and it is used as the feature extraction network of the image encoder, including one input convolutional layer and four residual convolutional layers; the input query image is... Unified adjustment to The dimensions are then input into the image encoder to extract 2048-dimensional pixel features. The corresponding representation is: ； in, This is a function representation of the image encoder; (22) Use the Gaussian spread algorithm to transform the reference image Human posture key point coordinates Expanded into the corresponding attitude heatmap The size of the attitude heatmap is consistent with the size of the query image, and is uniformly adjusted to... It has 18-channel attitude heatmap features corresponding to 18 key points. for: ； in, , For the first Two-dimensional horizontal and vertical coordinates corresponding to key points of an individual's posture; , These represent the distribution parameters of the attitude heatmap along the horizontal axis (X) and the vertical axis (Y), respectively; a ResNet-50 backbone network with fully connected layers removed is used as the attitude encoder. The feature extraction network extracts pose features from the input reference image pose heatmap, resulting in a set of 128-dimensional pose feature vectors, represented as follows: ； in, It is a 128-dimensional pose feature vector; This is a function representation of the attitude encoder; (23) Upsampling is achieved by using deconvolution, batch normalization and activation functions. ( ), will query the image pixel features and reference image Posture characteristics and a 256-dimensional noise vector obtained by sampling from a Gaussian distribution. The images are fused and the details of the transformed images are restored layer by layer to generate a pose-normalized generated image. ,as follows: ； in, This is a function representation of the image generator.

5. The person re-identification system based on gesture-guided generation and inference stage enhanced retrieval according to claim 4, characterized in that, The dual-discriminator constraint module uses an identity discriminator to authenticate the generated image. With query image Identity consistency is determined by a pose discriminator in the generated image. Compared with reference image To ensure pose consistency, the generated image has the same identity information as the query image and pose information consistent with the reference image. The specific steps are as follows: (31) Using a classifier consisting of convolution, batch normalization, fully connected layers, and activation functions, the generated images generated by the image generator in this round of training are processed. Compared to the generated images produced by the image generator in the previous training round. Or search for images The resulting sample pairs or To perform identity verification in order to distinguish... Image belonging to the query Or generate an image ; The training process is for the identity detector. With image generator Co-training enables the generation of images With query image Same identity information; co-trained binary cross-entropy identity loss for: ； in, Let be the mathematical expectation function. and They are respectively and Data distribution parameters; (32) Using a classifier consisting of convolution, batch normalization, fully connected layers, and activation functions, for the reference image Posture characteristics , and the generated image Or reference image The resulting sample pairs or Pose discrimination is performed to distinguish between the reference image and the generated image; the training process is the pose discriminator. With image generator Co-training enables the generation of images With reference image Same pose features Binary cross-entropy pose loss during co-training for: ； in, Let be the mathematical expectation function. and They are respectively and Data distribution parameters.

6. The person re-identification system based on gesture-guided generation and inference stage enhanced retrieval according to claim 5, characterized in that, The enhanced retrieval module in the inference phase, during the inference phase, uses a set of reference images. Reference image in The posture guides the query image sequentially Perform image generation to obtain the corresponding set of generated images. The query image and the generated image set are combined using a joint feature similarity score ranking algorithm, and then compared with the database image set. Each database image Perform cosine similarity calculation, weight each cosine similarity, and sort them in descending order; The database image ranking results for the person re-identification task under enhanced retrieval are obtained; SIMI (Joint Feature Similarity Score Ranking Algorithm) The expression is: ； in, To ensure that the similarity scores of the query images are sparsely weighted, Weighted coefficients are used to calculate the similarity scores between the generated image set and the images in the database. The search results are sorted from highest to lowest based on the joint similarity score, and the mean precision (mAP) is used as the evaluation metric, as follows: ； in, Indicates the first The first image of the query The query accuracy of each search result. To query the number of images, For the first The query returns the number of correct samples of an image in the database image collection.

7. A method for enhancing person re-identification based on gesture-guided generation and inference stages, based on the system described in any one of claims 1-6, characterized in that, The steps are as follows: 1) Calculate the similarity between the query image and the images in the database; 2) From the k most similar images in the database, select m images as reference images; 3) Extract pose key point information from the reference image; 4) Extract pixel features from the query image; 5) Convert the pose key points of the reference image into a heatmap and extract the corresponding pose features; 6) Fuse pixel features and pose features to restore image details layer by layer and generate a pose-normalized generated image; 7) Verify the identity of the generated image to ensure that the generated image has the same identity information as the queried image; 8) Perform pose identification on the generated image to ensure that the generated image has the same pose information as the reference image; 9) During the inference phase, the query image is sequentially guided to generate images by the poses of the reference images in the reference image set, thereby obtaining the corresponding generated image set; the obtained generated image set is combined with the query image and feature-matched with the database image to obtain the ranking result of the enhanced retrieval personnel re-identification task.

Images