A method and system for re-identifying pedestrians after changing clothes based on multi-positive classification learning

Abstract

This paper discloses a method and system for re-identifying people with clothing changes based on multi-positive classification learning. This method increases the generalizability of the dataset through a preprocessing data augmentation strategy, employs an improved ViT to enhance the extraction and representation capabilities of image features, and uses a batch normalization layer to make the distribution of features belonging to the same person more compact. A multi-positive clothing classification loss function is designed to learn other discriminative information unrelated to clothing, while simultaneously integrating identity cross-entropy loss, identity ternary loss, and clothing cross-entropy loss for joint optimization. Finally, this method improves the re-identification accuracy of people with the same and different clothing, enhancing overall performance. Furthermore, because it uses only RGB modalities, it offers the advantages of low cost, high speed, and high accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of image recognition and computer vision, and in particular relates to a method and system for re-identifying pedestrians who have changed clothes based on multi-positive class classification learning. Background Art

[0002] In recent years, with the success of deep learning-based video image information analysis and recognition, video surveillance has become a core tool for public safety, traffic management, and enterprise site management. Front-end cameras generate massive amounts of video footage daily. Uploading surveillance footage to the cloud can eliminate data silos and facilitate intelligent analysis. With the advent of smart cities, intelligent video surveillance for campus and urban security is rapidly developing.

[0003] Person re-identification (PIR) is a key tool in intelligent video surveillance, designed to search for target individuals in surveillance videos captured at different locations and times. It is closely related to people's work, life, and learning, and holds significant application value in today's increasingly vast surveillance networks. PIR can identify target individuals at different times, locations, and positions, resolving the challenge of manually searching for relevant clues in massive amounts of video. This not only saves manpower and resources but also improves retrieval efficiency. However, most current research assumes that pedestrians' clothing remains constant. In long-term PIR, clothing changes are inevitable, making the issue of PIR involving clothing changes unavoidable. PIR with clothing changes holds great promise in the current era of big data and the interconnectedness of everything. Applications include intelligent customer recommendations in shopping malls, unmanned systems, and photo album clustering.

[0004] Existing methods for re-identifying people with clothing changes utilize multimodal inputs (e.g., skeleton keypoints, silhouettes, body contours, and 3D shapes) or learn disentangled representations to avoid interference from clothing. However, multimodal-based methods require additional models or devices to capture multimodal information, and learning disentangled representations is often very time-consuming. In fact, the original RGB modality contains rich information unrelated to clothing, which is largely underutilized in current methods. Summary of the Invention

[0005] In order to address the shortcomings of the existing technology, the present invention proposes a method and system for pedestrian re-identification after changing clothes based on multi-classification adversarial learning. The method and system can improve the performance of the model in pedestrian re-identification after changing clothes as much as possible without sacrificing the performance of the model in traditional pedestrian re-identification, thereby achieving pedestrian re-identification with higher accuracy. At the same time, since only RGB mode is used, the method has the outstanding advantages of low cost, fast speed and high precision.

[0006] The technical solutions provided by the present invention are as follows:

[0007] On the one hand, a method for re-identifying people with changed clothes based on multi-positive classification learning includes:

[0008] Step 1: Preprocess the clothing-changing pedestrian re-identification data to obtain a training sample set;

[0009] Step 2: Build a pedestrian re-identification model;

[0010] The pedestrian re-identification model includes a backbone network, a processing head, and a classifier connected in sequence. The backbone network is an improved ViT network. The processing head includes a global pooling layer and a batch normalization processing layer connected in sequence. The classifier includes an identity classifier and a clothing classifier.

[0011] Step 3: Generate batch images with multiple identity labels and multiple clothing labels based on the identity labels and clothing labels of the pedestrians in the training sample set processed in step 1;

[0012] Each batch of images contains at least 4 images of each identity label, and the images of the same identity label contain at least two different clothing labels;

[0013] Step 4: Use the batch images obtained in step 3 to train the constructed pedestrian re-identification model and set an integrated loss function. The integrated loss function includes calculating the identity triple loss based on the normalized feature vector, calculating the identity cross entropy loss based on the identity classification features, calculating the clothing cross entropy loss based on the clothing classification features, and calculating the multi-positive clothing classification loss;

[0014] Step 5: Input the pedestrian image to be queried into the trained pedestrian re-identification model to obtain the pedestrian feature vector, and then calculate the similarity with the pedestrian feature vector in the pedestrian database to obtain the pedestrian label corresponding to the pedestrian feature vector with the highest similarity greater than the set threshold, and complete the pedestrian re-identification.

[0015] The range is -1 to 1. The closer to 1, the higher the similarity. The set threshold is not fixed and the optimal threshold needs to be determined based on the actual recognition situation.

[0016] Furthermore, the improved ViT network refers to the use of the ViT-Base network, and the pre-trained model is trained on the ImageNet-21k dataset. The specific working process is as follows:

[0017] A1: Use a sliding window to split the input image for training into patches with overlapping pixels. The step size is represented by S and the patch size is represented by P. The size of the overlapping area of two adjacent patches is (PS)×P. The input image with a resolution of H×W will be divided into N patches. B Patches:

[0018]

[0019] Where [] is rounded down, H, W are the height and width of the image, P, S are the patch size and the step size of the sliding window, and P>S;

[0020] Use sliding windows to segment the image into overlapping patches to avoid losing local neighboring structures around the patches;

[0021] A2: Each patch is linearly projected into the corresponding primitive through the fully connected layer;

[0022] A3: After performing 2D linear interpolation on the primitives, a position code is added to each primitive to record the position information of the input image. 2D linear interpolation can be used to process images of any size.

[0023] A4: The position-encoded primitives are passed through multiple transformation layers to output a global high-dimensional feature matrix.

[0024] Furthermore, the multi-layer conversion process is as follows:

[0025] After the position-encoded primitives are normalized, they pass through 12 attention heads, and finally the 12-head encodings are spliced together, then pass through a normalization layer, and finally pass through a multi-layer perceptron, and then projected back to output a high-dimensional global feature matrix.

[0026] Furthermore, the process of training the constructed person re-identification model is as follows:

[0027] Step 4.1: Input the image into the ViT-Base network to extract the high-dimensional feature matrix;

[0028] Step 4.2: Input the high-dimensional matrix features into the global pooling layer and perform a global pooling operation to obtain the feature vector f t , and then pass through the batch normalization processing layer to perform batch normalization operation to obtain the normalized feature vector f i ;

[0029] The batch normalization balances each dimension of the feature vector so that the features are Gaussian distributed around the hypersphere, making the identity cross entropy loss L ID It is easier to converge. In addition, batch normalization reduces the feature vector f t Loss L for identity triples Tri Constraints. From the identity cross entropy loss L ID The fewer the constraints, the more identity triples lose L Tri It is easier to converge, and batch normalization makes the distribution of features belonging to the same person more compact.

[0030] Step 4.3: Input the normalized feature vector into a fully connected layer simultaneously, wherein the fully connected layer includes an identity classifier and a clothing classifier, to obtain identity classification features and clothing classification features;

[0031] Step 4.4: Calculate the identity ternary loss based on the normalized feature vector, calculate the identity cross entropy loss based on the identity classification features, calculate the clothing cross entropy loss and the multi-positive clothing classification loss based on the clothing classification features, and then calculate the weighted sum of the four losses to obtain the overall loss;

[0032] Step 4.5: Calculate the overall loss gradient backpropagation, with the goal of minimizing the overall loss, iteratively update the weight parameters of the pedestrian re-identification model, use different batches of images with multiple identity labels and multiple clothing labels, and repeat steps 4.1 to 4.5 until the set number of iterations is reached.

[0033] Furthermore, the identity ternary loss L Tri as follows:

[0034] L Tri =[d p -d n +α] +

[0035] Among them, L Tri represents the identity triple loss, d p and d n is the feature vector f extracted from the positive sample pair and negative sample pair of identity labels in the training sample set t The Euclidean distance between them, α is the triple loss L Tri The boundary of [z] + Equal to max(z,0), that is, the maximum value between 0 and z;

[0036] The identity cross entropy loss L ID as follows:

[0037]

[0038]

[0039] Among them, L ID represents the identity cross entropy loss, N is the batch size during training, N ID The total number of identity labels in the training sample set, f i ID represents identity classification features, y ID represents the true identity label of the image, Represents the probability that the identity label of the image prediction is the i-th category, τ∈R + is the temperature parameter, R + represents a positive real number, In order to prevent the pedestrian re-identification model from overfitting the trained identity labels, ε is a label smoothing constant used to penalize the model's prediction of the identity, and e is a natural base;

[0040] The clothing cross entropy loss L C as follows:

[0041]

[0042] Among them, L C represents the cross entropy loss of clothes, N C is the total number of clothing labels in the training sample set, f i c Represents clothing classification features, y C represents the true value clothing label of the image, Indicates the probability that the clothing label predicted by the image is the i-th category;

[0043] The multi-positive clothing classification loss L CA as follows:

[0044]

[0045]

[0046] Among them, L CA represents the multi-positive clothing classification loss, is with A collection of clothing categories with the same identity, is with A collection of clothing categories with different identities; K is The number of categories in , q(c) is the weight of the cross entropy loss of the c-th clothing class; the positive class with the same clothing, The weights of the positive classes with different clothes are different from the positive classes with the same clothes: Positive categories with different clothing: and 0<δ<1 is a hyperparameter, which can be used to adjust the penalty intensity for each category by adjusting δ;

[0047] The overall loss function is as follows:

[0048] L=L ID +L C +L CA +λL Tri

[0049] Among them, L represents the overall loss function, λ represents the identity triple loss L Tri The weight parameter of .

[0050] Furthermore, the Adam optimization algorithm is used to optimize the training process of the pedestrian re-identification model. The maximum number of rounds of training on the training sample set is 60 rounds, and the multi-positive clothing classification loss L CA For training after 30 rounds, the initial learning rate is 3.5×10 -4 , and then the learning rate is divided by 10 after every 20 rounds until the pedestrian re-identification model converges or reaches the maximum number of iterations, and the final pedestrian re-identification model is obtained.

[0051] Furthermore, the cosine similarity between the pedestrian feature vector of the pedestrian image to be queried and the pedestrian feature vector in the pedestrian database is calculated:

[0052]

[0053] Among them, f q represents the pedestrian feature vector of the pedestrian image to be queried, f g It represents the pedestrian feature vector in the pedestrian database, n is the vector dimension size, and the cosine similarity ranges from -1 to 1. The closer the value is to 1, the higher the similarity.

[0054] Furthermore, the preprocessing includes image scaling, random horizontal flipping, random cropping, random erasing, and automatic enhancement.

[0055] The images in the training set use all the data augmentation strategies mentioned above, and the images in the test set only use image scaling and random horizontal flipping.

[0056] On the other hand, a system based on the above-mentioned method for re-identifying people after changing clothes based on multi-positive classification learning includes:

[0057] Training sample set construction unit: used to pre-process the clothing-changing pedestrian re-identification data to obtain the training sample set;

[0058] Person re-identification model construction unit: The person re-identification model includes a backbone network, a processing head, and a classifier connected in sequence. The backbone network is an improved ViT network. The processing head includes a global pooling layer and a batch normalization processing layer connected in sequence. The classifier includes an identity classifier and a clothing classifier.

[0059] Batch image construction unit: Generates batch images with multiple identity labels and multiple clothing labels based on the identity labels and clothing labels of pedestrians in the constructed training sample set;

[0060] Each batch of images contains at least 4 images of each identity label, and the images of the same identity label contain at least two different clothing labels;

[0061] Training unit: Use batch images to train the constructed pedestrian re-identification model and set an integrated loss function. The integrated loss function includes calculating the identity ternary loss based on the normalized feature vector, calculating the identity cross entropy loss based on the identity classification features, calculating the clothing cross entropy loss based on the clothing classification features, and calculating the multi-positive clothing classification loss;

[0062] Identification unit: The pedestrian image to be queried is input into the trained pedestrian re-identification model to obtain the pedestrian feature vector, and then the similarity is calculated with the pedestrian feature vector in the pedestrian database. The pedestrian label corresponding to the pedestrian feature vector with the highest similarity greater than the set threshold is obtained to complete pedestrian re-identification.

[0063] On the other hand, a readable storage medium stores a computer program, which is called by a processor to implement: the steps of the above-mentioned method for re-identifying pedestrians who have changed clothes based on multi-positive class classification learning.

[0064] Beneficial effects

[0065] The technical solution of the present invention can effectively complete the identification of target pedestrians when pedestrians change their clothes in an actual monitoring system, and is more robust to the identification of obscured pedestrians, thereby improving the accuracy of pedestrian re-identification in long-term monitoring systems. It effectively meets the urgent need for real-time identification of pedestrians who change their clothes in actual long-term monitoring scenarios, and greatly promotes the development of the field of intelligent security monitoring.

[0066] Compared with the existing clothing-changing pedestrian re-identification method, the advantages of the present invention are as follows:

[0067] 1. Preprocessing: By augmenting the dataset, the training data is distributed widely, covering a variety of situations, and enhancing the generalization and robustness of the clothing-changing person re-identification model.

[0068] 2. The technical solution of the present invention uses the ViT network as the backbone network to extract high-dimensional features of images. Due to the characteristics of ViT, theoretically, the larger the amount of data, the better the performance, and there is no upper limit;

[0069] 3. To address the difficulty of identifying pedestrians who change clothes in long-term monitoring systems, the present invention proposes a multi-positive clothing classification loss, which effectively mines information unrelated to clothing in RGB images. With the addition of the multi-positive classification loss, the model focuses more on the torso while also taking into account the head and shoes. The model learns information about body shape and posture, which it then uses as a basis for determining pedestrian identity.

[0070] 4. Compared with other clothing-changing pedestrian re-identification methods, the technical solution of the present invention only uses RGB modality data, which greatly reduces the model's inference time. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 It is the overall flow chart of the technical solution of the present invention;

[0072] Figure 2 This is the structural diagram of the clothing-changing pedestrian re-identification model proposed in the technical solution of the present invention, where Figure (a) shows the overall model architecture and Figure (b) shows the specific structure of the conversion layer;

[0073] Figure 3 This is a schematic diagram of the effect of the data enhancement strategy (random horizontal flipping, random erasing, and random cropping) used in the technical solution of the present invention;

[0074] Figure 4 This is a comparison diagram of the learning features of the loss function used in the technical solution of the present invention and other loss functions. Figure (a) shows two pictures of the same person changing clothes. Figure (b) shows the learning feature diagram using only identity ternary loss and identity cross entropy loss; Figure (c) shows the learning feature diagram using identity ternary loss, identity cross entropy loss, clothing cross entropy loss and multi-positive clothing classification loss. DETAILED DESCRIPTION

[0075] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0076] The embodiment of the present invention provides a method for re-identifying pedestrians who have changed clothes based on multi-positive classification learning. Figure 1 As shown, it is an overall flow chart of the technical solution of the present invention, which includes the following steps:

[0077] Step 1: Use data augmentation strategies to process the dataset of clothing-changing pedestrian re-identification;

[0078] Data augmentation effects such as Figure 3 As shown in Figure 2, the image processing steps for the training sample set are as follows:

[0079] (1) Scale the input image to a uniform size of 256*256;

[0080] (2) Randomly flip the image horizontally with a probability of 0.5;

[0081] (3) Fill a circle of 0 pixels with a width of 20 around the image and randomly crop a 256*256 image from the image;

[0082] (4) Randomly erase the image with a probability of 0.5. The aspect ratios of the erased area rectangle and the image are 0.3 and 0.33 respectively. Each pixel in the erased area is assigned the average value of the image pixels.

[0083] (5) Automatically enhance the image using the image enhancement strategies learned from other tasks;

[0084] The image processing steps for the test set are as follows:

[0085] (1) Scale the input image to a uniform size of 256*256;

[0086] (2) Randomly flip the image horizontally with a probability of 0.5;

[0087] Step 2: Build a pedestrian re-identification model;

[0088] The pedestrian re-identification model includes a backbone network, a processing head, and a classifier connected in sequence. The backbone network is an improved ViT network. The processing head includes a global pooling layer and a batch normalization processing layer connected in sequence. The classifier includes an identity classifier and a clothing classifier. The pre-trained model of the ViT network is trained on the ImageNet-21k dataset, and the patch size of the pre-trained model is set to 16×16.

[0089] First, the training image is divided into patches, and then each patch is linearly projected into the corresponding primitive through the fully connected layer. Then, a position code is added to each primitive to record the position information of the image, and the global features are output after passing through multiple layers of conversion layers.

[0090] The improved ViT network (such as Figure 2 (a) refers to the use of the ViT-Base network, and the pre-trained model is trained on the ImageNet-21k dataset, and the patch size of the pre-trained model is set to 16×16. The specific working process is as follows:

[0091] A1: Use a sliding window to split the input image for training into patches with overlapping pixels. The step size is represented by S and the patch size is represented by P. The size of the overlapping area of two adjacent patches is (PS)×P. The input image with a resolution of H×W will be divided into N patches. B Patches:

[0092]

[0093] Where [] is rounded down, H, W are the height and width of the image, P, S are the patch size and the step size of the sliding window, and P>S; the value of P is 16, which is consistent with the size of the supplementary size of the pre-trained model;

[0094] Use sliding windows to segment the image into overlapping patches to avoid losing local neighboring structures around the patches;

[0095] A2: Each patch is linearly projected into the corresponding primitive through the fully connected layer;

[0096] A3: After performing 2D linear interpolation on the primitives, a position code is added to each primitive to record the position information of the input image. 2D linear interpolation can be used to process images of any size.

[0097] A4: The position-encoded primitives are passed through multiple transformation layers to output a global high-dimensional feature matrix.

[0098] ViT is a general term for a type of network structure, which can be divided into ViT-B, ViT-L, and ViT-H according to the size of parameters. ViT-B refers to ViT-Base.

[0099] like Figure 2 As shown in (b), the multi-layer conversion process is as follows:

[0100] After the position-encoded primitives are normalized, they pass through 12 attention heads, and finally the 12-head encodings are spliced together, then pass through a normalization layer, and finally pass through a multi-layer perceptron, and then projected back to output a high-dimensional global feature matrix.

[0101] The multi-layer perceptron generally amplifies the dimension by four times, and the output dimension of the multi-layer perceptron is generally determined based on the total number of pedestrians in the data set;

[0102] Step 3: Generate batch images with multiple identity labels and multiple clothing labels based on the identity labels and clothing labels of the pedestrians in the training sample set processed in step 1;

[0103] Each batch of images contains at least 4 images of each identity label, and the images of the same identity label contain at least two different clothing labels;

[0104] In this embodiment, the batch size is set to 64, each batch contains 8 people with different identities, each person contains 8 images, and the 8 images contain at least two images with different clothing labels. The batch images are iteratively updated after each training round.

[0105] Step 4: Use the batch images obtained in step 3 to train the constructed pedestrian re-identification model and set an integrated loss function. The integrated loss function includes calculating the identity triple loss based on the normalized feature vector, calculating the identity cross entropy loss based on the identity classification features, calculating the clothing cross entropy loss based on the clothing classification features, and calculating the multi-positive clothing classification loss;

[0106] The process of training the constructed person re-identification model is as follows:

[0107] Step 4.1: Input the image into the ViT-Base network to extract the high-dimensional feature matrix;

[0108] Step 4.2: Input the high-dimensional matrix features into the global pooling layer and perform a global pooling operation to obtain the feature vector f t , and then pass through the batch normalization processing layer to perform batch normalization operation to obtain the normalized feature vector f i ;

[0109] The batch normalization balances each dimension of the feature vector so that the features are Gaussian distributed around the hypersphere, making the identity cross entropy loss L ID It is easier to converge. In addition, batch normalization reduces the feature vector f t Loss L for identity triples Tri Constraints. From the identity cross entropy loss L ID The fewer the constraints, the more identity triples lose L Tri It is easier to converge, and batch normalization makes the distribution of features belonging to the same person more compact.

[0110] Step 4.3: Input the normalized feature vector into a fully connected layer simultaneously, wherein the fully connected layer includes an identity classifier and a clothing classifier, to obtain identity classification features and clothing classification features;

[0111] The different functions of the classifier are obtained by the loss function constraints.

[0112] Step 4.4: Calculate the identity ternary loss based on the normalized feature vector, calculate the identity cross entropy loss based on the identity classification features, calculate the clothing cross entropy loss and the multi-positive clothing classification loss based on the clothing classification features, and then calculate the weighted sum of the four losses to obtain the overall loss;

[0113] Step 4.5: Calculate the overall loss gradient backpropagation, with the goal of minimizing the overall loss, iteratively update the weight parameters of the pedestrian re-identification model, use different batches of images with multiple identity labels and multiple clothing labels, and repeat steps 4.1 to 4.5 until the set number of iterations is reached.

[0114] Furthermore, the identity ternary loss L Tri as follows:

[0115] L Tri =[d p -d n +α] +

[0116] Among them, L Tri represents the identity triple loss, d p and d n is the feature vector f extracted from the positive sample pair and negative sample pair of identity labels in the training sample set t The Euclidean distance between them, α is the triple loss L Tri The boundary of [z]+ Equal to max(z,0), that is, the maximum value between 0 and z;

[0117] In the embodiment of the present invention, α is set to 0.3.

[0118] The identity cross entropy loss L ID as follows:

[0119]

[0120]

[0121] Among them, L ID represents the identity cross entropy loss, N is the batch size during training, N ID The total number of identity labels in the training sample set, f i ID represents identity classification features, y ID represents the true identity label of the image, Represents the probability that the identity label of the image prediction is the i-th category, τ∈R + is the temperature parameter, R + represents a positive real number, represents the identity label to prevent the pedestrian re-identification model from overfitting the trained identity label, and ε is the label smoothing constant used to penalize the model's prediction of the identity;

[0122] In the embodiment of the present invention, ε is set to 0.1.

[0123] The clothing cross entropy loss L C as follows:

[0124]

[0125] Among them, L C represents the cross entropy loss of clothes, N C is the total number of clothing labels in the training sample set, f i c Represents clothing classification features, y C represents the true value clothing label of the image, Indicates the probability that the clothing label predicted by the image is the i-th category;

[0126] The multi-positive clothing classification loss L CA All clothing classes belonging to the same identity are mutually positive. For example, given a sample x i , all belonging to their identity category All clothing categories are defined as their normal clothing categories;

[0127] The multi-positive clothing classification loss L CA as follows:

[0128]

[0129] Because in a long-term person re-identification system, re-identification of pedestrians without changing clothes is equally important as re-identification of pedestrians with changing clothes. When maximizing the dot product between the positive class agents of different clothes, the accuracy of re-identification of pedestrians with changing clothes can be improved, but the accuracy of re-identification of pedestrians without changing clothes may be reduced. In order to improve the model's re-identification ability of pedestrians with changing clothes without seriously reducing the accuracy of re-identification of pedestrians without changing clothes, q(c) is defined as follows;

[0130]

[0131] Among them, L CA represents the multi-positive clothing classification loss, is with A collection of clothing categories with the same identity, is with A collection of clothing categories with different identities; K is The number of categories in , q(c) is the weight of the cross entropy loss of the c-th clothing class; the positive class with the same clothing, The weights of the positive classes with different clothes are different from the positive classes with the same clothes: Positive categories with different clothing: and 0<δ<1 is a hyperparameter, which can be used to adjust the penalty intensity for each category by adjusting δ;

[0132] In the embodiment of the present invention, δ is set to 0.1.

[0133] The overall loss function is as follows:

[0134] L=L ID +L C +L CA +λL Tri

[0135] Among them, L represents the overall loss function, λ represents the identity triple loss L Tri The weight parameter of .

[0136] In the embodiment of the present invention, λ is set to 0.2.

[0137] Figure 4This is a feature comparison diagram of the loss function used in the present invention and other loss functions. Figure (a) shows two pictures of the same person changing clothes. Figure (b) shows a learning feature diagram using only identity ternary loss and identity cross entropy loss; Figure (c) shows a learning feature diagram using identity ternary loss, identity cross entropy loss, clothing cross entropy loss, and multi-positive clothing classification loss. All training settings for Figures (b) and (c) are the same, and both are generated using the Grad-CAM algorithm. Figure (b) only highlights the face and shoes as features unrelated to clothes, while Figure (c) highlights more features unrelated to clothes, such as face, shoes, body shape, hairstyle, etc. Because the same person in the training set mostly wears the same shoes in different clothes, the shoes in Figures (b) and (c) are also highlighted. This proves that the loss function proposed in the present invention can mine more features unrelated to clothes.

[0138] Step 5: Input the pedestrian image to be queried into the trained pedestrian re-identification model to obtain the pedestrian feature vector, and then calculate the similarity with the pedestrian feature vector in the pedestrian database to obtain the pedestrian label corresponding to the pedestrian feature vector with the highest similarity greater than the set threshold, and complete the pedestrian re-identification.

[0139] Calculate the cosine similarity between the pedestrian feature vector of the pedestrian image to be queried and the pedestrian feature vector in the pedestrian database:

[0140]

[0141] Among them, f q represents the pedestrian feature vector of the pedestrian image to be queried, f g It represents the pedestrian feature vector in the pedestrian database, n is the vector dimension size, and the cosine similarity ranges from -1 to 1. The closer the value is to 1, the higher the similarity.

[0142] The cosine similarity between each query image and all images in the database is calculated and sorted from highest to lowest similarity. The top-k hit rate (Top-k) and mean average precision (mAP) are calculated. Both top-k and mAP test the accuracy of the model's recognition results in the dataset.

[0143] Comparative experiments were conducted on the LTCC and PRCC datasets, respectively, to demonstrate the effectiveness of the proposed method.

[0144] Table 1 Comparison of indicators of the present invention and other methods on LTCC and PRCC

[0145]

[0146] Table 1 quantitatively lists the Top-1 and mAP performance of various methods on the LTCC and PRCC datasets, which are crucial evaluation criteria in person re-ID. The LTCC dataset contains relatively few images but includes images with varying lighting conditions and low resolution. The "normal" in the table represents a mixed subset of images with both changed and unchanged clothing. The proposed model improves the Top-1 performance by 0.9% / 1.6% and the mAP by 5.5% / 2.1% on the "normal" and "changed clothing" subsets, respectively, on the LTCC dataset. PRCC, a rigorous dataset for clothing-changing person re-ID, has served as the benchmark for most models tested since 2019. With its large dataset size, the proposed model improves the Top-1 performance by 1.1% and 0.8% on the "same clothing" and "changed clothing" subsets, respectively, demonstrating that the proposed model achieves improvements without sacrificing re-ID accuracy for people with the same clothing. Furthermore, it improves the re-ID accuracy for people with changed clothing, demonstrating the effectiveness of the proposed model in clothing-changing person re-ID.

[0147] An embodiment of the present invention further provides a system based on the above-mentioned method for re-identifying pedestrians who have changed clothes based on multi-positive classification learning, comprising:

[0148] Training sample set construction unit: used to pre-process the clothing-changing pedestrian re-identification data to obtain the training sample set;

[0149] Person re-identification model construction unit: The person re-identification model includes a backbone network, a processing head, and a classifier connected in sequence. The backbone network is an improved ViT network. The processing head includes a global pooling layer and a batch normalization processing layer connected in sequence. The classifier includes an identity classifier and a clothing classifier.

[0150] Batch image construction unit: Generates batch images with multiple identity labels and multiple clothing labels based on the identity labels and clothing labels of pedestrians in the constructed training sample set;

[0151] Each batch of images contains at least 4 images of each identity label, and the images of the same identity label contain at least two different clothing labels;

[0152] Training unit: Use batch images to train the constructed pedestrian re-identification model and set an integrated loss function. The integrated loss function includes calculating the identity ternary loss based on the normalized feature vector, calculating the identity cross entropy loss based on the identity classification features, calculating the clothing cross entropy loss based on the clothing classification features, and calculating the multi-positive clothing classification loss;

[0153] Identification unit: The pedestrian image to be queried is input into the trained pedestrian re-identification model to obtain the pedestrian feature vector, and then the similarity is calculated with the pedestrian feature vector in the pedestrian database. The pedestrian label corresponding to the pedestrian feature vector with the highest similarity greater than the set threshold is obtained to complete pedestrian re-identification.

[0154] It should be understood that the implementation process of each module can refer to the description of the aforementioned method. The above division of functional modules is merely a division of logical functions. In actual implementation, other division methods may be used. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not implemented. Furthermore, the above-mentioned integrated units may be implemented in the form of hardware or software functional units.

[0155] The technical solution of the present invention also provides a readable storage medium storing a computer program, which is called by a processor to implement: a step of a method for re-identifying pedestrians who have changed clothes based on the above-mentioned multi-positive classification learning method.

[0156] For the specific implementation process of each step, please refer to the description of the above method.

[0157] The readable storage medium is a computer-readable storage medium, which may be an internal storage unit of the controller described in any of the aforementioned embodiments, such as a hard disk or memory of the controller. The readable storage medium may also be an external storage device of the controller, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the controller. Furthermore, the readable storage medium may also include both an internal storage unit of the controller and an external storage device. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0158] Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0159] It should be emphasized that the examples described in the present invention are illustrative rather than restrictive. Therefore, the present invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solution of the present invention that do not depart from the purpose and scope of the present invention, whether modified or replaced, also fall within the scope of protection of the present invention.

Claims

1. A method for re-identifying people who have changed clothes based on multi-positive classification learning, characterized by: include: Step 1: Preprocess the clothing-changing pedestrian re-identification data to obtain a training sample set; Step 2: Build a pedestrian re-identification model; The pedestrian re-identification model includes a backbone network, a processing head, and a classifier connected in sequence. The backbone network is an improved ViT network. The processing head includes a global pooling layer and a batch normalization processing layer connected in sequence. The classifier includes an identity classifier and a clothing classifier. Step 3: Generate batch images with multiple identity labels and multiple clothing labels based on the identity labels and clothing labels of the pedestrians in the training sample set processed in step 1; Each batch of images contains at least 4 images of each identity label, and the images of the same identity label contain at least two different clothing labels; Step 4: Use the batch images obtained in step 3 to train the constructed pedestrian re-identification model and set an integrated loss function. The integrated loss function includes calculating the identity triple loss based on the normalized feature vector, calculating the identity cross entropy loss based on the identity classification features, calculating the clothing cross entropy loss based on the clothing classification features, and calculating the multi-positive clothing classification loss; Step 5: Input the pedestrian image to be queried into the trained pedestrian re-identification model to obtain the pedestrian feature vector. Then, calculate the similarity with the pedestrian feature vector in the pedestrian database, obtain the pedestrian label corresponding to the pedestrian feature vector with the highest similarity greater than the set threshold, and complete the pedestrian re-identification; The multi-positive clothing classification loss L CA as follows: Among them, L CA represents the multi-positive clothing classification loss, is with f i C A collection of clothing categories with the same identity, is with A collection of clothing categories with different identities; K is The number of categories in , q(c) is the weight of the cross entropy loss of the c-th clothing class; the positive class with the same clothing, The weights of the positive classes with different clothes are different from the positive classes with the same clothes: Positive categories with different clothing: and 0<δ<1 is a hyperparameter, which can be used to adjust the penalty intensity for each category. i c Indicates clothing classification characteristics, Indicates the probability that the clothing label predicted by the image is the i-th category.

2. The method according to claim 1, characterized in that The improved ViT network refers to the use of the ViT-Base network, and the pre-trained model is trained on the ImageNet-21k dataset. The specific working process is as follows: A1: Use a sliding window to split the input image for training into patches with overlapping pixels. The step size is represented by S and the patch size is represented by P. The size of the overlapping area of two adjacent patches is (PS)×P. The input image with a resolution of H×W will be divided into N patches. B Patches: Where [] is rounded down, H, W are the height and width of the image, P, S are the patch size and the step size of the sliding window, and P>S; A2: Each patch is linearly projected into the corresponding primitive through the fully connected layer; A3: After performing 2D linear interpolation on the primitives, a position code is added to each primitive to record the position information of the input image. 2D linear interpolation can be used to process images of any size. A4: The position-encoded primitives are passed through multiple transformation layers to output a global high-dimensional feature matrix.

3. The method according to claim 2, characterized in that The execution process of the multi-layer conversion layer is as follows: After the position-encoded primitives are normalized, they pass through 12 attention heads, and finally the 12-head encodings are spliced together, then pass through a normalization layer, and finally pass through a multi-layer perceptron, and then projected back to output a high-dimensional global feature matrix.

4. The method according to claim 1, wherein The process of training the constructed person re-identification model is as follows: Step 4.1: Input the input image into the ViT-Base network to extract the high-dimensional feature matrix; Step 4.2: Input the high-dimensional matrix features into the global pooling layer and perform a global pooling operation to obtain the feature vector f t , and then pass through the batch normalization processing layer to perform batch normalization operation to obtain the normalized feature vector f i ; Step 4.3: Input the normalized feature vector into a fully connected layer simultaneously, wherein the fully connected layer includes an identity classifier and a clothing classifier, to obtain identity classification features and clothing classification features; Step 4.4: Calculate the identity ternary loss based on the normalized feature vector, calculate the identity cross entropy loss based on the identity classification features, calculate the clothing cross entropy loss and the multi-positive clothing classification loss based on the clothing classification features, and then calculate the weighted sum of the four losses to obtain the overall loss; Step 4.5: Calculate the overall loss gradient backpropagation, with the goal of minimizing the overall loss, iteratively update the weight parameters of the pedestrian re-identification model, use different batches of images with multiple identity labels and multiple clothing labels, and repeat steps 4.1 to 4.5 until the set number of iterations is reached.

5. The method according to claim 4, characterized in that The identity triple loss L Tri as follows: L Tri =[d p -d n +a] + Among them, L Tri represents the identity triple loss, d p and d n is the feature vector f extracted from the positive sample pair and negative sample pair of identity labels in the training sample set t The Euclidean distance between them, α is the triple loss L Tri The boundary of [z] + Equal to max(z,0), that is, the maximum value between 0 and z; The identity cross entropy loss L ID as follows: Among them, L ID represents the identity cross entropy loss, N is the batch size during training, N ID The total number of identity labels in the training sample set, f i ID represents identity classification features, y ID represents the true identity label of the image, Represents the probability that the identity label of the image prediction is the i-th category, τ∈R + is the temperature parameter, R + represents a positive real number, In order to prevent the pedestrian re-identification model from overfitting the trained identity labels, ε is a label smoothing constant used to penalize the model's prediction of the identity, and e is a natural base; The overall loss function is as follows: L=L ID +L C +L CA +λL Tri Among them, L represents the overall loss function, λ represents the identity triple loss L Tri The weight parameter of .

6. The method according to claim 4, characterized in that The Adam optimization algorithm is used to optimize the training process of the pedestrian re-identification model. The maximum number of training rounds on the training sample set is 60 rounds, and the multi-positive clothing classification loss L CA For training after 30 rounds, the initial learning rate is 3.5×10 -4 , and then the learning rate is divided by 10 after every 20 rounds until the pedestrian re-identification model converges or reaches the maximum number of iterations, and the final pedestrian re-identification model is obtained.

7. The method according to claim 1, characterized in that Calculate the cosine similarity between the pedestrian feature vector of the pedestrian image to be queried and the pedestrian feature vector in the pedestrian database: Among them, f q represents the pedestrian feature vector of the pedestrian image to be queried, f g It represents the pedestrian feature vector in the pedestrian database, n is the vector dimension size, and the cosine similarity ranges from -1 to 1. The closer the value is to 1, the higher the similarity.

8. The method according to any one of claims 1 to 7, characterized in that The preprocessing includes image scaling, random horizontal flipping, random cropping, random erasing, and automatic enhancement.

9. A system based on the method for re-identifying pedestrians after changing clothes based on multi-positive class classification learning according to any one of claims 1 to 8, characterized in that: include: Training sample set construction unit: used to perform data enhancement processing on the clothing-changing pedestrian re-identification data to obtain the training sample set; Person re-identification model construction unit: The person re-identification model includes a backbone network, a processing head, and a classifier connected in sequence. The backbone network is an improved ViT network. The processing head includes a global pooling layer and a batch normalization processing layer connected in sequence. The classifier includes an identity classifier and a clothing classifier. Batch image construction unit: Generates batch images with multiple identity labels and multiple clothing labels based on the identity labels and clothing labels of pedestrians in the constructed training sample set; Each batch of images contains at least 4 images of each identity label, and the images of the same identity label contain at least two different clothing labels; Training unit: Use batch images to train the constructed pedestrian re-identification model and set an integrated loss function. The integrated loss function includes calculating the identity ternary loss based on the normalized feature vector, calculating the identity cross entropy loss based on the identity classification features, calculating the clothing cross entropy loss based on the clothing classification features, and calculating the multi-positive clothing classification loss; Identification unit: The pedestrian image to be queried is input into the trained pedestrian re-identification model to obtain the pedestrian feature vector, and then the similarity is calculated with the pedestrian feature vector in the pedestrian database. The pedestrian label corresponding to the pedestrian feature vector with the highest similarity greater than the set threshold is obtained to complete pedestrian re-identification.

10. A readable storage medium, characterized in that: A computer program is stored, and the computer program is called by a processor to implement the steps of the method according to any one of claims 1 to 8.

Images