Person re-identification methods, systems, devices, and media based on decoupled sample set learning

Abstract

This invention discloses a pedestrian re-identification method, system, device, and medium based on decoupled sample set learning. The method includes the following steps: acquiring a given query image and a given set of pedestrian images; performing feature extraction using a pre-trained feature extractor to obtain feature vectors of the query image and a set of feature vectors of the pedestrian image set; obtaining the Euclidean distance between each feature in the feature vector set of the query image and the feature vector set of the pedestrian image set, and obtaining a sorted list of the pedestrian image set with respect to the query image; and obtaining a pedestrian re-identification result based on the sorted list. This invention enhances the discriminative power of features by mining samples that simultaneously possess discriminative information and high-confidence pseudo-labels and incorporating these samples into the feature learning process; and by using a pre-trained feature extractor for feature extraction, accurate pedestrian re-identification results can be obtained based on discriminative features.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and pattern recognition technology, and relates to the field of pedestrian re-identification, and particularly to a pedestrian re-identification method, system, device and medium based on decoupled sample group learning. Background Technology

[0002] Given a pedestrian image in a real-world scene, the goal of pedestrian re-identification is to find pedestrian images with the same identity identifier as the given pedestrian image from a large set of images. This type of task has many meaningful practical applications in intelligent security systems and intelligent video surveillance systems, such as finding a given person among a large number of pedestrian images taken by different cameras.

[0003] Currently, in existing traditional pedestrian re-identification methods, the cluttered backgrounds of images in the image set and the significant variations in appearance due to different lighting conditions and viewing angles make it very difficult to find images with a given pedestrian identity. Furthermore, the intensive manual annotation process required to train a pedestrian re-identification model is very expensive.

[0004] To reduce the reliance of model training on data labels, unsupervised training is generally employed. Existing unsupervised training methods typically use iterative clustering to estimate pseudo-labels on unlabeled training data, and then use the estimated pseudo-labels to guide the feature learning process. However, in the absence of true labels, these methods fail to consider samples that simultaneously possess discriminative information and high-confidence pseudo-labels, thus affecting the learning of discriminative features and ultimately failing to obtain accurate pedestrian re-identification results. Summary of the Invention

[0005] The purpose of this invention is to provide a pedestrian re-identification method, system, device, and medium based on decoupled sample group learning, to solve one or more of the aforementioned technical problems. The technical solution provided by this invention enhances the discriminative power of features by mining samples that simultaneously possess discriminative information and high-confidence pseudo-labels and incorporating these samples into the feature learning process; a trained feature extractor is used for feature extraction, and accurate pedestrian re-identification results can be obtained based on the extracted discriminative features.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] The first aspect of this invention provides a person re-identification method based on decoupled sample group learning, comprising the following steps:

[0008] Retrieve the given query image and the given set of pedestrian images;

[0009] Based on the given query image and the given set of pedestrian images, a pre-trained feature extractor is used to extract features to obtain the feature vector of the query image and the feature vector set of the pedestrian image set.

[0010] Obtain the Euclidean distance between the feature vector of the query image and each feature in the feature vector set of the pedestrian image set, and obtain the sorted list of the pedestrian image set with respect to the query image;

[0011] Based on the sorted list, the pedestrian re-identification result is obtained;

[0012] The training steps of the pre-trained feature extractor include:

[0013] Each training image sample in the unlabeled pedestrian training image sample set is input into a pre-built feature extractor to obtain the features of each training image sample and store them in the feature library.

[0014] In each iteration of training, pedestrian features are generated for the current pedestrian based on the current feature library. Auxiliary vectors are constructed for each training image sample using the pedestrian features. Based on the auxiliary vectors, decoupled positive sample groups and decoupled negative sample groups are constructed for each training image sample. The decoupled positive sample groups and decoupled negative sample groups are introduced into an alternative decoupling learning loss function and a decoupling second-order similarity regularization to guide the feature learning process of the feature extractor, and the pre-trained feature extractor is obtained. After each iteration of the training process, the features of each training image sample are updated.

[0015] A further improvement to the method of the present invention is that the steps of generating pedestrian features for the current pedestrian based on the current feature library, constructing an auxiliary vector for each training image sample using the pedestrian features, and constructing a decoupled positive sample group and a decoupled negative sample group for each training image sample based on the auxiliary vector include:

[0016] Based on the input image x i The image number is used to extract the corresponding feature in the feature library. Where i represents the pedestrian sample number in the database, and M represents the feature library of the samples. Represents the characteristics of pedestrian i stored in M;

[0017] For the current image x i Constructing auxiliary vector sets in, Indicates pedestrian y l The prototype vector of the corresponding class has the same pseudo-label y. l The average of the feature vectors corresponding to the samples; y i Indicates the estimated x i pseudo-tags;

[0018] Based on auxiliary vector A i With image x i The relationship between the training image samples is calculated by removing image x from the training image sample set. i The remaining image samples and A i The histogram distribution of similarity probability between them is expressed as follows:

[0019]

[0020]

[0021] In the formula, Representing the image x i Features stored in feature library M;

[0022] Based on the obtained similarity probability histogram distribution P(x) i A i ), for image x i Constructing a decoupled positive sample group With decoupling negative sample groups The expression is,

[0023]

[0024]

[0025]

[0026]

[0027] In the formula, argTopk(P(x i A i )) k Represents the distribution P(x) i A i The index of the k-th largest value in ), argTopk(P(x j A j )) k ) represents the distribution P(x) j A j The index of the k-th largest value in the sample group; K represents the order of the decoupled sample group; x represents i k-th order decoupled positive sample group, x represents i The k-th order decoupled negative sample group, y i With y j Corresponding to the image x i With x j pseudo-tags, A i For x iThe corresponding auxiliary vector set.

[0028] A further improvement to the method of the present invention is that the steps of introducing an alternative decoupling learning loss function and a decoupling second-order similarity regularization-guided feature extraction process to train and obtain the pre-trained feature extractor include:

[0029] The generated decoupled positive sample group Generate for image x i Positive decoupling positive sample features The expression is,

[0030]

[0031] In the formula, ω represents the strength parameter; Represents a set The base is The number of elements in the middle;

[0032] The generated decoupled negative sample group Generate for image x i Negative decoupling of positive sample features The expression is,

[0033]

[0034] In the formula, Denotes the k-th order unentangled negative sample group. Samples with pseudo-label 'a';

[0035] According to the inequality of piano sounds, the following inequality relationship holds.

[0036] τ represents the mode parameter;

[0037] Based on the generated image x i By decoupling positive sample features and inequality relationships, an alternative decoupling learning loss function is obtained, expressed as follows:

[0038]

[0039] In the formula, c n This represents the prototype vector of the class corresponding to pedestrian n, where n ≠ y. i B represents the current training batch, and |B| represents the size of the current training batch; v i Indicates sample x i Features generated by feature extractor f, v i =f(x) i );

[0040] The generated decoupled positive sample group With decoupled negative samples To obtain the disturbed pedestrian features, the expression is:

[0041]

[0042]

[0043] In the formula, λ represents the interference parameter; This represents the k-th order feature after interference from pedestrian a;

[0044] Generate a feature for image x from the perturbed pedestrian features i Intimacy distribution The expression is,

[0045]

[0046] In the formula, N c Indicates the current number of pedestrians;

[0047] Generate a feature for image x from the generated pedestrian features j Based on pedestrian characteristics Intimacy distribution Q j ;where x j Indicates with x i Images featuring the same pedestrians;

[0048] By generated With Q j The expression for calculating the decoupling KL divergence regularity is as follows:

[0049]

[0050] By step and The total decoupling loss function is calculated as follows:

[0051]

[0052] Using L dise The feature extractor f is optimized, and its parameters are updated using the Adam algorithm; where the feature library M is updated by the current network, denoted as M[i]←v i .

[0053] A further improvement of the method of the present invention is that the feature extractor is a deep residual network or a visual translator network.

[0054] A second aspect of the present invention provides a pedestrian re-identification system based on decoupled sample group learning, comprising:

[0055] The data acquisition module is used to acquire a given query image and a given set of pedestrian images;

[0056] The feature extraction module is used to extract features based on the given query image and the given set of pedestrian images using a pre-trained feature extractor, so as to obtain the feature vector of the query image and the feature vector set of the set of pedestrian images.

[0057] The sorting list acquisition module is used to obtain the Euclidean distance between the feature vector of the query image and each feature in the feature vector set of the pedestrian image set, and to obtain the sorting list of the pedestrian image set with respect to the query image;

[0058] The result acquisition module is used to obtain the pedestrian re-identification result based on the sorted list;

[0059] The training steps of the pre-trained feature extractor include:

[0060] Each training image sample in the unlabeled pedestrian training image sample set is input into a pre-built feature extractor to obtain the features of each training image sample and store them in the feature library.

[0061] In each iteration of training, pedestrian features are generated for the current pedestrian based on the current feature library. Auxiliary vectors are constructed for each training image sample using the pedestrian features. Based on the auxiliary vectors, decoupled positive sample groups and decoupled negative sample groups are constructed for each training image sample. The decoupled positive sample groups and decoupled negative sample groups are introduced into an alternative decoupling learning loss function and a decoupling second-order similarity regularization to guide the feature learning process of the feature extractor, and the pre-trained feature extractor is obtained. After each iteration of the training process, the features of each training image sample are updated.

[0062] A further improvement to the system of the present invention is that the steps of generating pedestrian features for the current pedestrian based on the current feature library, constructing an auxiliary vector for each training image sample using the pedestrian features, and constructing a decoupled positive sample group and a decoupled negative sample group for each training image sample based on the auxiliary vector include:

[0063] Based on the input image x i The image number is used to extract the corresponding feature in the feature library. Where i represents the pedestrian sample number in the database, and M represents the feature library of the samples. Represents the characteristics of pedestrian i stored in M;

[0064] For the current image x i Constructing auxiliary vector sets in, Indicates pedestrian y lThe prototype vector of the corresponding class has the same pseudo-label y. l The average of the feature vectors corresponding to the samples; y i Indicates the estimated x i pseudo-tags;

[0065] Based on auxiliary vector A i With image x i The relationship between the training image samples is calculated by removing image x from the training image sample set. i The remaining image samples and A i The histogram distribution of similarity probability between them is expressed as follows:

[0066]

[0067]

[0068] In the formula, Representing the image x i Features stored in feature library M;

[0069] Based on the obtained similarity probability histogram distribution P(x) i A i ), for image x i Constructing a decoupled positive sample group With decoupling negative sample groups The expression is,

[0070]

[0071]

[0072]

[0073]

[0074] In the formula, argTopk(P(x i A i )) k Represents the distribution P(x) i A i The index of the k-th largest value in ), argTopk(P(x j A j )) k ) represents the distribution P(x) j A j The index of the k-th largest value in the sample group; K represents the order of the decoupled sample group; x represents i k-th order decoupled positive sample group, x represents i The k-th order decoupled negative sample group, y iWith y j Corresponding to the image x i With x j pseudo-tags, A i For x i The corresponding auxiliary vector set.

[0075] A further improvement to the system of the present invention is that the steps of introducing the decoupled positive sample group and the decoupled negative sample group into an alternative decoupling learning loss function and the decoupling second-order similarity regularization to guide the feature learning process of the feature extractor, and training to obtain the pre-trained feature extractor, include:

[0076] The generated decoupled positive sample group Generate for image x i Positive decoupling positive sample features The expression is,

[0077]

[0078] In the formula, ω represents the strength parameter; Represents a set The base is The number of elements in the middle;

[0079] The generated decoupled negative sample group Generate for image x i Negative decoupling of positive sample features The expression is,

[0080]

[0081] In the formula, Denotes the k-th order unentangled negative sample group. Samples with pseudo-label 'a';

[0082] According to the inequality of piano sounds, the following inequality relationship holds.

[0083] τ represents the mode parameter;

[0084] Based on the generated image x i By decoupling positive sample features and inequality relationships, an alternative decoupling learning loss function is obtained, expressed as follows:

[0085]

[0086] In the formula, c n This represents the prototype vector of the class corresponding to pedestrian n, where n ≠ y. i B represents the current training batch, and |B| represents the size of the current training batch; v i Indicates sample x iFeatures generated by feature extractor f, v i =f(x) i );

[0087] The generated decoupled positive sample group With decoupled negative samples To obtain the disturbed pedestrian features, the expression is:

[0088]

[0089]

[0090] In the formula, λ represents the interference parameter; This represents the k-th order feature after interference from pedestrian a;

[0091] Generate a feature for image x from the perturbed pedestrian features i Intimacy distribution The expression is,

[0092]

[0093] In the formula, N c Indicates the current number of pedestrians;

[0094] Generate a feature for image x from the generated pedestrian features j Based on pedestrian characteristics Intimacy distribution Q j ;where x j Indicates with x i Images featuring the same pedestrians;

[0095] By generated With Q j The expression for calculating the decoupling KL divergence regularity is as follows:

[0096]

[0097] By step and The total decoupling loss function is calculated as follows:

[0098]

[0099] Using L dise The feature extractor f is optimized, and its parameters are updated using the Adam algorithm; where the feature library M is updated by the current network, denoted as M[i]←v i .

[0100] A further improvement of the system of the present invention is that the feature extractor is a deep residual network or a visual translator network.

[0101] A third aspect of the present invention provides an electronic device comprising:

[0102] At least one processor; and,

[0103] A memory communicatively connected to the at least one processor; wherein,

[0104] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the pedestrian re-identification method according to any one of the first aspects of the present invention.

[0105] The fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the pedestrian re-identification method according to any one of the first aspects of the present invention.

[0106] Compared with the prior art, the present invention has the following beneficial effects:

[0107] The pedestrian re-identification method based on decoupled sample set learning provided in this invention differs from traditional deep learning methods, which require a large amount of dense manual annotation information. This invention can train a robust model without any manual annotation information. Specifically, addressing the problem that current unsupervised methods cannot simultaneously obtain high-confidence pseudo-labels and samples with discriminative information, this invention employs unsupervised training based on decoupled samples. This allows for the allocation of specific decoupled sample sets with sufficient discriminative information to each training sample under unlabeled supervision, while preserving the accuracy of pseudo-labels. By introducing these samples during training, the model can better distinguish between pedestrian-irrelevant and pedestrian-related factors among pedestrians. In this invention, the term "decoupled positive sample set" is used to represent samples that have the same pseudo-label as the query sample but possess dissimilar label-irrelevant information; the term "decoupled negative sample set" is used to represent samples that have different pseudo-labels as the query sample but possess similar label-related information.

[0108] In this invention, a specific decoupled sample group is assigned to each given unlabeled training image, and then these decoupled samples are added to the training process of the feature extractor f; thereby, a sample with effective training information is constructed for each sample, while retaining high-confidence pseudo-labels; at the same time, it also provides a new approach to solving the unsupervised pedestrian re-identification task. Attached Figure Description

[0109] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art are briefly introduced below; obviously, the drawings described below are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0110] Figure 1 This is a flowchart illustrating a pedestrian re-identification method based on decoupled sample group learning provided in an embodiment of the present invention;

[0111] Figure 2 This is a schematic diagram of the process of training and acquiring the feature extractor in an embodiment of the present invention;

[0112] Figure 3 This is a schematic diagram of constructing a decoupled sample group for the current input data in an embodiment of the present invention;

[0113] Figure 4 This is a flowchart illustrating the process of introducing the constructed decoupled sample group into the training process of the feature extractor in an embodiment of the present invention;

[0114] Figure 5 This is a schematic diagram of a pedestrian re-identification system based on decoupled sample group learning provided in an embodiment of the present invention. Detailed Implementation

[0115] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0116] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0117] The present invention will now be described in further detail with reference to the accompanying drawings:

[0118] Please see Figure 1 The present invention provides a person re-identification method based on decoupled sample group learning, comprising the following steps:

[0119] Step 1: Obtain the given query image and the given set of pedestrian images;

[0120] Step 2: Based on the given query image and the given set of pedestrian images, use a pre-trained feature extractor to extract features and obtain the feature vector of the query image and the feature vector set of the pedestrian image set.

[0121] Step 3: Obtain the Euclidean distance between the feature vector of the query image and each feature in the feature vector set of the pedestrian image set, and obtain the sorted list of the pedestrian image set with respect to the query image;

[0122] Step 4: Based on the sorted list, obtain the pedestrian re-identification result;

[0123] The training steps of the pre-trained feature extractor include:

[0124] Each training image sample in the unlabeled pedestrian training image sample set is input into a pre-built feature extractor to obtain the features of each training image sample and store them in the feature library.

[0125] In each iteration of training, pedestrian features are generated for the current pedestrian based on the current feature library. Auxiliary vectors are constructed for each training image sample using the pedestrian features. Based on the auxiliary vectors, decoupled positive sample groups and decoupled negative sample groups are constructed for each training image sample. The decoupled positive sample groups and decoupled negative sample groups are introduced into an alternative decoupling learning loss function and a decoupling second-order similarity regularization to guide the feature learning process of the feature extractor, and the pre-trained feature extractor is obtained. After each iteration of the training process, the features of each training image sample are updated.

[0126] In the technical solution provided by this invention, samples possessing both discriminative information and high-confidence pseudo-labels are mined and introduced into the feature learning process to enhance the discriminative power of the features. A trained feature extractor is then used for feature extraction, and accurate pedestrian re-identification results can be obtained based on the extracted discriminative features. Specifically, during the training process of the feature extractor, this invention constructs a pedestrian re-identification method based on decoupled sample group learning. A specific decoupled positive sample group and decoupled negative sample group are assigned to each sample using auxiliary vectors to approximate difficult samples. Furthermore, because the assigned samples have a one-to-many relationship, this sample group has higher accuracy than a single difficult sample. For example, the training strategy of this invention ensures that the final feature extractor can effectively distinguish pedestrian-related factors, such as pedestrian body features, and pedestrian-irrelevant factors, such as clothing, posture, and camera angle.

[0127] This invention provides a pedestrian re-identification method based on decoupled sample set learning. The goal is to find images with the same identity identifier as the query image from a given, unordered set of large pedestrian images and a given query image. Specific steps include:

[0128] First, a feature extractor (preferred for ease of exploration and illustrative purposes, the feature extractor used in this invention is a deep residual network (ResNet50), and the term "feature extractor" will be used to refer to the deep residual network) is used to extract feature vectors from the query image and to calculate a set of feature vectors for a given large set of pedestrian images. More specifically, the query image is input into the feature extractor to obtain d-dimensional query features, and then each image in the given large set of pedestrian images is input into the same feature extractor to calculate a set of d-dimensional features;

[0129] Subsequently, based on the Euclidean distance between the query features and each feature in the feature set, a sorted list of the image feature set with respect to the query image is obtained, and it can be assumed that the top k images in the sorted list have the same identity as the given query image.

[0130] In this embodiment of the invention, to achieve the above objectives, the feature extractor needs to be optimized based on the training algorithm. It is worth noting that, unlike mainstream deep learning training methods that require a large amount of sufficiently labeled information, this embodiment of the invention can train a good model without any manually labeled data, thus saving significant labeling costs.

[0131] Please see Figure 2 The training process of the feature extractor in this embodiment of the invention includes the following steps:

[0132] Step 1: Given a set of unlabeled pedestrian training images, we only know the total number of images in the set, but not the number of pedestrian icons contained in the training images.

[0133] Step 2: Initialize an empty feature library, which will be used to store the features of all pedestrian training images.

[0134] Step 3: Input all the unlabeled pedestrian images from Step 1 into the pre-built feature extractor to obtain the features of the unlabeled pedestrian images, and store these features in the feature library initialized in Step 2.

[0135] Step 4: Based on the feature library generated in Step 3, construct a specific decoupled sample group for each sample.

[0136] Step 5: Using the designed loss function, the features of the current input sample from Step 4 and its assigned decoupled sample group are introduced into the training process of the feature extractor, while updating the feature library in Step 3.

[0137] Return to step 3 and repeat steps 3 through 5 until the loss value of the feature extractor in step 5 converges.

[0138] In step 3 of this embodiment of the invention, the specific steps of storing the features in the feature library include:

[0139] Step 3.1: Input the image generated in Step 1 into the feature extractor to generate the corresponding features;

[0140] Step 3.2: Based on the features extracted in Step 3.1, a pseudo-label is assigned to each sample using a general clustering algorithm;

[0141] Step 3.3: Store the features generated in step 3.1 in the feature library according to the corresponding image index.

[0142] In step 4 of this embodiment of the invention, the specific steps for constructing a specific decoupled sample group for each sample include:

[0143] Step 4.1, for the input image, first, based on the input image x... i Extract the corresponding feature from the feature library based on the image number.

[0144] Step 4.2, for the current sample x i Constructing auxiliary vector sets in, Indicates pedestrian y l The prototype vector of the corresponding class, that is, the one with the same pseudo-label y. l The average of the feature vectors corresponding to the samples; y i This represents x estimated by step 3.2.i pseudo-tags;

[0145] Step 4.3, based on auxiliary vector A i With sample x i The relationship between x and x in the training set i All sample calculations and A i The histogram distribution of similarity probability between them is expressed as:

[0146]

[0147]

[0148] In the formula, Indicates sample x i Features stored in feature library M.

[0149] Step 4.4, based on the probability distribution P(x) generated in step 4.3 i A i ), which is the current sample x i Construct a specific decoupled positive sample group With decoupled negative samples This process can be formally represented as:

[0150]

[0151]

[0152]

[0153]

[0154] Where argTopk(P) k The index of the k-th largest value in distribution P; K represents the order of the decoupled sample group. x represents i k-th order decoupled positive sample group, x represents i The k-th order decoupled negative sample group.

[0155] In step 5 of this embodiment of the invention, the specific steps for training the feature extractor and updating the feature library include:

[0156] Step 5.1, the decoupled positive sample group generated in step 4.4 Generate for x i The positive decoupling positive sample features are represented as:

[0157]

[0158] Where ω represents the intensity parameter, Represents a set The cardinality, i.e. The number of elements in the middle.

[0159] Step 5.2, x-specific data generated from step 5.1 i The positive decoupling positive sample features, and the decoupling negative samples generated in step 4.4. Calculate alternative decoupling learning loss functions:

[0160]

[0161] Among them, c n Represents pedestrian n (n≠y) i The prototype vector of the class corresponding to ). In the k-th order unentangled sample group The samples belonging to person a (pseudo); B represents the current training batch, and |B| represents the size of the current training batch, τ represents the mode parameter, and v i Indicates sample x i The features generated by feature extractor f, i.e. v i =f(x) i ).

[0162] Step 5.3, the decoupled positive sample group generated in step 4.4 With decoupled negative samples Interference with pedestrian characteristics can be represented as follows:

[0163]

[0164]

[0165] Where λ represents the interference parameter; This represents the k-th order feature after interference from pedestrian a.

[0166] Step 5.4: Generate a sample x based on the perturbated pedestrian features generated in Step 5.3. i Intimacy distribution This generation process can be formalized as follows:

[0167]

[0168] Where, N c This indicates the current number of pedestrians.

[0169] Step 5.5: Generate a sample x based on the generated pedestrian features. j Based on pedestrian characteristics Intimacy distribution Q j , where x j Indicates with xi Images with the same pedestrians, Q j The generation process can be formally described as follows:

[0170]

[0171] Step 5.6, generated from step 5.4 With Q generated in step 5.5 j Calculate the decoupling KL divergence regularity:

[0172]

[0173] Step 5.7, generated from step 5.2 With the generated in step 5.6 The total decoupling loss function is calculated as follows:

[0174]

[0175] And using L dise The feature extractor f is optimized, and the parameters of the graph feature extractor are updated using the Adam algorithm.

[0176] Step 5.8, update the feature library M from the current network, represented as: M[i]←v i .

[0177] See Figures 2 to 4 The present invention provides a pedestrian re-identification method based on sample pair relation distillation, comprising the following steps:

[0178] Step 1: Acquire unlabeled pedestrian images;

[0179] Step 2: Initialize an empty feature library;

[0180] Step 3: Build a feature extractor, given a training image sequence X = {x ...} containing a single target. i} i＝1,...,N , where x i Let i represent the i-th training image. Use a feature extractor to extract features from the pedestrian image and store the features in the feature library.

[0181] For example, step 3 specifically includes:

[0182] Step 3.1: The image generated in Step 1 is fed into the feature extractor to generate corresponding features; for example, in this embodiment of the invention, ResNet50 is used to extract the feature vector {v} of the image. i} i＝1,...,N This generates 2048-dimensional features.

[0183] Step 3.2: Based on the features extracted in Step 3.1, a pseudo-label is assigned to each sample using a general clustering algorithm; for example, in this embodiment of the invention, ResNet50 is used to extract the feature vector {y} of the image. i} i＝1,...,N This generates 2048-dimensional features.

[0184] Step 3.2: Store the features generated in Step 3.1 into the feature library according to the corresponding image index. For example, in this embodiment of the invention, the formula used when generating the initial feature library is M[i]←v i .

[0185] For example, step 4 specifically includes the following steps:

[0186] Step 4.1, for the input image, first, based on the input image x... i Extract the corresponding feature from the feature library based on the image number.

[0187] Step 4.2, for the current sample x i Constructing auxiliary vector sets in Indicates pedestrian y l The prototype vector of the corresponding class, that is, the one with the same pseudo-label y. l The average of the feature vectors corresponding to the samples. i This represents x estimated by step 3.2. i Pseudo-tags.

[0188] Step 4.3, based on auxiliary vector A i With sample x i The relationship between x and x in the training set i All sample calculations and A i The histogram distribution of similarity probability between them is expressed as:

[0189]

[0190]

[0191] In the formula, Indicates sample x i Features stored in feature library M.

[0192] Step 4.4, based on the probability distribution P(x) generated in step 4.3 i A i ), which is the current sample x i Construct a specific decoupled positive sample group With decoupled negative samples This process can be formally represented as:

[0193]

[0194]

[0195]

[0196]

[0197] Where argTopk(P) k The index of the k-th largest value in distribution P; K represents the order of the decoupled sample group. x represents i k-th order decoupled positive sample group, x represents i The k-th order decoupled negative sample group.

[0198] For example, step 5 specifically includes the following steps:

[0199] Step 5.1, the decoupled positive sample group generated in step 4.4 Generate for x i The positive decoupling positive sample features are represented as:

[0200]

[0201] Where ω represents the intensity parameter, Represents a set The cardinality, i.e. The number of elements in the middle.

[0202] Step 5.2, x-specific data generated from step 5.1 i The positive decoupling positive sample features, and the decoupling negative samples generated in step 4.4. Calculate alternative decoupling learning loss functions:

[0203]

[0204] Among them, c n Represents pedestrian n (n≠y) i The prototype vector of the class corresponding to ) In the k-th order unentangled sample group The samples belonging to person a (pseudo) are denoted by B, which represents the current training batch, and |B| represents the size of the current training batch. τ represents the mode parameter, and v... i Indicates sample x i The features generated by feature extractor f, i.e. v i =f(x) i ).

[0205] Step 5.3, the decoupled positive sample group generated in step 4.4 With decoupled negative samples Interference with pedestrian characteristics can be represented as follows:

[0206]

[0207]

[0208] Where λ represents the interference parameter, This represents the k-th order feature after interference from pedestrian a.

[0209] Step 5.4: Generate a sample x based on the perturbated pedestrian features generated in Step 5.3. i Intimacy distribution This generation process can be formalized as follows:

[0210]

[0211] Where, N c This indicates the current number of pedestrians.

[0212] Step 5.5: Generate a sample x based on the generated pedestrian features. j Based on pedestrian characteristics Intimacy distribution Q j , where x j Indicates with x i Images featuring the same pedestrians. Q j The generation process can be formally described as follows:

[0213]

[0214] Step 5.6, generated from step 5.4 With Q generated in step 5.5 j Calculate the decoupling KL divergence regularity:

[0215]

[0216] Step 5.7, generated from step 5.2 With the generated in step 5.6 The total decoupling loss function is calculated as follows:

[0217]

[0218] And using L dise The feature extractor f is optimized, and the parameters of the graph feature extractor are updated using the Adam algorithm.

[0219] Step 5.8: Update the feature library M from the current network, denoted as M[i]←vi .

[0220] In summary, this invention is applicable to unsupervised training of person re-identification frameworks, finding images with the same identity as a given image within a large image set. The person re-identification method based on sample pair relation distillation of this invention first utilizes an initialized feature library and predicted pseudo-labels to construct auxiliary vector groups for each sample and assign decoupled sample groups to each training sample. During the optimization process of the feature extractor, these sample pairs with pseudo-labels are used to optimize the person re-identification model, bringing sample pairs with the same identity closer together and distancing sample pairs with different identities. This invention can generate high-quality samples for the person re-identification model in the absence of training labels, ultimately resulting in a significant improvement in re-identification performance.

[0221] Tables 1a, 1b, and 1c present the quantitative experimental results of this invention. Table 1a shows the experimental results of this invention on the Market-1501 dataset, Table 1b shows the experimental results of this invention on the DukeMTMC-reID dataset, and Table 1c shows the experimental results of this invention on the MSMT17 dataset.

[0222] Rank-1, Rank-5, Rank-10, and mAP are commonly used metrics for evaluating the quality of generated images; higher values ​​indicate better algorithm performance. Table 1a shows that on the Market-1501 dataset, the metrics generated using this invention achieve the highest Rank-1, Rank-5, Rank-10, and mAP. Table 1b shows that on the DukeMTMC-reID dataset, the metrics generated using this invention achieve the highest Rank-1, Rank-5, Rank-10, and mAP. Table 1c shows that on the MSMT17 dataset, the metrics generated using this invention achieve the highest Rank-1, Rank-5, Rank-10, and mAP. In summary, compared with other methods, our method achieves significant improvements in the unsupervised person re-identification problem.

[0223] Table 1a. Experimental results of this method on the Market-1501 dataset.

[0224]

[0225]

[0226] Table 1b. Experimental results of this method on the DukeMTMC-reID dataset.

[0227]

[0228] Table 1c. Experimental results of this method on the MSMT17 dataset.

[0229]

[0230] In summary, this invention discloses a pedestrian re-identification method based on decoupled sample group learning. The method includes the following steps: Step 1, acquiring unlabeled pedestrian images; Step 2, initializing an empty feature library; Step 3, inputting all unlabeled pedestrian images from Step 1 into a pre-constructed feature extractor to obtain the features of the unlabeled images, generating pseudo-labels using these features, and storing these features in the feature library initialized in Step 2; Step 4, generating pedestrian features using the features in the feature library, generating pedestrian features for the current pedestrian (pseudo) based on the current feature library, constructing auxiliary vectors for each sample using these pedestrian features, and using the auxiliary vectors to guide the construction of specific decoupled positive sample groups and decoupled negative sample groups for that sample; Step 5, introducing these decoupled sample groups into a designed alternative decoupled feature learning loss function and a decoupled second-order similarity regularization-guided feature extractor feature learning process to train and obtain the feature extractor; after each iteration of the training process, updating the features of each training image sample.

[0231] The following are embodiments of the apparatus of the present invention, which can be used to execute embodiments of the method of the present invention. For details not disclosed in the apparatus embodiments, please refer to the embodiments of the method of the present invention.

[0232] Please see Figure 5 The present invention provides a pedestrian re-identification system based on decoupled sample set learning, comprising:

[0233] The data acquisition module is used to acquire a given query image and a given set of pedestrian images;

[0234] The feature extraction module is used to extract features based on the given query image and the given set of pedestrian images using a pre-trained feature extractor, so as to obtain the feature vector of the query image and the feature vector set of the set of pedestrian images.

[0235] The sorting list acquisition module is used to obtain the Euclidean distance between the feature vector of the query image and each feature in the feature vector set of the pedestrian image set, and to obtain the sorting list of the pedestrian image set with respect to the query image;

[0236] The result acquisition module is used to obtain the pedestrian re-identification result based on the sorted list;

[0237] The training steps of the pre-trained feature extractor include:

[0238] Each training image sample in the unlabeled pedestrian training image sample set is input into a pre-built feature extractor to obtain the features of each training image sample and store them in the feature library.

[0239] In each iteration of training, pedestrian features are generated for the current pedestrian based on the current feature library. Auxiliary vectors are constructed for each training image sample using the pedestrian features. Based on the auxiliary vectors, decoupled positive sample groups and decoupled negative sample groups are constructed for each training image sample. The decoupled positive sample groups and decoupled negative sample groups are introduced into an alternative decoupling learning loss function and a decoupling second-order similarity regularization to guide the feature learning process of the feature extractor, and the pre-trained feature extractor is obtained. After each iteration of the training process, the features of each training image sample are updated.

[0240] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of a pedestrian re-identification method based on decoupled sample group learning.

[0241] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the person re-identification method based on decoupled sample group learning in the above embodiments.

[0242] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0243] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0244] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0245] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0246] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A person re-identification method based on decoupled sample set learning, characterized in that, Includes the following steps: Retrieve the given query image and the given set of pedestrian images; Based on the given query image and the given set of pedestrian images, a pre-trained feature extractor is used to extract features to obtain the feature vector of the query image and the feature vector set of the pedestrian image set. Obtain the Euclidean distance between the feature vector of the query image and each feature in the feature vector set of the pedestrian image set, and obtain the sorted list of the pedestrian image set with respect to the query image; Based on the sorted list, the pedestrian re-identification result is obtained; The training steps of the pre-trained feature extractor include: Each training image sample in the unlabeled pedestrian training image sample set is input into a pre-built feature extractor to obtain the features of each training image sample and store them in the feature library. In each iteration of training, pedestrian features are generated for the current pedestrian based on the current feature library. An auxiliary vector is constructed for each training image sample using these pedestrian features. Based on the auxiliary vector, a decoupled positive sample group and a decoupled negative sample group are constructed for each training image sample. The decoupled positive sample group and the decoupled negative sample group are introduced into an alternative decoupling learning loss function and a decoupling second-order similarity regularization to guide the feature learning process of the feature extractor, thus training the pre-trained feature extractor. After each iteration of the training process, the features of each training image sample are updated. The steps of generating pedestrian features for the current pedestrian based on the current feature library, constructing auxiliary vectors for each training image sample using the pedestrian features, and constructing decoupled positive sample groups and decoupled negative sample groups for each training image sample based on the auxiliary vectors include: Based on the input image x i The image number is used to extract the corresponding feature in the feature library. Where i represents the pedestrian sample number in the database, and M represents the feature library of the samples. Represents the characteristics of pedestrian i stored in M; For the current image x i Constructing auxiliary vector sets in, Indicates pedestrian y l The prototype vector of the corresponding class has the same pseudo-label y. l The average of the feature vectors corresponding to the samples; y i Indicates the estimated x i pseudo-tags; Based on auxiliary vector A i With image x i The relationship between the training image samples is calculated by removing image x from the training image sample set. i The remaining image samples and A i The histogram distribution of similarity probability between them is expressed as follows: In the formula, Representing the image x i Features stored in feature library M; Based on the obtained similarity probability histogram distribution P(x) i A i ), for image x i Constructing a decoupled positive sample group With decoupling negative sample groups The expression is, In the formula, argTopk(P(x i A i )) k Represents the distribution P(x) i A i The index of the k-th largest value in ), argTopk(P(x j A j )) k Represents the distribution P(x) j A j The index of the k-th largest value in the sample group; K represents the order of the decoupled sample group; x represents i k-th order decoupled positive sample group, x represents i The k-th order decoupled negative sample group, y i With y j Corresponding to the image x i With x j pseudo-tags, A i For x i The corresponding auxiliary vector set.

2. The person re-identification method based on decoupled sample set learning according to claim 1, characterized in that, The steps of introducing alternative decoupling learning loss functions into the decoupling positive sample group and the decoupling second-order similarity regularization to guide the feature learning process of the feature extractor, and training to obtain the pre-trained feature extractor include: The generated decoupled positive sample group Generate for image x i Positive decoupling positive sample features The expression is, In the formula, ω represents the strength parameter; Represents a set The base is The number of elements in the middle; The generated decoupled negative sample group Generate for image x i Negative decoupling of positive sample features The expression is, In the formula, In the k-th order unentangled negative sample group Samples with pseudo-label 'a'; According to the inequality of piano sounds, the following inequality relationship holds. τ represents the mode parameter; Based on the generated image x i By decoupling positive sample features and inequality relationships, an alternative decoupling learning loss function is obtained, expressed as follows: In the formula, c n This represents the prototype vector of the class corresponding to pedestrian n, where n ≠ y. i B represents the current training batch, and |B| represents the size of the current training batch; v i Indicates sample x i Features generated by feature extractor f, v i =f(x) i ); The generated decoupled positive sample group With decoupled negative samples To obtain the disturbed pedestrian features, the expression is: In the formula, λ represents the interference parameter; This represents the k-th order feature after interference from pedestrian a; Generate a feature for image x from the perturbed pedestrian features i Intimacy distribution The expression is, In the formula, N c Indicates the current number of pedestrians; Generate a feature for image x from the generated pedestrian features j Based on pedestrian characteristics Intimacy distribution Q j ;where x j Indicates with x i Images featuring the same pedestrians; By generated With Q j The expression for calculating the decoupling KL divergence regularity is as follows: By step and The total decoupling loss function is calculated as follows: Using L dise The feature extractor f is optimized, and its parameters are updated using the Adam algorithm; where the feature library M is updated by the current network, denoted as M[i]←v i .

3. The pedestrian re-identification method based on decoupled sample set learning according to claim 1, characterized in that, The feature extractor is a deep residual network or a visual translator network.

4. A pedestrian re-identification system based on decoupled sample set learning, characterized in that, include: The data acquisition module is used to acquire a given query image and a given set of pedestrian images; The feature extraction module is used to extract features based on the given query image and the given set of pedestrian images using a pre-trained feature extractor, so as to obtain the feature vector of the query image and the feature vector set of the set of pedestrian images. The sorting list acquisition module is used to obtain the Euclidean distance between the feature vector of the query image and each feature in the feature vector set of the pedestrian image set, and to obtain the sorting list of the pedestrian image set with respect to the query image; The result acquisition module is used to obtain the pedestrian re-identification result based on the sorted list; The training steps of the pre-trained feature extractor include: Each training image sample in the unlabeled pedestrian training image sample set is input into a pre-built feature extractor to obtain the features of each training image sample and store them in the feature library. In each iteration of training, pedestrian features are generated for the current pedestrian based on the current feature library. An auxiliary vector is constructed for each training image sample using these pedestrian features. Based on the auxiliary vector, a decoupled positive sample group and a decoupled negative sample group are constructed for each training image sample. The decoupled positive sample group and the decoupled negative sample group are introduced into an alternative decoupling learning loss function and a decoupling second-order similarity regularization to guide the feature learning process of the feature extractor, thus training the pre-trained feature extractor. After each iteration of the training process, the features of each training image sample are updated. The steps of generating pedestrian features for the current pedestrian based on the current feature library, constructing auxiliary vectors for each training image sample using the pedestrian features, and constructing decoupled positive sample groups and decoupled negative sample groups for each training image sample based on the auxiliary vectors include: Based on the input image x i The image number is used to extract the corresponding feature in the feature library. Where i represents the pedestrian sample number in the database, and M represents the feature library of the samples. Represents the characteristics of pedestrian i stored in M; For the current image x i Constructing auxiliary vector sets in, Indicates pedestrian y l The prototype vector of the corresponding class has the same pseudo-label y. l The average of the feature vectors corresponding to the samples; y i Indicates the estimated x i pseudo-tags; Based on auxiliary vector A i With image x i The relationship between the training image samples is calculated by removing image x from the training image sample set. i The remaining image samples and A i The histogram distribution of similarity probability between them is expressed as follows: In the formula, Representing the image x i Features stored in feature library M; Based on the obtained similarity probability histogram distribution P(x) i A i ), for image x i Constructing a decoupled positive sample group With decoupling negative sample groups The expression is, In the formula, argTopk(P(x i A i )) k Represents the distribution P(x) i A i The index of the k-th largest value in ), argTopk(P(x j A j )) k Represents the distribution P(x) j A j The index of the k-th largest value in the sample group; K represents the order of the decoupled sample group; x represents i k-th order decoupled positive sample group, x represents i The k-th order decoupled negative sample group, y i With y j Corresponding to the image x i With x j pseudo-tags, A i For x i The corresponding auxiliary vector set.

5. A pedestrian re-identification system based on decoupled sample set learning according to claim 4, characterized in that, The steps of introducing alternative decoupling learning loss functions into the decoupling positive sample group and the decoupling second-order similarity regularization to guide the feature learning process of the feature extractor, and training to obtain the pre-trained feature extractor include: The generated decoupled positive sample group Generate for image x i Positive decoupling positive sample features The expression is, In the formula, ω represents the strength parameter; Represents a set The base is The number of elements in the middle; The generated decoupled negative sample group Generate for image x i Negative decoupling of positive sample features The expression is, In the formula, In the k-th order unentangled negative sample group Samples with pseudo-label 'a'; According to the inequality of piano sounds, the following inequality relationship holds. τ represents the mode parameter; Based on the generated image x i By decoupling positive sample features and inequality relationships, an alternative decoupling learning loss function is obtained, expressed as follows: In the formula, c n This represents the prototype vector of the class corresponding to pedestrian n, where n ≠ y. i B represents the current training batch, and |B| represents the size of the current training batch; v i Indicates sample x i Features generated by feature extractor f, v i =f(x) i ); The generated decoupled positive sample group With decoupled negative samples To obtain the disturbed pedestrian features, the expression is: In the formula, λ represents the interference parameter; This represents the k-th order feature after interference from pedestrian a; Generate a feature for image x from the perturbed pedestrian features i Intimacy distribution The expression is, In the formula, N c Indicates the current number of pedestrians; Generate a feature for image x from the generated pedestrian features j Based on pedestrian characteristics Intimacy distribution Q j ;where x j Indicates with x i Images featuring the same pedestrians; By generated With Q j The expression for calculating the decoupling KL divergence regularity is as follows: By step and The total decoupling loss function is calculated as follows: Using L dise The feature extractor f is optimized, and its parameters are updated using the Adam algorithm; where the feature library M is updated by the current network, denoted as M[i]←v i .

6. The pedestrian re-identification system based on decoupled sample set learning according to claim 4, characterized in that, The feature extractor is a deep residual network or a visual translator network.

7. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the pedestrian re-identification method as described in any one of claims 1 to 3.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the pedestrian re-identification method according to any one of claims 1 to 3.

Images