Pedestrian re-identification model training method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a pedestrian re-recognition model training method and device, electronic equipment and a storage medium, and relates to the technical field of big data, and the method comprises the steps: obtaining an unlabeled pedestrian data set and a camera type corresponding to the pedestrian data set; performing at least one image transformation operation on the pedestrian data set, and performing feature extraction on the transformed pedestrian data set to obtain pedestrian image features corresponding to the pedestrian data set; clustering pedestrian images in the pedestrian data set according to the pedestrian image features to obtain a pseudo tag set corresponding to the pedestrian data set; estimating the probability that each pseudo label is wrongly marked according to the camera category, and dividing the pseudo label set into a confidence data set and a potential noise data set based on the probability; and performing model training according to the confidence data set and the potential noise data set to obtain a pedestrian re-identification model, thereby improving the accuracy of model identification and improving the robustness of noise caused by camera change.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a training method for a pedestrian re-identification model, a training device for a pedestrian re-identification model, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the maturity of artificial intelligence technology and the improvement of computing resources, video surveillance technology has been widely used in smart cities, smart homes, smart monitoring, smart retail and other fields. However, the popularity of monitoring systems has also brought about massive video and image data. Traditional manual monitoring and analysis requires a lot of manpower. It is inefficient and costly to find the target person in massive video data by human eyes alone. In this regard, pedestrian re-identification technology, as a key technology for intelligent application and analysis of massive data videos, has extremely high application value. This technology uses computer vision-related algorithms to determine whether there are pedestrian targets in a sequence of videos or images. It is widely regarded as a sub-field problem of image retrieval and is mainly used to solve pedestrian recognition and retrieval in cross-camera scenarios. In the recognition process, high-quality face images are usually not obtained due to relatively low camera resolution, incorrect shooting angles, and long shooting distances. Therefore, pedestrian re-identification technology can assist face recognition technology and be used to query and match pedestrians whose faces cannot be clearly photographed across cameras.

[0003] In practical applications, there is inevitably corresponding noise in the process of pedestrian re-identification. This kind of erroneous information will be continuously amplified with the iterative training of the model, misleading the learning process of the model and thus affecting the final prediction effect of the model. Summary of the invention

[0004] The embodiments of the present invention provide a training method, device, electronic device and computer-readable storage medium for a pedestrian re-identification model to solve or partially solve the problem that the pedestrian re-identification model is easily affected by noise, which reduces the model prediction effect.

[0005] The embodiment of the present invention discloses a method for training a pedestrian re-identification model, comprising:

[0006] Obtain an unlabeled pedestrian data set and a camera category corresponding to the pedestrian data set;

[0007] Performing at least one image transformation operation on the pedestrian data set, and performing feature extraction on the transformed pedestrian data set to obtain pedestrian image features corresponding to the pedestrian data set;

[0008] Clustering the pedestrian images in the pedestrian data set according to the pedestrian image features to obtain a pseudo label set corresponding to the pedestrian data set, wherein the pseudo label set includes pseudo labels corresponding to the pedestrian images;

[0009] estimating the probability that each of the pseudo labels is incorrectly labeled according to the camera category, and dividing the pseudo label set into a confidence data set and a potential noise data set based on the probability, wherein the confidence data set contains reliable pseudo labels and the potential noise data set contains pseudo labels that may contain noise;

[0010] Model training is performed based on the confidence data set and the potential noise data set to obtain a pedestrian re-identification model, where the pedestrian re-identification model is used to identify and match the same pedestrian in surveillance videos from different cameras, at different times or at different locations.

[0011] The embodiment of the present invention further discloses a training device for a pedestrian re-identification model, comprising:

[0012] A data acquisition module, used to acquire an unlabeled pedestrian data set and a camera category corresponding to the pedestrian data set;

[0013] An image transformation module, used to perform at least one image transformation operation on the pedestrian data set, and perform feature extraction on the transformed pedestrian data set to obtain pedestrian image features corresponding to the pedestrian data set;

[0014] A clustering module, used for clustering pedestrian images in the pedestrian data set according to the pedestrian image features to obtain a pseudo label set corresponding to the pedestrian data set, wherein the pseudo label set includes pseudo labels corresponding to the pedestrian images;

[0015] a label modeling module, configured to estimate the probability that each of the pseudo labels is incorrectly labeled according to the camera category, and divide the pseudo label set into a confidence data set and a potential noise data set based on the probability, wherein the confidence data set contains reliable pseudo labels and the potential noise data set contains pseudo labels that may contain noise;

[0016] A model training module is used to perform model training based on the confidence data set and the potential noise data set to obtain a pedestrian re-identification model, wherein the pedestrian re-identification model is used to identify and match the same pedestrian in surveillance videos from different cameras, at different times or at different locations.

[0017] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0018] The memory is used to store computer programs;

[0019] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0020] The embodiment of the present invention further discloses a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, enables the processors to execute the method described in the embodiment of the present invention.

[0021] The embodiments of the present invention include the following advantages:

[0022] In an embodiment of the present invention, an unlabeled pedestrian data set and a camera category corresponding to the pedestrian data set are obtained, and then at least one image transformation operation is performed on the pedestrian data set, and feature extraction is performed on the transformed pedestrian data set to obtain pedestrian image features corresponding to the pedestrian data set, and pedestrian images in the pedestrian data set are clustered according to the pedestrian image features to obtain a pseudo-label set corresponding to the pedestrian data set, the pseudo-label set including pseudo-labels corresponding to the pedestrian images, and then the probability of each pseudo-label being incorrectly labeled is estimated according to the camera category, and the pseudo-label set is divided into a confidence data set and a potential noise data set based on the probability, the confidence data set contains reliable pseudo-labels, and the potential noise data set contains reliable pseudo-labels. The data set contains pseudo labels that may contain noise. The model is then trained based on the confidence data set and the potential noise data set to obtain a pedestrian re-identification model. The pedestrian re-identification model is used to identify and match the same pedestrian in surveillance videos from different cameras, at different times or at different locations. Therefore, in the process of building the pedestrian re-identification model, the samples are divided into cleanliness levels by generating pseudo labels, and the training data is divided into relatively clean data and noisy data. Model training is performed based on the classified data, which improves the accuracy of model recognition. At the same time, camera categories are introduced so that the model can adapt to scenes with camera deviations, effectively improving the robustness to noise caused by camera changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flowchart of a method for training a pedestrian re-identification model provided in an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of an application scenario provided in an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of the result of cross-camera pedestrian re-identification provided in an embodiment of the present invention;

[0026] Figure 4 It is a structural block diagram of a training device for a pedestrian re-identification model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] As an example, pedestrian re-identification technology, as a key technology for intelligent application and analysis of massive data videos, has extremely high application value. This technology uses computer vision-related algorithms to determine whether there are pedestrian targets in a sequence of videos or images. It is widely regarded as a sub-field problem of image retrieval and is mainly used to solve pedestrian recognition and retrieval in cross-camera scenarios. In the recognition process, high-quality facial images are usually not available due to relatively low camera resolution, incorrect shooting angles, and long shooting distances. Therefore, pedestrian re-identification technology can assist facial recognition technology in cross-camera query and matching of pedestrians whose faces cannot be clearly photographed.

[0029] Among them, in related technologies, unsupervised person re-identification usually generates pseudo labels based on clustering. A common problem is that its clustering results usually inevitably contain noise, and this erroneous information will continue to amplify with iterative training, misleading the model's learning process and affecting the final effect of the model. How to distinguish between correct label information and noise is particularly important. At the same time, previous work on unsupervised person re-identification ignored the noise caused by differences in data distribution between cameras, resulting in clustering of in-class samples from different cameras into different labels, and no explicit online identification and correction of noisy labels.

[0030] In this regard, in the present invention, an unlabeled pedestrian dataset and a camera category corresponding to the pedestrian dataset are obtained, and then at least one image transformation operation is performed on the pedestrian dataset, and feature extraction is performed on the transformed pedestrian dataset to obtain pedestrian image features corresponding to the pedestrian dataset, and pedestrian images in the pedestrian dataset are clustered according to the pedestrian image features to obtain a pseudo-label set corresponding to the pedestrian dataset, the pseudo-label set including pseudo-labels corresponding to the pedestrian images, and then the probability of each pseudo-label being incorrectly labeled is estimated according to the camera category, and the pseudo-label set is divided into a confident dataset and a potential noise dataset based on the probability, the confident dataset containing reliable pseudo-labels and the potential noise dataset containing reliable pseudo-labels. The data set contains pseudo labels that may contain noise. The model is then trained based on the confidence data set and the potential noise data set to obtain a pedestrian re-identification model. The pedestrian re-identification model is used to identify and match the same pedestrian in surveillance videos from different cameras, at different times or at different locations. Therefore, in the process of building the pedestrian re-identification model, the samples are divided into cleanliness levels by generating pseudo labels, and the training data is divided into relatively clean data and noisy data. Model training is performed based on the classified data, which improves the accuracy of model recognition. At the same time, camera categories are introduced so that the model can adapt to scenes with camera deviations, effectively improving the robustness to noise caused by camera changes.

[0031] Reference Figure 1 , shows a flowchart of a method for training a pedestrian re-identification model provided in an embodiment of the present invention, which may specifically include the following steps:

[0032] Step 101, obtaining an unlabeled pedestrian dataset and a camera category corresponding to the pedestrian dataset;

[0033] In an embodiment of the present invention, in the process of constructing a pedestrian re-identification model, an unlabeled pedestrian data set and a camera category corresponding to the pedestrian data set may be first obtained. The pedestrian data set may include images or video frames of pedestrians and corresponding shooting time and location information, etc. The camera category may represent different camera identifiers, that is, pedestrian images may include images collected by different cameras from different angles, different times, and different locations, etc. By obtaining the camera category, the data distribution differences between different cameras may be introduced during the model training process, solving the problem of clustering class samples of different cameras into different labels, and effectively alleviating the problem of pseudo-label noise generated by clustering.

[0034] Step 102, performing at least one image transformation operation on the pedestrian dataset, and performing feature extraction on the transformed pedestrian dataset to obtain pedestrian image features corresponding to the pedestrian dataset;

[0035] Before training the model using the pedestrian dataset, at least one image transformation operation can be performed on the pedestrian images in the pedestrian dataset to increase the diversity of the data and improve the generalization ability of the model. After completing the image transformation operation, feature extraction can be performed on the transformed data to obtain the pedestrian image features contained in the pedestrian images in the pedestrian dataset.

[0036] In a specific implementation, at least one image transformation operation can be performed on the pedestrian image to obtain the target pedestrian image, increase the diversity of data, and improve the generalization ability of the model, and then feature extraction is performed on the target pedestrian image to obtain the corresponding pedestrian image features. The image transformation operation includes at least one of rotation, flipping, scaling, cropping, brightness adjustment, contrast adjustment, color jittering, and normalization.

[0037] For example, the following image transformation operations can be performed on an image:

[0038] (1) Rotation and flipping: Randomly rotating and flipping images to simulate different perspectives and directions helps the model learn features that are not affected by direction.

[0039] (2) Scaling and Cropping: By scaling and randomly cropping images, we simulate different observation distances and viewing angles to ensure that the model can recognize features of different scales.

[0040] (3) Brightness and contrast adjustment: Change the brightness and contrast of the image to adapt to image changes under different lighting conditions and improve the robustness of the model in different lighting environments.

[0041] (4) Color jitter and normalization: Make slight perturbations in color and normalize image pixels to balance the weights of each channel, thereby improving the stability of the model.

[0042] According to the above process, after obtaining the target pedestrian dataset after image transformation, the target pedestrian dataset can be feature extracted through the corresponding encoder network. The enhanced image can be converted into feature representation through the encoder network to obtain the corresponding pedestrian image features.

[0043] For example, image features can be extracted through pre-trained models stacked with multiple layers of convolutions, such as Resnet-50. Through the coordination between data enhancement and feature extraction, meaningful features can be effectively extracted from the image, laying a good data foundation for subsequent visual feature clustering, noise pseudo-label modeling, and noise pseudo-label correction, thereby improving the effect of model training.

[0044] Step 103, clustering the pedestrian images in the pedestrian data set according to the pedestrian image features to obtain a pseudo label set corresponding to the pedestrian data set, wherein the pseudo label set includes pseudo labels corresponding to the pedestrian images;

[0045] After obtaining the pedestrian image features, the pedestrian images in the pedestrian dataset can be clustered based on the extracted pedestrian image features to obtain the pseudo-label corresponding to each pedestrian image, and then a pseudo-label set corresponding to the pedestrian dataset can be constructed based on all the pseudo-labels. For example, the extracted feature vectors can be clustered using a clustering algorithm. For example, assuming that the images are divided into 10 categories, the feature vectors are clustered using a clustering algorithm; the cluster labels corresponding to each image are obtained.

[0046] In some feasible implementations, the DBSCAN algorithm can be used for clustering. The algorithm is a density-based unsupervised learning method suitable for sample data sets with complex shapes and noise. The feature vector corresponding to the pedestrian image can be used as input. The DBSCAN algorithm identifies areas with sufficiently high density by defining the distance threshold and the minimum number of neighbors between sample points, and clusters the sample points in these areas into the same group. Among them, sample points in different density areas will be divided into different clusters, while noise points are marked as outliers and are not assigned to any cluster. Thus, through DBSCAN clustering, the module assigns a pseudo label to each pedestrian image, that is, the ID of the cluster. It should be noted that the pseudo label can not only be used as a data label for subsequent model training, but also effectively improve the model's learning ability for unsupervised data. Finally, after clustering, it can be output that each pedestrian image feature will be accompanied by a corresponding pseudo label for subsequent noise pseudo label modeling and noise pseudo label correction.

[0047] Step 104, estimating the probability that each of the pseudo labels is incorrectly labeled according to the camera category, and dividing the pseudo label set into a confidence data set and a potential noise data set based on the probability, wherein the confidence data set contains reliable pseudo labels and the potential noise data set contains pseudo labels that may contain noise;

[0048] In an embodiment of the present invention, in order to further process possible noisy pseudo-labels, the present invention introduces camera categories and estimates the probability of each pseudo-label being incorrectly labeled based on the camera category, and then divides the pseudo-label set into a confident data set and a potential noise data set based on the calculated probability. The confident data set contains reliable pseudo-labels, and the potential noise data set contains pseudo-labels that may contain noise, thereby achieving cleanliness processing of the data, identifying and processing samples that may be incorrectly labeled, so as to improve the robustness and accuracy of the model when processing unlabeled data.

[0049] It should be noted that, assuming that there are different camera categories in the data set, the data in each camera category can be regarded as a mixed Gaussian distribution. In this embodiment of the present invention, the EM (Expectation-Maximization) algorithm can be used to fit these Gaussian mixture models (Camera-Aware GMM) to more accurately capture the distribution differences between camera categories. In an optional manner, a latent variable is introduced to represent the posterior probability that the sample belongs to a specific distribution, and the latent variable formula is re-derived by the Bayesian formula, and the posterior probability of each sample is calculated in the E step, and then in the M step, the parameters of the mixed distribution, including the mean and variance, are updated based on the current estimated posterior probability. This process is iterated continuously until convergence or the maximum number of iterations is reached. Finally, we estimate the probability of clean or noisy for each sample through the posterior probability, thereby identifying and processing those samples that may be mislabeled. This process effectively improves the robustness and accuracy of the model when processing unlabeled data.

[0050] In a specific implementation, the camera hybrid perception model corresponding to the camera category can be obtained first, and then the camera category can be input into the camera hybrid perception model to estimate the probability of each pseudo label being incorrectly labeled. Specifically, the probability p(l) that the target pedestrian image is incorrectly labeled can be calculated by the following formula:

[0051]

[0052] Among them, ω c represents the c-th camera category, λ k Represents the kth distribution corresponding to the camera.

[0053] In one example, the EM algorithm can be used to fit the camera mixed perception model, specifically including: in the E step, introducing the latent variable γ c,k =p(c,k|l) represents the posterior probability, and the latent variable formula is rewritten using the Bayesian formula as follows:

[0054]

[0055] Get Gamma c,k (l), use M steps to estimate the distribution parameters Its mean and variance are calculated as follows:

[0056]

[0057] Then, update ω c and λ k As shown below:

[0058]

[0059] Repeat the above steps E and M until convergence or the maximum number of iterations is reached;

[0060] Finally, the probability of each pseudo-label being incorrectly labeled is calculated by the following formula:

[0061]

[0062] Among them, P(k|l i ) represents the probability that the pseudo-label is incorrectly labeled.

[0063] Through the above process, the probability of each pseudo-label being mislabeled can be accurately calculated. The larger the probability value, the greater the possibility that the pseudo-label is mislabeled, and the greater the noise of the pseudo-label. Conversely, the smaller the possibility that the pseudo-label is mislabeled, the smaller the noise of the pseudo-label, and the higher the confidence.

[0064] Furthermore, after obtaining the probability that each pseudo-label is incorrectly labeled, the pseudo-label with a probability greater than or equal to a preset threshold can be used as a first pseudo-label that may contain noise, and a corresponding potential noise data set is constructed based on the first pseudo-label; the pseudo-label with a probability less than the preset threshold is used as a credible second pseudo-label, and a corresponding confidence data set is constructed based on the second pseudo-label, thereby dividing the samples into highly credible samples and questionable samples. By modeling the noise labels, the labels of the noise samples are corrected, and the corrected samples are re-sent into the model for learning, thereby improving the robustness to noise caused by camera changes.

[0065] It should be noted that, for the confidence data set, it may include the corresponding first pedestrian image and the first pseudo label corresponding to the first pedestrian figure. Similarly, the potential noise data set may include the corresponding second pedestrian image and the second pseudo label corresponding to the second pedestrian image.

[0066] Step 105 , performing model training based on the confidence data set and the potential noise data set to obtain a pedestrian re-identification model, wherein the pedestrian re-identification model is used to identify and match the same pedestrian in surveillance videos from different cameras, at different times or at different locations.

[0067] After the data is divided into a confidence data set and a potential noise data set according to the probability of being mislabeled, model training can be performed based on the confidence data set and the potential noise data set to obtain a pedestrian re-identification model. Through the pedestrian re-identification model, the same pedestrian can be identified and matched in surveillance videos from different cameras, at different times or at different locations. Therefore, in the process of building the pedestrian re-identification model, the samples are divided into cleanliness levels through pseudo-label generation, and the training data is divided into relatively clean data and noise data. Model training is performed based on the classified data, which improves the accuracy of model recognition. At the same time, camera categories are introduced so that the model can adapt to scenes with camera deviations, effectively improving the robustness to noise caused by camera changes.

[0068] In some feasible implementations, a neural network encoder corresponding to a confidence data set and a potential noise data set can be obtained, the neural network encoder includes at least a first neural network encoder and a second neural network encoder, and then the confidence data set and the potential noise data set are input into the first neural network encoder for processing to obtain a corresponding first category probability distribution, and the confidence data set and the potential noise data set are input into the second neural network encoder for processing to obtain a corresponding second category probability distribution, and then model training is performed according to the first category probability distribution and the second category probability distribution to obtain a pedestrian re-identification model.

[0069] It should be noted that two independent neural networks are used to collaboratively process and correct the noise labels in the training data. Specifically, the two networks divide the training data for each other. Since different classification networks have different decision boundaries, they show different learning abilities when processing noise labels. For the classification problem of M types of pedestrians, the training data is divided into a confidence data set and a potential noise data set through the probability obtained in the above embodiment. The confidence data set contains labels that are considered to be reliable, while the potential noise data set contains labels that may contain noise. Then, the two neural network encoders can be used to predict the category probability distribution respectively. In addition, the predicted category probability distributions of the corresponding EMA (Exponential Moving Average) models of the two neural networks are respectively arithmetic averaged to obtain the final predicted probability and re-learned from the corrected samples.

[0070] In the specific implementation, represents the confidence data set, represents the potential noise data set, and the first neural network encoder is recorded as The second neural network encoder is denoted as The category probability distribution predicted by the first neural network encoder is recorded as p(·), the category probability distribution predicted by the second neural network encoder is recorded as q(·), the category probability distribution predicted by the corresponding exponential moving average model in the first neural network encoder is recorded as p′(·), and the category probability distribution predicted by the corresponding exponential moving average model in the second neural network encoder is recorded as q′(·), and then the probability of the pseudo-label being incorrectly marked is corrected by the following formula:

[0071]

[0072] is the average predicted probability, and then the average predicted probability is sharpened to obtain the final probability:

[0073]

[0074] in, The average predicted probability The function for sharpening, T represents the temperature coefficient,

[0075] Get the following loss function:

[0076]

[0077] The model is trained using a potential noise data set, a confidence data set, a first target probability corrected by each first pseudo-label in the potential noise data set, a second target probability corrected by each second pseudo-label in the confidence data set, and a pedestrian data set, and iterated according to the loss function until the loss function meets a preset condition, and the iteration is stopped to obtain a pedestrian re-identification model;

[0078] in, for The maximum value in the predicted probability distribution; r is a preset threshold, which is used to determine whether the maximum value of the predicted probability exceeds the preset threshold; is the cross entropy loss function, which is used to measure the predicted probability distribution With the true label The difference between them; M is the total number of camera categories; For the model The predicted probability distribution of .

[0079] In the above process, the predicted category probability distributions of the EMA (Exponential Moving Average) models corresponding to the two neural networks are respectively arithmetic averaged to obtain the final predicted probability and then re-learned from the corrected samples. Thus, by re-sending the corrected samples into the network for learning, the robustness of the model to the relevant noise processing in the prediction process is effectively improved.

[0080] It should be noted that the embodiments of the present invention include but are not limited to the above examples. It is understandable that those skilled in the art can also make settings according to actual needs under the guidance of the ideas of the embodiments of the present invention, and the present invention is not limited to this.

[0081] In an embodiment of the present invention, an unlabeled pedestrian data set and a camera category corresponding to the pedestrian data set are obtained, and then at least one image transformation operation is performed on the pedestrian data set, and feature extraction is performed on the transformed pedestrian data set to obtain pedestrian image features corresponding to the pedestrian data set, and pedestrian images in the pedestrian data set are clustered according to the pedestrian image features to obtain a pseudo-label set corresponding to the pedestrian data set, the pseudo-label set including pseudo-labels corresponding to the pedestrian images, and then the probability of each pseudo-label being incorrectly labeled is estimated according to the camera category, and the pseudo-label set is divided into a confidence data set and a potential noise data set based on the probability, the confidence data set contains reliable pseudo-labels, and the potential noise data set contains reliable pseudo-labels. The data set contains pseudo labels that may contain noise. The model is then trained based on the confidence data set and the potential noise data set to obtain a pedestrian re-identification model. The pedestrian re-identification model is used to identify and match the same pedestrian in surveillance videos from different cameras, at different times or at different locations. Therefore, in the process of building the pedestrian re-identification model, the samples are divided into cleanliness levels by generating pseudo labels, and the training data is divided into relatively clean data and noisy data. Model training is performed based on the classified data, which improves the accuracy of model recognition. At the same time, camera categories are introduced so that the model can adapt to scenes with camera deviations, effectively improving the robustness to noise caused by camera changes.

[0082] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following examples are used for exemplary description:

[0083] As an example, see Figure 2 , showing a schematic diagram of an application scenario provided in an embodiment of the present invention. In an embodiment of the present invention, a two-stage method for noise pseudo-label recognition and correction can be provided. The method can be implemented by a visual feature extraction module, a visual feature clustering module, a noise pseudo-label modeling module, and a noise pseudo-label correction module, among which the core modules are the noise pseudo-label modeling module and the noise pseudo-label correction module.

[0084] The visual feature extraction module is responsible for extracting features from the input image using a deep neural network. It includes a data enhancement submodule that performs multiple transformations on the input image to increase the diversity of the data and improve the generalization ability of the model. Specific operations include:

[0085] (1) Rotation and flipping. Randomly rotate and flip the image to simulate different perspectives and orientations. This helps the model learn features that are not affected by orientation.

[0086] (2) Scaling and cropping: By scaling and randomly cropping images, we simulate different viewing distances and viewing angles to ensure that the model can recognize features of different scales.

[0087] (3) Brightness and contrast adjustment: Change the brightness and contrast of the image to adapt to image changes under different lighting conditions and improve the robustness of the model under different lighting environments.

[0088] (4) Color jitter and normalization: Slightly perturb the color and normalize the image pixels to balance the weights of each channel, thereby improving the stability of the model.

[0089] The encoder network is the core of the visual feature extraction module, responsible for converting the data-enhanced image into a feature representation, such as the common Resnet-50 and other multi-layer convolution stacked pre-trained models. Through the cooperation of the data enhancement submodule and the encoder network, the visual feature extraction module can effectively extract meaningful features from the image, laying the foundation for subsequent visual feature clustering, noise pseudo-label modeling, and noise pseudo-label correction.

[0090] For the visual feature clustering module, it mainly assigns pseudo labels to all samples (pedestrian image features) through visual feature clustering. By adopting the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm, which is a density-based unsupervised learning method, it is particularly suitable for sample data sets with complex shapes and noise. Specifically, first, the module takes the feature vector of the pedestrian image as input. The DBSCAN algorithm identifies areas with sufficiently high density by defining the distance threshold and the minimum number of neighbors between sample points, and clusters the sample points in these areas into the same group. Sample points in different density areas will be divided into different clusters, while noise points are marked as outliers and are not assigned to any cluster. Through DBSCAN clustering, the module assigns a pseudo label, i.e., the ID of the cluster, to each pedestrian image. Pseudo labels can not only be used as data labels for subsequent model training, but also effectively improve the model's learning ability for unsupervised data. In the output of the module, each pedestrian image feature will be accompanied by a corresponding pseudo label for subsequent noise pseudo label modeling and noise pseudo label correction.

[0091] For the noisy pseudo-label modeling module, in order to further deal with possible noisy pseudo-labels, a camera-aware GMM is introduced, which estimates the probability of each sample being mislabeled online through the loss of the sample. Assuming that there are different camera categories in the dataset, the data within each camera category can be regarded as a mixed Gaussian distribution. We use the EM (Expectation-Maximization) algorithm to fit these Gaussian mixture models (GMM) to more accurately capture the distribution differences between camera categories. Specifically, we introduce latent variables to represent the posterior probability that a sample belongs to a specific distribution. The latent variable formula is re-derived through the Bayesian formula, and the posterior probability of each sample is calculated in the E step. In the M step, the parameters of the mixture distribution, including the mean and variance, are updated based on the current estimated posterior probability. The process is iterated until convergence or the maximum number of iterations is reached. Finally, we estimate the probability of each sample being clean or noisy through the posterior probability, thereby identifying and processing those samples that may be mislabeled. This process effectively improves the robustness and accuracy of the model when processing unlabeled data.

[0092] For the noisy pseudo-label correction module, in order to solve the label noise problem, this module proposes a strategy based on dual-network collaborative correction, so that the model can learn more effectively from the corrected labels. Specifically, we use two independent neural networks to jointly process and correct the noisy labels in the training data. In this method, the two networks divide the training data for each other. Since different classification networks have different decision boundaries, they show different learning abilities when processing noisy labels. For the classification problem of M types of pedestrians, we define two data sets: the confidence data set and the potential noise data set. The confidence data set contains labels that are considered reliable, while the potential noise data set contains labels that may contain noise. The two neural network encoders are used to predict the category probability distribution respectively. In addition, the predicted category probability distributions of the corresponding EMA (Exponential Moving Average) models of the two neural networks are arithmetic averaged to obtain the final predicted probability and re-learned from the corrected samples.

[0093] For example, given an unlabeled pedestrian dataset Where N represents the number of samples, represents all camera IDs in the dataset. The image feature encoder is defined as f θ (x), in the first stage, the clustering algorithm is used to cluster the data set Cluster into W clusters to form a new data set in represents the pseudo label of the i-th sample.

[0094] The Camera-Aware GMM is used to online estimate the probability p(l) that a sample is mislabeled based on the loss of the sample.

[0095]

[0096] Among them, ω c represents the c-th camera category, λ k It represents the kth distribution corresponding to the camera, and uses the EM algorithm to fit the (camera mixed perception model) GMM.

[0097] Specifically, the hidden variable γ is introduced c,k =p(c,k|l) represents the posterior probability, and the latent variable formula is rewritten using the Bayesian formula as follows:

[0098]

[0099] Assuming that γ is given c,k (l), use M steps to estimate the distribution parameters Its mean and variance are calculated as follows:

[0100]

[0101] Then repeat the above steps E and M until convergence or the maximum number of iterations is reached. Finally, the probability of the sample being clean or noisy is obtained through the posterior probability.

[0102]

[0103] In order to filter label noise, this patent uses two networks to calibrate together so that the model can learn from the calibrated labels. The two networks divide the training data for each other. Different classification networks have different decision boundaries and therefore have different learning capabilities. For the classification of M types of people, represents the confidence data set, represents the potential noise data set. The two neural network encoders are denoted as and The category probability distribution predicted by the model is recorded as p(·) and q(·). The category probability distribution predicted by the EMA model corresponding to the two neural network encoders is recorded as p′(·) and q′(·). The final predicted corrected probability is as follows:

[0104]

[0105] is the average predicted probability, which is sharpened to obtain the final probability:

[0106] Update c and λ k As shown below:

[0107]

[0108]

[0109] Where T represents the temperature coefficient, the loss function is as follows:

[0110]

[0111] Finally, the pedestrian re-identification model trained by the unsupervised noise pseudo-label recognition and correction method is loaded for pedestrian re-identification to achieve the purpose of cross-camera pedestrian retrieval.

[0112] As described above, refer to Figure 3 , shows a schematic diagram of the result of cross-camera pedestrian re-identification provided in an embodiment of the present invention, wherein, Figure 3 (a) shows the results of cross-camera pedestrian re-identification without using the pseudo-label correction model (PLMC), and the figure shows the recognition effect under different training rounds (Epoch); Figure 3 (b) shows the results of cross-camera pedestrian re-identification when using the pseudo-label correction model (PLMC). The figure also shows the recognition effect under different training rounds (Epoch). Among them, 10Epoch: represents the recognition result after 10 rounds of training; 40Epoch: represents the recognition result after 40 rounds of training. It can be seen that the effect of cross-camera pedestrian re-identification with and without the pseudo-label correction model (PLMC) is shown by comparison. By observing the recognition results under different training rounds, it can be evaluated that the pseudo-label correction model has a significant improvement effect on the performance of pedestrian re-identification.

[0113] Through the above process, the pseudo-label generation is divided into cleanliness samples through the noise pseudo-label modeling module, and the training data is divided into relatively clean data and noisy data. The clean data is directly sent to the network for learning, which reduces errors and uncertainties, thereby improving the credibility of the counting results.

[0114] At the same time, the noise pseudo-label correction module models the noise pseudo-labels, corrects the labels of the noise samples, and re-sends the corrected samples into the network for learning, effectively improving the robustness to the noise caused by camera changes.

[0115] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0116] Reference Figure 4 , shows a structural block diagram of a training device for a pedestrian re-identification model provided in an embodiment of the present invention, which may specifically include the following modules:

[0117] A data acquisition module 401 is used to acquire an unlabeled pedestrian data set and a camera category corresponding to the pedestrian data set;

[0118] An image transformation module 402, configured to perform at least one image transformation operation on the pedestrian dataset, and perform feature extraction on the transformed pedestrian dataset to obtain pedestrian image features corresponding to the pedestrian dataset;

[0119] A clustering module 403 is used to cluster the pedestrian images in the pedestrian data set according to the pedestrian image features to obtain a pseudo label set corresponding to the pedestrian data set, wherein the pseudo label set includes pseudo labels corresponding to the pedestrian images;

[0120] a label modeling module 404, configured to estimate the probability that each of the pseudo labels is incorrectly labeled according to the camera category, and divide the pseudo label set into a confidence data set and a potential noise data set based on the probability, wherein the confidence data set contains reliable pseudo labels and the potential noise data set contains pseudo labels that may contain noise;

[0121] The model training module 405 is used to perform model training based on the confidence data set and the potential noise data set to obtain a pedestrian re-identification model, and the pedestrian re-identification model is used to identify and match the same pedestrian in surveillance videos from different cameras, at different times or at different locations.

[0122] In some feasible implementations, the pedestrian dataset includes pedestrian images, and the image transformation module 402 is specifically used for:

[0123] Performing at least one image transformation operation on the pedestrian image to obtain a target pedestrian image;

[0124] Extracting features of the target pedestrian image to obtain corresponding pedestrian image features;

[0125] The image transformation operation includes at least one of rotation, flipping, scaling, cropping, brightness adjustment, contrast adjustment, color jittering and normalization.

[0126] In some feasible implementations, the label modeling module 404 is specifically used to:

[0127] Obtaining a camera hybrid perception model corresponding to the camera category;

[0128] The camera category is input into the camera mixed perception model to estimate the probability that each of the pseudo labels is incorrectly labeled.

[0129] In some feasible implementations, the label modeling module 404 is specifically used to:

[0130] The probability p(l) that the target pedestrian image is mislabeled is calculated by the following formula:

[0131]

[0132] Among them, ω c represents the c-th camera category, λ k Represents the kth distribution corresponding to the camera.

[0133] In some feasible implementations, the label modeling module 404 is specifically used to:

[0134] The EM algorithm is used to fit the camera mixed perception model, including:

[0135] In the E step, the hidden variable γ is introduced c,k =p(c,k|l) represents the posterior probability, and the latent variable formula is rewritten using the Bayesian formula as follows:

[0136]

[0137] Get Gamma c,k (l), use M steps to estimate the distribution parameters Its mean and variance are calculated as follows:

[0138]

[0139] Then, update ω c and λ k As shown below:

[0140]

[0141] Repeat the above steps E and M until convergence or the maximum number of iterations is reached;

[0142] Finally, the probability that each pseudo-label is incorrectly labeled is calculated by the following formula:

[0143]

[0144] Wherein, the P(k|l i ) represents the probability that the pseudo-label is incorrectly labeled.

[0145] In some feasible implementations, the label modeling module 404 is specifically used to:

[0146] Taking a pseudo label with a probability greater than or equal to a preset threshold as a first pseudo label that may contain noise, and constructing a corresponding potential noise data set based on the first pseudo label;

[0147] The pseudo-label with a probability less than the preset threshold is taken as a credible second pseudo-label, and a corresponding confidence data set is constructed based on the second pseudo-label.

[0148] In some feasible implementations, the model training module 405 is specifically used to:

[0149] Acquire a neural network encoder corresponding to the confidence data set and the potential noise data set, wherein the neural network encoder comprises at least a first neural network encoder and a second neural network encoder;

[0150] Inputting the confidence data set and the potential noise data set into the first neural network encoder for processing to obtain a corresponding first category probability distribution;

[0151] Inputting the confidence data set and the potential noise data set into the second neural network encoder for processing to obtain a corresponding second category probability distribution;

[0152] Model training is performed according to the first category probability distribution and the second category probability distribution to obtain a pedestrian re-identification model.

[0153] In some possible implementations, represents the confidence data set, represents a potential noise data set, and the first neural network encoder is recorded as The second neural network encoder is denoted as The category probability distribution predicted by the first neural network encoder is recorded as p(·), the category probability distribution predicted by the second neural network encoder is recorded as q(·), the category probability distribution predicted by the corresponding exponential moving average model in the first neural network encoder is recorded as p′(·), and the category probability distribution predicted by the corresponding exponential moving average model in the second neural network encoder is recorded as q′(·), and the model training module is specifically used for:

[0154] The probability of the pseudo-label being incorrectly labeled is corrected by the following formula:

[0155]

[0156] Said is the average predicted probability, and then the average predicted probability is sharpened to obtain the final probability:

[0157]

[0158] Among them, the is the average predicted probability The function of sharpening is, T represents the temperature coefficient,

[0159] Get the following loss function:

[0160]

[0161] The model is trained by using the potential noise data set, the confidence data set, the first target probability corrected by each first pseudo-label in the potential noise data set, and the second target probability corrected by each second pseudo-label in the confidence data set, and iterating according to the loss function until the loss function meets a preset condition, stopping the iteration, and obtaining a pedestrian re-identification model;

[0162] Among them, the For the The maximum value in the predicted probability distribution; r is a preset threshold value, which is used to determine whether the maximum value of the predicted probability exceeds the preset threshold value; is the cross entropy loss function, which is used to measure the predicted probability distribution With the true label The difference between; the M is the total number of camera categories; the For the model The predicted probability distribution of .

[0163] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0164] In addition, an embodiment of the present invention further provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, each process of the training method embodiment of the above-mentioned pedestrian re-identification model is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0165] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the training method embodiment of the above-mentioned pedestrian re-identification model is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0166] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0167] It should be understood by those skilled in the art that the embodiments of the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, EEPROM, Flash, and eMMC, etc.) containing computer-usable program codes.

[0168] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0169] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0170] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0171] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0172] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0173] The above is a detailed introduction to a training method for a pedestrian re-identification model and a training device for a pedestrian re-identification model provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for training a person re-identification model, characterized in that: include: Obtain an unlabeled pedestrian data set and a camera category corresponding to the pedestrian data set; Performing at least one image transformation operation on the pedestrian data set, and performing feature extraction on the transformed pedestrian data set to obtain pedestrian image features corresponding to the pedestrian data set; Clustering the pedestrian images in the pedestrian data set according to the pedestrian image features to obtain a pseudo label set corresponding to the pedestrian data set, wherein the pseudo label set includes pseudo labels corresponding to the pedestrian images; estimating the probability that each of the pseudo labels is incorrectly labeled according to the camera category, and dividing the pseudo label set into a confidence data set and a potential noise data set based on the probability, wherein the confidence data set contains reliable pseudo labels and the potential noise data set contains pseudo labels that may contain noise; Model training is performed based on the confidence data set and the potential noise data set to obtain a pedestrian re-identification model, where the pedestrian re-identification model is used to identify and match the same pedestrian in surveillance videos from different cameras, at different times or at different locations.

2. The method according to claim 1, characterized in that The pedestrian data set includes pedestrian images, and performing at least one image transformation operation on the pedestrian data set and extracting features from the transformed pedestrian data set to obtain pedestrian image features corresponding to the pedestrian data set includes: Performing at least one image transformation operation on the pedestrian image to obtain a target pedestrian image; Extracting features of the target pedestrian image to obtain corresponding pedestrian image features; The image transformation operation includes at least one of rotation, flipping, scaling, cropping, brightness adjustment, contrast adjustment, color jittering and normalization.

3. The method according to claim 1, characterized in that The estimating the probability that each of the pseudo labels is incorrectly labeled according to the camera category includes: Obtaining a camera hybrid perception model corresponding to the camera category; The camera category is input into the camera mixed perception model to estimate the probability that each of the pseudo labels is incorrectly labeled.

4. The method according to claim 3, characterized in that The step of inputting the camera category into the camera mixed perception model to estimate the probability that each of the pseudo labels is incorrectly labeled includes: The probability that the target pedestrian image is mislabeled is calculated by the following formula: Among them, ω c represents the c-th camera category, λ k Represents the kth distribution corresponding to the camera.

5. The method according to claim 4, characterized in that The step of inputting the camera category into the camera mixed perception model to estimate the probability that each of the pseudo labels is incorrectly labeled includes: The EM algorithm is used to fit the camera mixed perception model, including: In step E, the hidden variable is introduced Represents the posterior probability, and the latent variable formula is rewritten using the Bayesian formula as follows: Get Estimating distribution parameters using the M-step Its mean and variance are calculated as follows: Then, update ω c and λ k As shown below: Repeat the above steps E and M until convergence or the maximum number of iterations is reached; Finally, the probability that each pseudo-label is incorrectly labeled is calculated by the following formula: Among them, the Characterizes the probability that the pseudo-label is incorrectly labeled.

6. The method according to claim 1 or 5, characterized in that: The dividing the pseudo label set into a confidence data set and a potential noise data set based on the probability comprises: Taking a pseudo label with a probability greater than or equal to a preset threshold as a first pseudo label that may contain noise, and constructing a corresponding potential noise data set based on the first pseudo label; The pseudo-label with a probability less than the preset threshold is taken as a credible second pseudo-label, and a corresponding confidence data set is constructed based on the second pseudo-label.

7. The method according to claim 1, characterized in that The performing model training according to the confidence data set and the potential noise data set to obtain a pedestrian re-identification model includes: Acquire a neural network encoder corresponding to the confidence data set and the potential noise data set, wherein the neural network encoder comprises at least a first neural network encoder and a second neural network encoder; Inputting the confidence data set and the potential noise data set into the first neural network encoder for processing to obtain a corresponding first category probability distribution; Inputting the confidence data set and the potential noise data set into the second neural network encoder for processing to obtain a corresponding second category probability distribution; Model training is performed according to the first category probability distribution and the second category probability distribution to obtain a pedestrian re-identification model.

8. The method according to claim 7, characterized in that Will represents the confidence data set, represents a potential noise data set, and the first neural network encoder is recorded as The second neural network encoder is denoted as The category probability distribution predicted by the first neural network encoder is recorded as p(·), the category probability distribution predicted by the second neural network encoder is recorded as q(·), the category probability distribution predicted by the corresponding exponential moving average model in the first neural network encoder is recorded as p′(·), and the category probability distribution predicted by the corresponding exponential moving average model in the second neural network encoder is recorded as q′(·), and the model training is performed according to the first category probability distribution and the second category probability distribution to obtain a pedestrian re-identification model, including: The probability of the pseudo-label being incorrectly labeled is corrected by the following formula: Said is the average predicted probability, and then the average predicted probability is sharpened to obtain the final probability: Among them, the is the average predicted probability The function of sharpening is, T represents the temperature coefficient, Get the following loss function: The model is trained by using the potential noise data set, the confidence data set, the first target probability corrected by each first pseudo-label in the potential noise data set, and the second target probability corrected by each second pseudo-label in the confidence data set, and iterating according to the loss function until the loss function meets a preset condition, stopping the iteration, and obtaining a pedestrian re-identification model; Among them, the For the The maximum value in the predicted probability distribution; r is a preset threshold value, which is used to determine whether the maximum value of the predicted probability exceeds the preset threshold value; is the cross entropy loss function, which is used to measure the predicted probability distribution With the true label The difference between; the M is the total number of camera categories; the For the model The predicted probability distribution of .

9. A training device for a pedestrian re-identification model, characterized in that: include: A data acquisition module, used to acquire an unlabeled pedestrian data set and a camera category corresponding to the pedestrian data set; An image transformation module, used to perform at least one image transformation operation on the pedestrian data set, and perform feature extraction on the transformed pedestrian data set to obtain pedestrian image features corresponding to the pedestrian data set; A clustering module, used for clustering pedestrian images in the pedestrian data set according to the pedestrian image features to obtain a pseudo label set corresponding to the pedestrian data set, wherein the pseudo label set includes pseudo labels corresponding to the pedestrian images; a label modeling module, configured to estimate the probability that each of the pseudo labels is incorrectly labeled according to the camera category, and divide the pseudo label set into a confidence data set and a potential noise data set based on the probability, wherein the confidence data set contains reliable pseudo labels and the potential noise data set contains pseudo labels that may contain noise; A model training module is used to perform model training based on the confidence data set and the potential noise data set to obtain a pedestrian re-identification model, wherein the pedestrian re-identification model is used to identify and match the same pedestrian in surveillance videos from different cameras, at different times or at different locations.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 7 when executing the program stored in the memory.

11. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 7.

Images