A lifelong pedestrian re-identification method based on bridging network knowledge review

Abstract

The application provides a lifelong pedestrian re-identification method based on bridging network knowledge review, comprising the following steps: training a pedestrian re-identification model with a data set; copying the trained model parameters into another pedestrian re-identification model; building a bridging network between the current model and the old model; obtaining the current model prediction value; calculating the overall loss according to the prediction value, and propagating the overall loss to the current model to update the parameters of the current model; repeating the above steps to continuously iterate the pedestrian re-identification model to obtain the final pedestrian re-identification model. The application connects the current model and the old model through the bridging network, and constructs a mixed feature map according to the weight factor generated by the bridging network. The knowledge transfer of the old model can effectively increase the review of the old knowledge by the current model, and more effectively transfer the knowledge of the old model to the current model, and enhance the anti-forgetting property of the current model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and computer vision, in particular to a lifelong person re-identification method based on bridging network knowledge review. BACKGROUND

[0002] Person re-identification (Re-ID) is a technology that judges whether a specific person exists in images or videos from non-overlapping cameras through a deep learning model, which has attracted widespread attention due to its wide application in monitoring and public security. In recent years, person re-identification technology has achieved good performance in supervised or unsupervised scenarios under the efforts of scholars. However, most of the current training methods assume that all training data can be prepared in advance for model learning, which is not suitable for the deployment of Re-ID in real situations. In real situations, Re-ID data is essentially captured from different locations or cameras over time, so studying how to better implement the person re-identification task in the face of increasing data flow has become an important research direction in the field.

[0003] Data flow is a collection of images collected from different locations or cameras, and their data distribution usually has obvious differences. When continuously training the re-identification model on the incremental data sequence over time, the model will often adapt to the current data distribution, thereby affecting the previously learned knowledge, causing the model to suffer catastrophic forgetting. In order to better deploy person re-identification in the real world, how to train a person re-identification model with good anti-forgetting ability is a problem that needs to be solved at present. Therefore, recently scholars have devoted to studying lifelong person re-identification (LRe-ID), which aims to use past and present learned knowledge to build a person re-identification model that can adapt to new data and maintain stable performance on old data.

[0004] In order to overcome the influence of catastrophic forgetting on the generalization performance of the model, recent research methods focus on using the storage of the old model to retain past learned knowledge, that is, freezing the model trained for the last task and storing it as an old model. When the model expands to learn new tasks, the consistency of the outputs of the current model and the old model is constrained to strengthen the review of the old knowledge by the model. However, this direct consistency constraint often affects the generalization ability of the model to new tasks, especially when the data distribution difference between the new task and the old task is particularly large. If not corrected, this reduction in current task generalization ability will lead to misleading knowledge distillation during the next task training. After several iterations of task training, the model will eventually be biased towards the data distribution of the first task, but not suitable for the data distribution of other new tasks, which seriously reduces the overall generalization performance of the model. SUMMARY

[0005] The purpose of the present application is to provide a lifelong pedestrian re-identification method based on bridging network knowledge review, to solve the problem of poor model generalization performance.

[0006] The present application is implemented as follows: a lifelong pedestrian re-identification method based on bridging network knowledge review, comprising the following steps:

[0007] A, select the data set of t tasks, first train the pedestrian re-identification model with the data set of the first task t=1, and construct a sample playback device at the same time;

[0008] B, copy the trained model parameters to another pedestrian re-identification model, and mark the pedestrian re-identification model as an old model, and the pedestrian re-identification model being trained as a current model; combine the current model and the old model to jointly train the data set of the remaining re-identification task (t>1), and the sample playback device is mixed with the data set of the new task and input into the pedestrian re-identification model for training in the subsequent task training;

[0009] C, build a bridging network between the current model and the old model; the bridging network comprises a pooling layer, an adaptive full connection layer and a multi-layer perception machine, the bridging network extracts new deep feature maps G n and old deep feature maps G o as inputs, respectively, to generate new fusion weight factors α n and old fusion weight factors α o ;

[0010] D, dynamically generate the mixed feature map G inter of the bridging network through the new fusion weight factor α n and the old fusion weight factor α o , input the mixed feature map G inter and the new deep feature map G n into the current pedestrian re-identification model to obtain the bridging network prediction value P ni and the current model prediction value P n , input the mixed feature map G inter and the old deep feature map G o into the old model to obtain the bridging network prediction value P oi and the old model prediction value P o ;

[0011] E, according to the current model prediction value P n , the old model prediction value P o , the bridging prediction value P ni and the prediction value P oi , calculate the weighted cross-entropy loss L inter , the cross-entropy loss L ce , and the style fusion consistency loss Lkd and the weight factor diversity loss L div , according to the weighted cross-entropy loss L inter , the cross-entropy loss L ce , the style fusion consistency loss L kd and the weight factor diversity loss L div , the overall loss L is calculated , and the overall loss L is back-propagated to the current model to update the parameters of the current model;

[0012] F, the pedestrian re-identification model is repeatedly trained after mixing the data set of the next task and the sample playback constructed by the previous task, that is, steps B-E are repeated to iteratively train the pedestrian re-identification model until the final pedestrian re-identification model is obtained after training the data set of the last task.

[0013] Further, the present application can be implemented according to the following technical solutions: in the step A, when training the data set of the first task, t = 1, t represents the task number, a given data set D t is sampled according to the identity to obtain a batch of pictures , wherein x i represents the i-th picture, and N = p x k represents the number of a batch of pictures, p and k represent the number of persons and the number of pictures sampled per identity respectively; according to the input of the pictures, the pedestrian re-identification model extracts the predicted value , which is represented as:

[0014]

[0015] , wherein θ n represents the current model, and here the current model = θ n , t is the task number, represents the predicted value of the i-th picture by the current model;

[0016] According to P i , the cross-entropy loss L ce is used to guide the iterative training of the model, which is represented as:

[0017]

[0018] , wherein y i is the true label of the i-th picture.

[0019] In the step A, the samples of the data set are randomly sampled according to the identity and stored in the sample playback, and the sample playback contains N r = C o x K samples, wherein C o ​K is the total number of identities for the previous task, and Ks is the number of randomly sampled samples for each identity.

[0020] In the step B, the parameters of the pedestrian re-identification model θ n in the current training are copied to another pedestrian re-identification model, and marked as the old model θ o , and the pedestrian re-identification model in the training is marked as the current model θ n ; when training the current task, the parameters of the old model θ o will be frozen and not participate in the update of back propagation, expressed as:

[0021] θ o ← θ n

[0022] Wherein, ← represents the transfer of model parameters.

[0023] In the step C, the bridge network includes two pooling layers, an adaptive fully connected layer and a multi-layer perceptron, which are inserted into the hidden layers of the backbone network of the current model and the old model respectively, and the current model θn is split into sub-models and sub-models The old model θ o is split into sub-models and sub-models The formula is expressed as:

[0024]

[0025] Wherein, m represents the position of the inserted hidden layer, k∈{n,o} represents the number of models, n represents the current model, and o represents the old model; represents the first m hidden layers of the deep model, represents the structure after the mth hidden layer of the deep model.

[0026] In the step C, given the input picture x k ∈R h×w×c , the bridge network will extract deep feature maps as input through m-stage convolutional blocks, that is:

[0027]

[0028] Wherein, k∈{n,o} represents the model number, the deep feature map G n extracted by the m-stage convolutional block, G o ∈R h×w×c represents the feature representation of the current model and the old model respectively, which contains rich pedestrian features; the bridge network first extracts new deep feature maps G ntwo one-dimensional feature representations of the current model are obtained after concatenating the one-dimensional feature representations extracted by the max-pooling layer and the average-pooling layer

[0029] Then, the old deep feature map G o two one-dimensional feature representations of the old model are obtained after concatenating the one-dimensional feature representations extracted by the max-pooling layer and the average-pooling layer

[0030] is represented as:

[0031]

[0032]

[0033]

[0034] wherein, respectively represent the feature maps after max-pooling and average-pooling, and the one-dimensional feature representations after the pooling layer are transmitted to the adaptive fully connected layer first, and the key features representing the model style are adaptively extracted, and are represented as:

[0035]

[0036] wherein, Cat represents a vector concatenation operation, and F k is the key feature representing the model style;

[0037] After the concatenated pooling features are transmitted to the adaptive fully connected layer to adaptively extract the key features, finally, the key features of the current model and the old model are added and transmitted to the multi-layer perceptron, and the old fusion weight factor α o and the new fusion weight factor α n are obtained by the multi-layer perceptron, and are formally represented as:

[0038] α n , α o = softmax(MLP(F n +F o ))

[0039] wherein, α n , α o respectively represent the new fusion weight factor of the current model and the old fusion weight factor of the old model, softmax represents a normalization operation on the weight factor by using a softmax function, and MLP represents a multi-layer perceptron, which is composed of multiple fully connected layers and activation functions.

[0040] In the step D, the mixed feature map G interBy new fusion weight factor α n and old fusion weight factor α o Obtained by feature weighting and feature weighting method, denoted as:

[0041] G inter = α n · G n + α o · G o

[0042] Wherein, G n , G o represent new deep feature map and old deep feature map respectively;

[0043] In the step D, the mixed feature map G inter ∈R h×w×c is transmitted into the current model and the old model respectively to continue extracting features, and finally transmitted into the classifier to obtain the intermediate prediction output of the bridging network, that is:

[0044]

[0045] Wherein, k∈{n, o} is the model number, P ki ∈R bs×c represent the bridging prediction value extracted by the current model and the old model respectively, and c is the total number of line human categories;

[0046] Then, the new deep feature map G n and the old deep feature map G o are directly transmitted into the current model and the old model respectively to obtain the current model prediction value P ni and the old model prediction value P oi , which can be represented as:

[0047]

[0048] Wherein, k∈{n, o} is the model number, P k ∈R bs×c represent the prediction value of the current model and the prediction value of the old model respectively.

[0049] In the step E, the weighted cross entropy loss L inter is calculated based on the intermediate interpolation, which is represented as:

[0050]

[0051] Wherein, y represents the true label of the pedestrian sample, and α n is the model style weight factor;

[0052] According to P n , the cross entropy loss Lce The guiding model iterative training is represented as:

[0053]

[0054] Wherein, y i is the real label of the ith picture.

[0055] The style fusion consistency loss L kd is represented as:

[0056] L kd = D KL (P n ||P ni )+D KL (P oi ||P o )

[0057] Wherein, D KL (·||·) represents the Kullback-Leible (KL) divergence;

[0058] The weight factor diversity loss L div is represented as:

[0059]

[0060] Wherein, bs represents the number of samples in a batch, and σ(·) represents the standard deviation of all weight factors in a batch,

[0061] In the training process, the overall loss L used for optimization is a combination of the weighted cross-entropy loss L inter , the cross-entropy loss L ce , the style fusion consistency loss L kd and the weight factor diversity loss L div , and is represented as:

[0062]

[0063] Wherein, λ1 and λ2 are the weighting coefficients of different loss terms.

[0064] In the step F, first, after mixing the data set of the second task and the sample playback constructed by the first task, the pedestrian re-identification model is repeatedly trained, that is, steps B-E are repeated to iteratively train the pedestrian re-identification model, and then after mixing the data set of the third task and the sample playback constructed by the second task, the pedestrian re-identification model is repeatedly trained, that is, steps B-E are repeated to iteratively train the pedestrian re-identification model, until the last task data set is trained to obtain the final pedestrian re-identification model.

[0065] The application bridges the current model and the old model through the bridging network, and constructs a mixed feature map according to the weight factor generated by the bridging network. The mixed feature map fuses the styles of the current model and the old model, and balances the anti-forgetting ability and generalization ability of the model. By indirectly constraining the output consistency of the current model and the mixed feature map, the knowledge of the old model is transferred to the current model in a smoother way, reducing the influence of direct consistency constraint on the generalization performance of the current model. Through the knowledge transfer of the old model, the review of the old knowledge by the current model can be effectively increased, so that the knowledge of the old model can be more effectively transferred to the current model, and the anti-forgetting ability of the current model is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 is a model training framework diagram of the application.

[0067] Figure 2 is a structural diagram of the bridging network. DETAILED DESCRIPTION

[0068] As shown in Figure 1 , the lifelong person re-identification method based on bridging network knowledge review of the application includes the following steps:

[0069] A, select the data set of t tasks, first train the person re-identification model with the data set of the first task t=1, and construct a sample playback device, which is mixed with the data set of the new task and transmitted into the person re-identification model for training in the subsequent task training.

[0070] When training the data set of the first task, t=1, t represents the task number, and the optimization goal of the model is to enhance the discrimination ability of the person picture given the data set D t , a batch of pictures are sampled according to the identity , which are transmitted into the person re-identification model, wherein x i represents the i-th picture, and N=p x k represents the number of a batch of pictures, and p and k represent the number of batches and the number of pictures sampled per identity respectively; according to the input of the picture, the person re-identification model extracts the prediction value , which is represented as:

[0071]

[0072] , wherein θ n represents the current model, and here the current model = θn, t is the task number, represents the prediction value of the current model for the i-th picture;

[0073] In order to emphasize the discrimination ability of the current model for the person pictures in the data set of the current task, the cross entropy loss L i is adoptedce The guiding model iterative training is represented as:

[0074]

[0075] wherein y i is the real label of the ith picture; the loss is constantly updated by back propagation according to the preset number of iterations, and finally the pedestrian re-identification model trained for the first task is obtained.

[0076] In order to reduce the forgetting of the model to the knowledge of the previous task, the samples of the dataset are randomly sampled according to the identity and stored in the sample playback device. The sample playback device contains N r =C o x K samples, wherein C o is the total number of identities of the previous task, and K is the number of randomly sampled samples for each identity.

[0077] The sample playback device is mixed with the dataset of the new task and transmitted into the model training in the subsequent task training. When t>1, the generalization ability and the knowledge review ability of the model for the learned task need to be considered. When training task t, the sample playback device is constructed according to task t-1, and then the sample playback device and the dataset of task t are mixed and transmitted into the model training.

[0078] B, the trained model parameters are copied to another pedestrian re-identification model, and the pedestrian re-identification model is marked as an old model, and the pedestrian re-identification model being trained is marked as a current model. The dataset of the remaining re-identification tasks (t>1) is trained in combination with the current model and the old model. The sample playback device is mixed with the dataset of the new task and transmitted into the pedestrian re-identification model for training in the subsequent task training.

[0079] In order to increase the review of the model to the learned task when training the current task, after the model completes the training of the previous task, the parameters of the pedestrian re-identification model θ n being trained are copied to another pedestrian re-identification model and marked as an old model θ o , and the pedestrian re-identification model being trained is marked as a current model θ n ; when training the current task, the parameters of the old model θ o will be frozen and not participate in the update of the back propagation, which is represented as:

[0080] θ o ← θ n

[0081] wherein <- represents the transmission of the model parameters. Since the parameters of the old model θ o are not updated when training the current task, the old model θ oThe knowledge of the previous task is included for the current model θ n When t>2, the pedestrian re-identification model parameters after the end of task t-1 training are copied to another pedestrian re-identification model, and the copied pedestrian re-identification model is marked as an old model. That is, when task t is trained, the pedestrian re-identification model after the end of task t-1 training is an old model.

[0082] C, a bridge network (also referred to as a bridge model) is built between the current model and the old model, as shown in Figure 2 The bridge network includes a pooling layer, an adaptive fully connected layer and a multi-layer perceptron, the bridge network extracts new deep feature maps G n and old deep feature maps G o from the current model and the old model respectively as inputs, and generates new fusion weight factors α n and old fusion weight factors α o respectively.

[0083] The bridge network includes two pooling layers, an adaptive fully connected layer and a multi-layer perceptron, the bridge network is inserted into the hidden layer of the backbone network of the current model and the old model respectively, and the current model θ n is split into a sub-model and a sub-model The old model θ o is split into a sub-model and a sub-model The pedestrian re-identification model includes a backbone network and a classifier, and the backbone network is used to extract features. The formula is represented as:

[0084]

[0085] Wherein, m represents the position of the inserted hidden layer, k∈{n, o} represents the number of the model, n represents the current model, and o represents the old model; represents the first m hidden layers of the deep model, represents the structure after the mth hidden layer of the deep model.

[0086] In order to enhance the generalization ability of the current model to the learned task, it is inevitable to increase the review of the learned task knowledge in the process of training the current model. The old model contains rich knowledge of the last task, and the knowledge transfer of the old model can effectively increase the review of the old knowledge by the current model. However, the review process is not easy to control, and too much attention to the review of the old knowledge in the training process of the current model will directly affect the generalization ability of the model to the new task, but the lack of review of the old knowledge will directly lead to the catastrophic forgetting of the model. Therefore, the application proposes a bridge network to connect two models, find a proper 'path' between them, so as to more effectively transfer the knowledge of the old model to the current model, and enhance the anti-forgetting of the current model. According to the hidden layer inserted by the bridge network in the backbone network, the model theta k can be divided into two sub-models and Wherein k represents the number of models in {n, p}, theta o represents the old model, and theta n represents the current model. Given the input picture x k ∈R h×w×c , the bridge network will extract the deep feature map as the input through m-stage convolution block, that is:

[0087]

[0088] Wherein k represents the number of models in {n, o}, the deep feature map G n extracted by m-stage convolution block, and G o ∈R h×w×c represent the feature representation of the current model and the old model respectively, which contains rich pedestrian features;

[0089] The bridge network first extracts one-dimensional feature representation of the new deep feature map G n through the maximum pooling layer and the average pooling layer in the pooling layer respectively, and then splices the one-dimensional feature representation extracted by the maximum pooling layer and the average pooling layer to obtain two one-dimensional feature representations of the current model

[0090] Then, the one-dimensional feature representation of the old deep feature map G o is extracted through the maximum pooling layer and the average pooling layer in the pooling layer respectively, and the one-dimensional feature representation extracted by the maximum pooling layer and the average pooling layer is spliced to obtain two one-dimensional feature representations of the old model The detailed extraction process is as follows:

[0091]

[0092]

[0093] wherein, respectively represent the feature maps after max-pooling and average-pooling, although the pooling layer can quickly compress the dimension to facilitate calculation, it will take away many subtle features of the deep feature maps. Therefore, the one-dimensional features after the pooling layer are transmitted to the adaptive fully connected layer, which adaptively extracts key features representing the model style, denoted as:

[0094]

[0095] wherein, Cat represents the vector splicing operation, F k is the key feature representing the model style;

[0096] After splicing the pooling features and transmitting them into the adaptive fully connected layer to adaptively extract the key features, the key features of the current model and the old model are finally added and transmitted into the multi-layer perceptron, and the old fusion weight factor a o and the new fusion weight factor a n are obtained by the multi-layer perceptron, which is formally represented as:

[0097] a n , a o = softmax(MLP(F n +F o ))

[0098] wherein, a n , a o represent the new fusion weight factor of the current model and the old fusion weight factor of the old model respectively, softmax represents the normalization operation of the weight factor using the softmax function, and MLP represents the multi-layer perceptron, which is composed of multiple fully connected layers and activation functions.

[0099] D. dynamically generating the hybrid feature map G inter of the bridge network through the new fusion weight factor a n and the old fusion weight factor a o , transmitting the hybrid feature map G inter and the new deep feature map G n into the current pedestrian re-identification model to obtain the bridge network prediction value P ni and the current model prediction value P n , transmitting the hybrid feature map G inter and the old deep feature map G o into the old model to obtain the bridge network prediction value P oi and the old model prediction value P o , obtaining the prediction value for calculating the loss, updating the parameters of the model by backpropagation, and finally obtaining the optimal lifelong pedestrian re-identification model.

[0100] The feature maps are mixed by style weight factors. In order to transfer the knowledge of the old model to the current model more smoothly, the mixed feature maps are constructed according to the weight factors generated by the bridge network. The mixed feature maps fuse the styles of the current model and the old model, and balance the anti-forgetting ability and the generalization ability of the model. By indirectly constraining the consistency of the output of the current model and the output of the mixed feature maps, the knowledge of the old model will be transferred to the current model in a more smooth way, and the influence of the direct consistency constraint on the generalization performance of the current model is reduced.

[0101] The mixed feature map G inter is obtained by the new fusion weight factor a n and the old fusion weight factor a o is obtained by feature weighting and feature weighting method, and is expressed as:

[0102] G inter = a n .G n + a o .G o

[0103] wherein G n and G o represent the new deep feature map and the old deep feature map respectively;

[0104] The mixed feature map G inter ∈ R h×w×c is transmitted into the current model and the old model respectively to continue to extract features, and finally transmitted into the classifier to obtain the intermediate prediction value output of the bridge network, that is:

[0105]

[0106] wherein k∈{n,o} is the model number, P ki ∈ R bs×c represent the bridge prediction values extracted by the current model and the old model respectively, and c is the total number of human categories;

[0107] Then, the new deep feature map G n and the old deep feature map G o are directly transmitted into the current model and the old model respectively to obtain the current model prediction value P n and the old model prediction value P o , which can be expressed as:

[0108]

[0109] wherein k∈{n,o} is the model number, P k ∈ R bs×c represent the prediction value of the current model and the prediction value of the old model respectively.

[0110] E. Based on the current model prediction value P n The old model predicted value P o and bridging prediction value P ni and predicted value P oi Calculate the weighted cross-entropy loss L inter Cross-entropy loss L ce Style fusion consistency loss L kd and weight factor diversity loss L div According to the weighted cross-entropy loss L inter Cross-entropy loss L ce Style fusion consistency loss L kd and weight factor diversity loss L div Calculate the overall loss and overall loss Backpropagation is performed to the current model to update its parameters;

[0111] Weighted cross-entropy loss L inter The weighted cross-entropy loss based on intermediate interpolation is expressed as:

[0112]

[0113] Where y represents the true label of the pedestrian sample, α n L represents the model style weighting factor and the weighted cross-entropy loss. inter It is emphasized that the intermediate feature predictions after style fusion can also accurately distinguish pedestrians. Since the bridging network shares the same optimizer as the current model, according to L... inter The fully connected layers of the bridging network also train appropriate style weight factors in a self-learning manner, thereby ensuring that suitable intermediate features can be found.

[0114] According to P n Using cross-entropy loss L ce Guiding model training to enhance its generalization ability for the current task is represented as:

[0115]

[0116] Among them, y i Let be the true label of the i-th image.

[0117] To leverage intermediate features to enhance the current model's recall of knowledge from older models, style fusion consistency loss L... kd , is represented as:

[0118] L kd =D KL (P n ||P ni )+D KL (Poi ||P o )

[0119] wherein, D KL represents the Kullback-Leible (KL) divergence; through this indirect prediction consistency constraint, the knowledge of the old model can be more smoothly transferred to the current model without affecting the knowledge accumulation of the current model for the current task.

[0120] In order to avoid the weight factor tending to a certain model in the process of training (such as {0, 1} occurs), the weight factor diversity loss L div is expressed as:

[0121]

[0122] wherein, bs represents the number of samples in a batch, and σ(·) represents the standard deviation of all weight factors in a batch, by minimizing L div , the bridging network can be forced to be as diverse as possible to model the intermediate features of the optimal "path".

[0123] In the training process, the overall loss L for optimization is a combination of the weighted cross-entropy loss L inter , the cross-entropy loss L ce , the style fusion consistency loss L kd and the weight factor diversity loss L div , and is expressed as:

[0124]

[0125] wherein, λ1 and λ2 are weighting coefficients of different loss terms, and the overall loss will be back-propagated and update the parameters of the model at each iteration.

[0126] F, the data set of the next task and the sample playback constructed by the previous task are mixed, and the pedestrian re-identification model is repeatedly trained, that is, steps B-E are repeated to iteratively train the pedestrian re-identification model, until the last task data set is trained to obtain the final pedestrian re-identification model.

[0127] First, the data set of the second task and the sample playback constructed by the first task are mixed, and the pedestrian re-identification model is repeatedly trained, that is, steps B-E are repeated to iteratively train the pedestrian re-identification model, and then the data set of the third task and the sample playback constructed by the second task are mixed, and the pedestrian re-identification model is repeatedly trained, that is, steps B-E are repeated to iteratively train the pedestrian re-identification model, until the last task data set is trained to obtain the final pedestrian re-identification model.

[0128] The network is trained according to the designed framework, the maximum number of training rounds and the number of iterations for each round of training. The relevant parameters are set as follows: the maximum number of training rounds for each data set is 60, the pictures of each data set are iterated once for each round, the learning rate is 0.00035, and the learning rate is decayed after every 20 rounds with a decay coefficient of 0.01. For each iteration, 4 images are randomly selected from 16 pedestrians for training. After adjusting the training images to 256x128 size, they are fed into the network with random erasing data augmentation, and finally into the network for training. We set λ1 to 0.5 and λ2 to 0.2 in the overall loss function. According to step A, the pedestrian re-identification task is continuously input for training. In order to train a lifelong pedestrian re-identification model with anti-forgetting, each data set is sequentially input into the model for training as a task. Except for the first data set, which does not consider knowledge review, the other data sets are trained while considering knowledge review according to the output of the old model as a reference. When the last task is trained, the trained task is input as the final model to test the generalization performance of the learned task.

Claims

1. A life-long person re-identification method based on bridging network knowledge review, characterized in that, Comprising the following steps: A, select A data set of the first task, first train the pedestrian re-identification model with the data set of the first task =1, and construct a sample playback device; B. Copy the trained model parameters to another pedestrian re-identification model, and mark the pedestrian re-identification model as an old model, and the pedestrian re-identification model being trained as a current model; combine the current model and the old model to jointly train the data set of the remaining re-identification task, and the sample playback device is mixed with the data set of the new task and input into the pedestrian re-identification model for training in subsequent task training; C. a bridge network is built between the current model and the old model; the bridge network includes a pooling layer, an adaptive full connection layer and a multi-layer perception, and the bridge network extracts a new deep feature map from the current model and an old deep feature map from the old model respectively as inputs, to generate a new fusion weight factor and an old fusion weight factor respectively D. by a new fusion weight factor and an old fusion weight factor dynamically generating a hybrid feature map of the bridge network , the hybrid feature map and the new deep feature map into the current pedestrian re-identification model to obtain a current bridge network prediction value and a current model prediction value , the hybrid feature map and the old deep feature map into the old model to obtain an old bridge network prediction value and an old model prediction value ; E、the current model prediction value , the old model prediction value , and the current bridge network prediction value , and the old bridge network prediction value , the weighted cross-entropy loss , the cross-entropy loss , the style fusion consistency loss , and the weight factor diversity loss , the overall loss , the cross-entropy loss , the style fusion consistency loss , and the weight factor diversity loss , and the overall loss is calculated by weighted summation, and the overall loss is back-propagated to the current model to update the parameters of the current model. F. After mixing the data set of the next task and the sample playback device constructed by the last task, the pedestrian re-identification model is repeatedly trained, that is, steps B-E are repeated to iteratively train the pedestrian re-identification model, until the last task data set is trained to obtain the final pedestrian re-identification model. In the step C, the bridge network includes two pooling layers, one adaptive fully connected layer and a multi-layer perception, the bridge network is respectively inserted into the hidden layers of the backbone network of the current model and the old model, and the current model is respectively split into the sub-model and the sub-model , the old model is split into the sub-model and the sub-model ; the formula is as follows: wherein, represents a position of an inserted hidden layer, represents a number of a model, represents a current model, represents an old model; represents a structure of a front hidden layer of a deep model, represents a structure after a hidden layer of a deep model.

2. The method of claim 1, wherein the bridge network knowledge review-based lifelong person re-identification method is characterized by In the step A, the data set of the first task is trained , representing the task sequence number, the given data set , a batch of pictures is sampled according to the identity The pedestrian re-identification model is input, wherein, represents the first picture, represents the number of a batch of pictures, and respectively represent the number of identities and the pictures sampled for each identity; according to the input of the pictures, the pedestrian re-identification model extracts the prediction value through convolution and pooling operation , which is represented as: wherein, represents the current model, represents the prediction value of the current model for the picture; According to Using cross-entropy loss The guiding model is iteratively trained, denoted as: wherein, is the true label of the image. 3.The lifelong person re-identification method based on bridging network knowledge review according to claim 1, characterized in that, in In the step A, the samples of the dataset are randomly sampled according to the identities and stored in a sample replay buffer, which contains samples, where is the total number of identities of the previous task, is the number of samples randomly sampled for each identity. 4.The life-long person re-identification method based on bridging network knowledge review according to claim 1, characterized in that, in In step B, the currently trained pedestrian re-identification model will be... The parameters are copied into another pedestrian re-identification model and labeled as the old model. The pedestrian re-identification model currently being trained is marked as the current model. When training the current task, the old model The parameters will be frozen and will not participate in the backpropagation update, represented as: wherein passing of the representative model parameters.

5. The method of claim 2, wherein the bridge network knowledge review-based lifelong person re-identification method is characterized by In step C, the given input picture , is the total number of line classes; the bridging network will pass through the deep feature map extracted by the convolutional block in the stage as input, i.e.: wherein represents a model number, and the model is trained by a deep feature map extracted by a stage convolutional block respectively represent the feature representations of the current model and the old model, and contain rich pedestrian features; the bridging network first extracts one-dimensional feature representations of a new deep feature map through a maximum pooling layer and an average pooling layer in a pooling layer , then splices the one-dimensional feature representations extracted by the maximum pooling layer and the average pooling layer to obtain two one-dimensional feature representations of the current model ; Then, one-dimensional feature representations are extracted by a maximum pooling layer and an average pooling layer in the pooling layer respectively from the old deep feature map , Indicated as: wherein represents a model number, respectively represent the feature maps after max-pooling and average-pooling, and the one-dimensional features after the pooling layer are transmitted to an adaptive fully connected layer, which adaptively extracts key features representing the model style, and is expressed as: wherein, represents a vector concatenation operation, is a key feature representing the model style; After the spliced and pooled features are adaptively extracted by the adaptive fully connected layer, the key features of the current model and the old model are added and transmitted into the multi-layer perceptron, and the old fusion weight factor and the new fusion weight factor are obtained by the multi-layer perceptron and the new fusion weight factor , which is formally represented as: wherein, respectively represent a new fusion weight factor of a current model and an old fusion weight factor of an old model, represent a normalization operation on the weight factor using a function, MLP represents a multi-layer perception composed of a plurality of fully connected layers and an activation function.

6. The method of claim 5, wherein the bridge network knowledge review-based lifelong person re-identification method is characterized by In step D, the mixed feature map by a new fusion weight factor and an old fusion weight factor obtained by feature weighting and feature weighting manner, denoted as: wherein respectively represent a new deep feature map and an old deep feature map.

7. The method of claim 6, wherein the bridge network knowledge review-based lifelong person re-identification method is characterized by The step D in the present application, The current model and the old model are respectively transmitted to continue extracting features, and finally transmitted to the classifier to obtain the intermediate prediction value output of the bridging network, that is: wherein, is a model number, respectively represent the bridging predicted values extracted by the current model, the old model, is the total number of line classes; Then, directly passing the new deep feature map and the old deep feature map into the current model and the old model respectively will get the current model prediction and the old model prediction , which can be represented as: wherein, is a model number, respectively represent a prediction value of a current model and a prediction value of an old model; represents a sample quantity within a batch.

8. The method of claim 7, wherein the bridge network knowledge review-based lifelong person re-identification method is characterized by In step E, the weighted cross-entropy loss The weighted cross-entropy loss is calculated based on the intermediate interpolation, denoted as: wherein, represents the true label of the pedestrian sample, is a model style weight factor; According to Using cross-entropy loss The guiding model is iteratively trained, denoted as: wherein, is the true label of the image; Style fusion consistency loss is represented as: wherein, represents the Kullback-Leible (KL) divergence; Weight factor diversity loss is represented as: wherein, represents the number of samples within a batch, represents the standard deviation of all weight factors within a batch, During training, the overall loss is used for optimization. It is weighted cross-entropy loss Cross-entropy loss Loss of style integration and consistency and weight factor diversity loss The combination of is represented as: wherein , are weighting coefficients for different loss terms.

9. The method of claim 1, wherein the bridge network knowledge review-based lifelong person re-identification method is characterized by In the step F, first, the data set of the second task and the sample playback device constructed by the first task are mixed, and the pedestrian re-identification model is repeatedly trained, that is, steps B-E are repeated to iteratively train the pedestrian re-identification model, then the data set of the third task and the sample playback device constructed by the second task are mixed, and the pedestrian re-identification model is repeatedly trained, that is, steps B-E are repeated to iteratively train the pedestrian re-identification model, until the last task data set is trained to obtain the final pedestrian re-identification model.

Images