Lifelong learning person re-identification method based on knowledge updating and knowledge integration

Abstract

The present invention discloses a lifelong learning pedestrian re-identification method and device based on knowledge updating and knowledge integration, which specifically divides the task learning process in lifelong learning into three stages: knowledge replay, knowledge updating and knowledge integration. Knowledge replay receives local task data and buffer task data, and maintains the working model with the help of pseudo labels generated by the memory model. At the same time, the response generated by the working model is passed to the memory model for self-update, thereby generating higher-quality pseudo labels. These two modes alternately update the model parameters to achieve two-way information transmission. After the task training is completed, the knowledge integration stage is entered. At the model level, the present invention designs a sliding average method to achieve the integration of working and memory models to facilitate the deployment training in the next stage. In the testing stage, the present invention provides a feature space integration to improve the generalization of prediction results in visible and invisible domains.

Description

Technical Field

[0001] The present invention relates to a pedestrian re-identification method for lifelong learning, and belongs to the fields of deep learning and computer vision. Background Art

[0002] Deep learning has been widely applied in artificial intelligence applications, profoundly empowering fields such as computer vision. With the growing development and maturity of surveillance and security, smart cities, and autonomous driving, person re-identification (PIR) has become a key technology. PIR aims to identify pedestrians with the same identity from a group of pedestrians captured by different cameras. Pedestrian images can contain significant disturbances (such as occlusion, pose changes, and camera perspective), which poses a major challenge to PIR.

[0003] The current pedestrian re-identification task dominated by deep learning follows the assumption of independent and identically distributed training sets during training. However, due to the needs of real-world applications and the protection of dataset privacy, pedestrian re-identification for lifelong learning overcomes the limitations of the original settings and can sustainably acquire knowledge from datasets with different distributions, ultimately learning a pedestrian re-identification model with anti-forgetting and strong generalization capabilities.

[0004] Evaluating the learning capabilities of lifelong learning models is generally done from two perspectives: forward transfer and backward transfer. Existing lifelong learning methods use techniques such as distillation, replay, and parameter regularization to reduce negative backward transfer, thereby preventing catastrophic forgetting in the model. These methods have the following drawbacks: 1) While maintaining model stability, they reduce the model's plasticity, impairing performance on some new tasks. 2) They only consider backward transfer, lacking consideration of forward transfer. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: in order to achieve forward and backward migration, the main problems of lifelong learning pedestrian re-identification technology so far are: 1) Distribution mismatch: There are natural distribution differences between the gradually increasing task data, and the existing methods use direct distillation between different tasks, ignoring this distribution difference, which in turn affects the ability of forward migration and reduces the plasticity of the model; 2) Representation capability mismatch: The data sets input at different times have different scales. The existing methods distill the previous model into the current model, which may give a biased supervision signal, thereby affecting the generalization of the model.

[0006] In order to solve the above technical problems, the technical solution of the present invention discloses a lifelong learning pedestrian re-identification method and device based on knowledge updating and knowledge integration, comprising the following steps:

[0007] Step 1: At the beginning of the initial task t=0, t is the current task number, given a data set Randomly sample a batch of pictures to get N b is the batch size, is the i-th picture and is input into the working model based on convolutional neural network (CNN) to extract features and obtain the softmax prediction score q(·) is the softmax function used by the working model;

[0008] Step 2: Learn the discriminability of the working model for the pedestrian images of the current task and calculate the cross entropy loss function based on the current batch of images in is the i-th prediction score No. Item value;

[0009] Step 3: Further optimize the feature distribution in the metric space and use the triple loss function for the extracted current batch features Among them, d(·,·) represents the Euclidean distance function, m is the set threshold parameter, Represents the anchor features, positive features, and negative features of the i-th pedestrian in the current batch extracted by the working model;

[0010] Step 4: Combine with cross entropy loss function and triplet loss function Obtain the loss function L0 at stage t=0, and update the network parameters through reverse gradient propagation until the preset number of iterations is reached;

[0011] Step 5: For tasks at time t>0, the learning network incorporates both a memory model and a working model. The goal is to improve the performance of the current task and previous tasks through the joint learning of the memory model and the working model, thereby improving the overall generalization of the model.

[0012] At the beginning of task t, the model first enters the knowledge playback phase, and the input at this time is the data set at time t Buffer storage area M t It is used to store some samples and their labels seen from time 0 to t-1. The learning rates of the working model and the memory model are γ and η respectively. and M t Randomly sample a batch of samples from and They are respectively sent to the working model and the memory model, and the corresponding softmax prediction scores are output, that is, Among them, p(·) and q(·) represent the prediction scores output by the working model and memory model respectively with (·) as input;

[0013] Step 6: In order to prevent the working model from forgetting the old tasks, knowledge distillation is performed at the prediction score level to solve catastrophic forgetting. The prediction scores generated by the memory model are used as pseudo labels to guide the learning of the working model. The anti-forgetting loss function Expressed as Where T represents the distillation temperature, SD(·) represents the gradient separation operator, and JS(·) represents the Jesen-Shannon divergence;

[0014] Step 7: In order to make the model better adapt to the current task, design the corresponding adaptive loss function Among them, the adaptive loss function includes a cross entropy loss of the current task and a triplet loss for the current task

[0015]

[0016]

[0017] Represents the anchor features, positive features, and negative features of the i-th pedestrian in the current batch extracted by the working model; Represents the anchor features, positive features, and negative features of the i-th pedestrian in the current batch extracted by the memory model;

[0018] Step 8: Obtain the overall loss function of the working model in the knowledge replay phase Then the overall loss function L w Perform gradient backpropagation and update the parameters of the working model by the gradient descent algorithm That is, for Among them, θ is the parameter of the working model, including the parameters of the feature extractor and classifier. The loss function L obtained by the current batch of training samples w The calculated gradient, γ is 1;

[0019] Step 9: Knowledge update phase:

[0020] Responsible for updating the memory model, that is, using the knowledge of the working model to correct the response of the memory model to achieve smoother knowledge transfer;

[0021] Step 10: Enhance the ability of the memory model to handle the current task and calculate the memory loss function including cross entropy loss and triplet loss

[0022]

[0023]

[0024] Among them, the cross entropy loss Calculated on the sample of the current task, while the triplet loss Calculate samples in the buffer memory;

[0025] Step 11: The loss applied to the memory model is calculated as Then by L m Perform gradient backpropagation and update the parameters of the working model That is, for Wherein, η is 0.1;

[0026] Step 12: Repeat steps 6 to 11 until the alternating training is completed;

[0027] Step 13: After knowledge replay and knowledge update, knowledge integration is performed. At this stage, there are two trained working models in the learning network. and memory model Integrate the model space to prepare for the next stage of training and deployment. The resulting composite model can obtain a larger proportion of knowledge from the memory model, thereby better integrating past knowledge.

[0028] Step 14: Next, perform knowledge integration of the feature space for testing. Given a picture to be tested, input the feature extractors of the working model and the memory model respectively to obtain two feature vectors. and In order to better utilize the patterns captured by the working model and the memory model, a hybrid feature is obtained by concatenation. Finally, the mixed features Used for pedestrian retrieval;

[0029] Step 15: After task t is completed, perform buffer storage module M t To update, first initialize M t+1 =M t , randomly select multiple IDs from the label library of the new task, and obtain the feature average of each ID as the prototype of the ID in the feature space;

[0030] Step 16: For each selected ID, select the two features with the greatest distance between them in its feature space and update the corresponding image and label storage buffer to better characterize the decision boundary of the new task category;

[0031] Step 17: Expand the new classifier and initialize the neuron parameters of the new classifier on the new task;

[0032] Step 18: Repeat steps 5 to 17 until all tasks are completed.

[0033] Preferably, the following steps are further included before step 1: performing a series of preprocessing on the input image, and configuring a training optimizer and a training plan.

[0034] Preferably, in step 1, the working model performs feature extraction and obtains the softmax prediction score, including the following steps:

[0035] Step 101: The working model consists of five stages:

[0036] The first stage is the preprocessing of the input, including a convolutional layer, a batch normalization layer, a ReLU activation function, and a maximum pooling layer;

[0037] The subsequent four stages consist of three, four, six, and three BottleNeck layers, respectively;

[0038] Step 102: The feature map processed in the five stages is fed into a generalized pooling layer and a batch normalization layer to obtain normalized features.

[0039] Step 103: During the training process, the normalized features are input into an unbiased linear classifier to obtain a prediction score. Finally, the network has three outputs during the training process: normalized features, unnormalized features, and prediction scores. The unnormalized features are used as the triplet loss during the training process, and the normalized features are only used to obtain the prediction score. The prediction score is used as the distillation and cross entropy loss during the training process after softmax.

[0040] Step 104: During the test, only the normalized features are output and the features of the query object are and find objects The similarity matrix is ​​calculated based on the features of in, Is the qth in the query queue n Features, Is to find the g in the object library m features; then, the similarity ranking of each query object and all search objects can be calculated based on the similarity matrix and the recognition accuracy can be calculated.

[0041] Preferably, in step 3, the anchor feature is the feature extracted from the i-th pedestrian image by the working model, the positive feature is the feature that has the same identity as the anchor feature in the current batch of features and has the farthest Euclidean distance, and the negative feature is the feature that has a different identity from the anchor feature in the current batch of features and has the closest Euclidean distance.

[0042] Preferably, in step 4, the loss function at stage t=0 is

[0043] Preferably, in step 4, the network parameter updating process includes the following steps:

[0044] Step 401: Freeze the memory model parameters, input the current task sample into the memory model to obtain the predicted score as the pseudo label, and input it into the working model to obtain the predicted score and the pseudo label to calculate the JS divergence to obtain the distillation loss; add the distillation loss to the current task loss and perform gradient backpropagation to update the parameters of the working model;

[0045] Step 502: Freeze the working model parameters, input the current task sample into the working model to obtain a predicted score as a pseudo-label, and simultaneously input the predicted score into the memory model to calculate the JS divergence with the pseudo-label to obtain the distillation loss; add the distillation loss to the current task loss and perform gradient backpropagation to update the parameters of the memory model;

[0046] Step 503: Repeat steps 401 to 402 until a preset number of iterations is reached.

[0047] Preferably, the memory correction loss function is used in step 9

[0048] Preferably, in step 13, the working model parameters of stage t+1 are Updated to The new memory model parameters are also updated as

[0049] Preferably, in step 15, the buffer storage area M t The update method includes the following steps:

[0050] Step 1501: After task t is completed, perform buffer storage module M t To update, first initialize M t+1 =M t ,Will Batch input working model, specify the number of iterations, so that the model extracts All the characteristics of Randomly select 250 IDs from the tag library to get {id1,…,id 250}, and extract the average value of all features for each ID

[0051] Step 1502: For each selected IDid k , and all its features are combined with Sort by the Euclidean distance and select the two features with the farthest distance And update the corresponding image and label storage buffer: Finally got the updated M t .

[0052] Preferably, in step 17, the new classifier expansion process includes the following steps:

[0053] Step 1701: Extract the training data of the new task, input the working model trained in the previous stage, extract the feature set of each category of the new task, and average the feature set to obtain the prototype of each category of the new task in the feature space.

[0054] Step 1702: The prototype obtained in the previous step is used as the initialization of the neuron parameters of the classifier on the new task. The neuron parameters of the classifier on the old task inherit the original weights as the initialization.

[0055] The present invention divides the task learning process in lifelong learning into three stages: knowledge replay, knowledge updating, and knowledge integration. Knowledge replay receives local task data and buffer task data, and maintains the working model with the help of pseudo labels generated by the memory model. At the same time, the response generated by the working model is passed to the memory model for self-update, thereby generating better pseudo labels. These two modes alternately update the model parameters to achieve two-way information transmission. After the task training is completed, the knowledge integration stage begins. At the model level, the present invention designs a sliding average method to achieve the integration of working and memory models to facilitate the deployment training in the next stage. In the testing stage, the present invention provides a feature space integration to improve the generalization of prediction results in visible and invisible domains.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] This paper provides a new knowledge update and integration algorithm for lifelong learning person re-identification. It not only effectively addresses the problem of catastrophic forgetting but also effectively implements both forward and backward transfer. Based on knowledge replay, this paper proposes a knowledge update mechanism that uses a dynamically updated memory model to guide the learning of the working model, thereby achieving two-way knowledge interaction and better transfer results. Furthermore, this paper introduces a knowledge integration method to enhance the long-term stability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 Training and testing flowchart for lifelong learning of person re-identification;

[0059] Figure 2 The algorithm framework based on knowledge playback, knowledge update and knowledge integration is illustrated;

[0060] Figure 3 For knowledge playback and knowledge update graph;

[0061] Figure 4 Figure 2 illustrates model knowledge integration. DETAILED DESCRIPTION

[0062] Below in conjunction with specific embodiment, further set forth the present invention.Should be understood that these embodiments are only used to illustrate the present invention and are not used in limiting the scope of the present invention.In addition, should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall equally within the scope limited by the appended claims of the application.

[0063] This embodiment discloses a lifelong learning pedestrian re-identification method and apparatus based on knowledge updating and knowledge integration, and provides a lifelong learning pedestrian re-identification learning framework that organizes the learning process into three stages: knowledge replay, knowledge updating, and knowledge integration. Knowledge replay and knowledge updating are interdependent and alternately optimized to achieve two-way knowledge transfer. Knowledge integration is divided into knowledge integration in the feature space and knowledge integration in the model space, responsible for organically integrating the knowledge of the working model and the memory model after the task is completed, to enhance the long-term stability of the pedestrian re-identification model.

[0064] Combine Figure 1 The method disclosed in this embodiment can be specifically described as follows:

[0065] Step 1: Perform a series of preprocessing on the input image and configure the training optimizer and training plan.

[0066] The input image is preprocessed in the following steps:

[0067] a) Scale the image to 256×128 and interpolate to 3.

[0068] b) Randomly flip the image horizontally with a probability of 0.5.

[0069] c) Randomly crop the image.

[0070] d) Randomly erase the image with a probability of 0.5.

[0071] e) Normalize the images, setting the means to 0.485, 0.456, and 0.406, and the variances to 0.229, 0.224, and 0.225.

[0072] Configuring the optimizer and training plan for training involves the following steps:

[0073] Configure the Adam optimizer for the working model and the memory model with a decay coefficient of 0.0005. Set the learning rate of the working model optimizer to 1 and the learning rate of the memory model optimizer to 0.1.

[0074] The learning rate was adjusted using the Warmup Scheduler. Specifically, for the first task, the initial learning rate was set to 0.00035, and the learning rate was decreased at a decay rate of 0.1 after the 30th epoch. For subsequent tasks, the initial learning rate was set to 0.00035, and the learning rate was decreased at a decay rate of 0.1 after the 40th and 70th epochs. Each task was trained for 80 epochs.

[0075] Step 2: At the beginning of the initial task t=0, t is the current task number, given the data set Randomly sample a batch of pictures to get N b is the batch size, is the i-th picture and is input into the working model based on convolutional neural network (CNN) to extract features and obtain the softmax prediction score q(·) is the softmax function used by the working model.

[0076] The working model is based on ResNet50 and has been partially replaced. The working model performs feature extraction and obtains the softmax prediction score in the following steps:

[0077] Step 201: The working model includes five stages:

[0078] The first stage is the preprocessing of the input, including a convolution layer, a batch normalization layer, a ReLU activation function and a maximum pooling layer. The convolution kernel size is 7×7, the number is 64, and the stride is 2. The kernel size of the maximum pooling layer is 3×3 and the stride is 2.

[0079] The subsequent four stages consist of three, four, six, and three BottleNeck layers, respectively. The BottleNeck layer contains three convolution layers (including batch normalization and ReLU functions, with kernel sizes of 1×1, 3×3, and 1×1, respectively), an optional downsampling layer, and a residual operator.

[0080] After the five stages are completed, the output feature dimensions are 64, 256, 512, 1024, and 2048 respectively.

[0081] Step 202: The feature maps processed through the five stages are fed into a generalized mean pooling layer and a batch normalization layer. The weights of the batch normalization layer are initialized to 1, the bias is initialized to 0, and the reverse gradient update of the bias is cleared. This results in normalized features, each with a dimension of 2048.

[0082] Step 203: During the training process, the normalized features need to be input into an unbiased linear classifier to obtain the prediction score. Finally, the network has three outputs during the training process: normalized features, unnormalized features, and prediction scores. Among them, the unnormalized features are used as triplet loss in the training process, and the normalized features are only used to obtain the prediction score. The prediction score is used as distillation and cross entropy loss in the training process after softmax.

[0083] Step 204: During the test, only the normalized features are output and the features of the query object are and find objects The similarity matrix is ​​calculated based on the features of in, Is the qth in the query queue n Features, Is to find the g in the object library m Features. Then, the similarity ranking of each query object and all search objects can be calculated based on the similarity matrix and the recognition accuracy can be calculated. Figure 1 As shown, the testing process includes testing in the unseen domain as well as testing in all seen domains.

[0084] Step 3: In order to learn the discriminability of the working model for the pedestrian images of the current task, calculate the cross entropy loss function based on the current batch of images in is the i-th prediction score No. Item value.

[0085] Step 4: In order to further optimize the feature distribution in the metric space, a triplet loss function is applied to the extracted current batch features. Where d(·,·) represents the Euclidean distance function, m is the set threshold parameter (greater than 0), Represents the anchor feature, positive feature, and negative feature of the i-th pedestrian in the current batch extracted by the working model. The anchor feature is the feature extracted by the working model for the i-th pedestrian image. The positive feature is the feature with the same identity as the anchor feature in the current batch and has the longest Euclidean distance. The negative feature is the feature with a different identity from the anchor feature in the current batch and has the shortest Euclidean distance.

[0086] Step 5: Combine cross entropy loss function and triplet loss function Get the loss function at stage t=0 The network parameters are updated through reverse gradient propagation until the preset number of iterations is reached.

[0087] The network parameter update process is as follows: Figure 3 As shown, the following steps are included:

[0088] Step 501: Freeze the memory model parameters. Input the current task sample into the memory model to obtain a predicted score as a pseudo-label. Simultaneously input the predicted score into the working model and calculate the JS divergence with the pseudo-label to obtain the distillation loss. The distillation temperature is set to 2. The distillation loss is added to the current task loss and the gradient is backpropagated to update the working model parameters.

[0089] Step 502: Freeze the working model parameters. Input the current task sample into the working model to obtain a predicted score as a pseudo-label. Simultaneously input the predicted score into the memory model. Calculate the JS divergence with the pseudo-label to obtain the distillation loss. The distillation temperature is set to 2. Add the distillation loss to the current task loss and perform gradient backpropagation to update the memory model parameters.

[0090] Step 503: Repeat steps 501 to 502 until a preset number of iterations is reached.

[0091] Step 6: For tasks at time t>0, the learning network contains both a memory model and a working model. The goal is to improve the performance of the current task and previous tasks through the joint learning of the memory model and the working model, and improve the overall generalization of the model.

[0092] At the beginning of task t, the model first enters the knowledge playback phase. Specifically, the input at this time is the data set at time t Buffer storage area M t It is used to store the partial samples seen from time 0 to t-1, i.e. their labels. The learning rates of the working model and the memory model are γ and η respectively. and M t Randomly sample a batch of samples from and Represents pictures respectively The corresponding label is sent to the working model and the memory model respectively, and the corresponding softmax prediction score is output, that is, Among them, p(·) and q(·) represent the prediction scores output by the working model and memory model respectively with (·) as input.

[0093] Step 7: To prevent the working model from forgetting old tasks, knowledge distillation is performed at the prediction score level to address catastrophic forgetting. Specifically, the prediction scores generated by the memory model can be used as pseudo labels to guide the learning of the working model. Anti-forgetting loss function Can be specifically expressed as Where T represents the distillation temperature, SD(·) represents the gradient separation operator, and JS(·) represents the Jesen-Shannon divergence.

[0094] Step 8: In order to make the model better adapt to the current task, design the corresponding adaptive loss function Among them, the adaptive loss function includes a cross entropy loss of the current task and a triplet loss for the current task

[0095]

[0096]

[0097] Represents the anchor features, positive features, and negative features of the i-th pedestrian in the current batch extracted by the working model; Represents the anchor features, positive features, and negative features of the i-th pedestrian in the current batch extracted by the memory model.

[0098] Step 9: Obtain the overall loss function of the working model in the knowledge replay phase Then the overall loss function L w Perform gradient backpropagation and update the parameters of the working model by the gradient descent algorithm That is, for Among them, θ is the parameter of the working model, including the parameters of the feature extractor and classifier, The loss function L obtained by the current batch of training samples w The calculated gradient, γ is set to 1.

[0099] Step 10: Knowledge Update Phase:

[0100] Mainly responsible for updating the memory model, that is, using the knowledge of the working model to correct the response of the memory model to achieve smoother knowledge transfer. This embodiment specifically adopts the memory correction loss function

[0101] Step 11: Enhance the ability of the memory model to handle the current task and calculate the memory loss function including cross entropy loss and triplet loss

[0102]

[0103]

[0104] Among them, the cross entropy loss Calculated on the sample of the current task, while the triplet loss Count samples in the buffer memory.

[0105] Step 12: The loss applied to the memory model can be calculated as Then by L m Perform gradient backpropagation and update the parameters of the working model That is, for Wherein, η is taken as 0.1.

[0106] Step 13: Repeat steps 7 to 12 until the alternating training is completed.

[0107] Step 14: After finishing knowledge playback and knowledge update, knowledge integration is performed. At this stage, there are two trained working models in the learning network. and memory model The model space is integrated to prepare for the training of the next new stage. Specifically, the working model parameters of stage t+1 are Updated to Similarly, the new memory model parameters are also updated as The composite model obtained by integrating the model space obtains a larger proportion of knowledge from the memory model, making it more stable, that is, it can better integrate past knowledge.

[0108] Combine Figure 4 , the model space integration process includes the following steps:

[0109] a) Perform a sliding average weighted sum on the working model parameters and the memory model parameters to obtain a sliding average model, which is then used to initialize the working model for the next stage.

[0110] b) The next stage memory model can be initialized in the same way. The next stage memory model is obtained by deep copying the working model.

[0111] Step 15: Then, perform knowledge integration of feature space for testing. Given a picture to be tested, input the feature extractors of the working model and the memory model respectively to obtain two feature vectors and In order to better utilize the patterns captured by the working model and the memory model, a hybrid feature is obtained by concatenation. Finally, the mixed features Used for pedestrian retrieval.

[0112] Step 16: After task t is completed, perform buffer storage module M t First, initialize M t+1 =M t Randomly select 250 IDs from the label library of the new task and obtain the feature average of each ID as the prototype of the ID in the feature space.

[0113] In this embodiment, the buffer storage area M t The update method includes the following steps:

[0114] a) After task t is completed, the buffer storage module M is executed t First, initialize M t+1 =M t .Will Batch input working model, specify the number of iterations, so that the model extracts All the characteristics of Randomly select 250 IDs from the tag library to get {id1,…,id 250}, and extract the average value of all features for each ID

[0115] b) For each selected IDid k , and all its features are combined with Sort by the Euclidean distance and select the two features with the farthest distance And update the corresponding image and label storage buffer: Finally got the updated M t .

[0116] Step 17: For each selected ID, select the two features that are farthest apart in its feature space and update the corresponding image and label storage buffer to better characterize the decision boundary of the new task category.

[0117] Step 18: Expand the new classifier and initialize the neuron parameters of the new classifier on the new task.

[0118] Step 19: Repeat steps 6 to 18 until all tasks are completed.

[0119] In step 18, the expansion process of the new classifier includes the following steps:

[0120] Step 1801: Extract the training data of the new task and input it into the working model trained by the previous task to extract the feature set of each category of the new task. Average the feature set to obtain the prototype of each category of the new task in the feature space.

[0121] Step 1802: The prototype obtained in the previous step is used as the initialization of the neuron parameters of the classifier on the new task, and the neuron parameters of the classifier on the old task inherit the original weights as initialization.

Claims

1. A lifelong learning person re-identification method based on knowledge updating and knowledge integration, characterized by: The following steps are involved: Step 1: At the beginning of the initial task t=0, t is the current task number, given a data set Randomly sample a batch of pictures to get N b is the batch size, is the i-th picture, and is input into the working model based on the convolutional neural network to extract features and obtain the softmax prediction score q(·) is the softmax function used by the working model; Step 2: In order to learn the discriminability of the working model for the pedestrian images of the current task, calculate the cross entropy loss based on the current batch of images in is the i-th prediction score No. Item value; Step 3: In order to further optimize the feature distribution in the metric space, triple loss is applied to the extracted current batch features Among them, d(·,·) represents the Euclidean distance function, m is the set threshold parameter, Represents the anchor features, positive features, and negative features of the i-th pedestrian in the current batch extracted by the working model; Step 4: Combine cross entropy loss and triplet loss Obtain the loss function L0 at stage t=0, and update the network parameters through reverse gradient propagation until the preset number of iterations is reached; Step 5: For tasks at time t>0, the learning network incorporates both a memory model and a working model. The goal is to improve the performance of the current task and previous tasks through the joint learning of the memory model and the working model, thereby improving the overall generalization of the model. At the beginning of task t, the model first enters the knowledge playback phase, and the input at this time is the data set at time t Buffer storage area M t It is used to store the partial samples seen from time 0 to t-1, i.e. their labels. The learning rates of the working model and the memory model are γ and η respectively. and M t Randomly sample a batch of samples from and They are respectively sent to the working model and the memory model, and the corresponding softmax prediction scores are output, that is, Among them, p(·) and q(·) represent the prediction scores output by the working model and memory model respectively with (·) as input; Step 6: In order to prevent the working model from forgetting the old tasks, knowledge distillation is performed at the prediction score level to solve catastrophic forgetting. The prediction scores generated by the memory model can be used as pseudo labels to guide the learning of the working model and resist forgetting loss. Expressed as Where T represents the distillation temperature, SD(·) represents the gradient separation operator, and JS(·) represents the Jesen-Shannon divergence; Step 7: In order to make the model better adapt to the current task, design the corresponding adaptive loss function Among them, the adaptive loss function includes a cross entropy loss of the current task and a triplet loss for the current task Represents the anchor features, positive features, and negative features of the i-th pedestrian in the current batch extracted by the working model; Represents the anchor features, positive features, and negative features of the i-th pedestrian in the current batch extracted by the memory model; Step 8: Obtain the overall loss of the working model in the knowledge replay phase Then the overall loss L w Perform gradient backpropagation and update the parameters of the working model by the gradient descent algorithm That is, for Among them, θ is the parameter of the working model, including the parameters of the feature extractor and classifier, is the total loss L obtained from the current batch of training samples w The calculated gradient, γ is 1; Step 9: Knowledge update phase: Responsible for updating the memory model, that is, using the knowledge of the working model to correct the response of the memory model to achieve smoother knowledge transfer; Step 10: Enhance the ability of the memory model to handle the current task and calculate the memory loss function including cross entropy loss and triplet loss Among them, the cross entropy loss Calculated on the sample of the current task, while the triplet loss Calculate samples in the buffer memory; Step 11: The loss applied to the memory model is calculated as is the memory correction loss, and then L m Perform gradient backpropagation and update the parameters of the memory model That is, for Wherein, η is 0.1; Step 12: Repeat steps 6 to 11 until the alternating training is completed; Step 13: After knowledge replay and knowledge update, knowledge integration is performed. In this stage, the learning network stores two trained working models and a memory model. The model space is integrated to prepare for future training. The composite model obtained through model space integration obtains a larger proportion of knowledge from the memory model, making it more stable, that is, it can better integrate past knowledge. Step 14: Next, perform knowledge integration of the feature space for testing. Given a picture to be tested, input the feature extractors of the working model and the memory model respectively to obtain two feature vectors. and In order to better utilize the patterns captured by the working model and the memory model, a hybrid feature is obtained by concatenation. Finally, the mixed features Used for pedestrian retrieval; Step 15: After task t is completed, perform buffer storage module M t To update, first initialize M t+1 =M t , randomly select multiple IDs from the label library of the new task, and obtain the feature average of each ID as the prototype of the ID in the feature space; Step 16: For each selected ID, select the two features with the greatest distance between them in its feature space and update the corresponding image and label storage buffer to better characterize the decision boundary of the new task category; Step 17: Expand the new classifier and initialize the neuron parameters of the new classifier on the new task; Step 18: Repeat steps 5 to 17 until all tasks are completed.

2. The lifelong learning person re-identification method based on knowledge updating and knowledge integration as claimed in claim 1, characterized in that: Before step 1, the following steps are also included: a series of preprocessing is performed on the input image, and a training optimizer and training plan are configured.

3. The lifelong learning person re-identification method based on knowledge updating and knowledge integration as claimed in claim 1, characterized in that: In step 1, the working model performs feature extraction and obtains the softmax prediction score, which includes the following steps: Step 101: The working model consists of five stages: The first stage is the preprocessing of the input, including a convolutional layer, a batch normalization layer, a ReLU activation function, and a maximum pooling layer; The subsequent four stages consist of three, four, six, and three BottleNeck layers, respectively; Step 102: The feature map processed in the five stages is fed into a generalized pooling layer and a batch normalization layer to obtain normalized features. Step 103: During the training process, the normalized features are input into an unbiased linear classifier to obtain a prediction score. Finally, the network has three outputs during the training process: normalized features, unnormalized features, and prediction scores. The unnormalized features are used as the triplet loss during the training process, and the normalized features are only used to obtain the prediction score. The prediction score is used as the distillation and cross entropy loss during the training process after softmax. Step 104: During the test, only the normalized features are output and the features of the query object are Finding the characteristics of the object Calculate the similarity matrix in, Is the qth in the query queue n Features, Is to find the g in the object library m features; then, the similarity ranking of each query object and all search objects can be calculated based on the similarity matrix and the recognition accuracy can be calculated.

4. The lifelong learning person re-identification method based on knowledge updating and knowledge integration as claimed in claim 1, characterized in that: In step 3, the anchor feature is the feature extracted by the working model from the i-th pedestrian image. The positive feature is the feature that has the same identity as the anchor feature in the current batch of features and has the farthest Euclidean distance from it. The negative feature is the feature that has a different identity from the anchor feature in the current batch of features and has the closest Euclidean distance from it.

5. The lifelong learning person re-identification method based on knowledge updating and knowledge integration as claimed in claim 1, characterized in that: In step 4, the loss function at stage t=0 is 6. The lifelong learning person re-identification method based on knowledge updating and knowledge integration as claimed in claim 1, characterized in that: In step 4, the network parameter update process includes the following steps: Step 401: Freeze the memory model parameters, input the current task sample into the memory model to obtain the predicted score as the pseudo label, and input it into the working model to obtain the predicted score and the pseudo label to calculate the JS divergence to obtain the distillation loss; add the distillation loss to the current task loss and perform gradient backpropagation to update the parameters of the working model; Step 502: Freeze the working model parameters, input the current task sample into the working model to obtain a predicted score as a pseudo-label, and simultaneously input the predicted score into the memory model to calculate the JS divergence with the pseudo-label to obtain the distillation loss; add the distillation loss to the current task loss and perform gradient backpropagation to update the parameters of the memory model; Step 503: Repeat steps 401 to 402 until a preset number of iterations is reached.

7. The lifelong learning person re-identification method based on knowledge updating and knowledge integration as claimed in claim 1, characterized in that: Memory correction loss is used in step 9 8. The lifelong learning person re-identification method based on knowledge updating and knowledge integration as claimed in claim 1, characterized in that: In step 13, the parameters of the working model in stage t+1 are Updated to The parameters of the new memory model are also updated as 9. The lifelong learning person re-identification method based on knowledge updating and knowledge integration as claimed in claim 1, characterized in that: In step 15, the buffer storage area M t The update method includes the following steps: Step 1501: After task t is completed, the buffer storage area M t To update, first initialize M t+1 =M t ,Will Batch input working model, specify the number of iterations, so that the model extracts All the characteristics of Randomly select 250 IDs from the tag library to get {id1,…,id 250 }, and extract the average value of all features for each ID Step 1502: For each selected IDid k , and all its features are combined with Sort by the Euclidean distance and select the two features with the farthest distance And update the corresponding image and label storage buffer: Finally got the updated M t .

10. The lifelong learning person re-identification method based on knowledge updating and knowledge integration as claimed in claim 1, characterized in that: In step 17, developing a new classifier includes the following steps: Step 1701: Extract the training data of the new task, input the working model trained in the previous stage, extract the feature set of each category of the new task, and average the feature set to obtain the prototype of each category of the new task in the feature space. Step 1702: The prototype obtained in the previous step is used as the initialization of the neuron parameters of the classifier on the new task, and the neuron parameters of the classifier on the old task inherit the original weights as initialization.

Images