Pedestrian re-identification method and device

By introducing quality regression branches and multi-stage training methods into the pedestrian recognition model, the problem of long-term, high cost and low accuracy of model training in the pedestrian recognition algorithm is solved, and a more efficient pedestrian recognition effect is achieved.

CN115147870BActive Publication Date: 2024-11-19SHENZHEN XUMI YUNTU SPACE TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202210843728.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-18
Publication Date
2024-11-19
Estimated Expiration
2042-07-18

AI Technical Summary

Technical Problem

The existing pedestrian re-identification algorithm has problems such as long-term training, high cost and low accuracy.

Method used

The pedestrian recognition model is adopted, including pedestrian recognition branches and mass regression branches. By obtaining the pre-trained model for initialization and multi-stage training, the dynamically changing interval factor is used to improve the model accuracy and reduce training costs and time.

Benefits of technology

It effectively reduces the time and cost of pedestrian re-identification model training, improves the accuracy of the model, and can more efficiently identify specific people from monitoring pictures.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115147870B_ABST
    Figure CN115147870B_ABST
Patent Text Reader

Abstract

The present disclosure relates to the technical field of machine learning, and provides a method and device for pedestrian re-identification. The method includes: constructing a pedestrian re-identification model, wherein the pedestrian re-identification model includes: a pedestrian re-identification branch and a quality regression branch; obtaining a pre-trained model, and using the pre-trained model to initialize the pedestrian re-identification model; performing multi-stage training on the initialized pedestrian re-identification model; obtaining a pedestrian data set to be classified, wherein the pedestrian data set includes: multiple pictures of multiple people; using the pedestrian re-identification model after multi-stage training to classify the pictures in the pedestrian data set, and obtain one or more pictures of each person. The above-mentioned technical means are adopted to solve the problems in the prior art that the model training is time-consuming, costly, and has low accuracy in the current pedestrian re-identification algorithm.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of machine learning, and in particular to a method and device for pedestrian re-identification. Background Art

[0002] In a period of time, among all the surveillance images obtained, one person may have multiple surveillance images. If all the surveillance images are divided into each person who appears under the surveillance device in the period of time, and then the multiple images of each person are used to analyze the person, the accuracy and correctness of the analysis can be greatly improved. The above technology of finding the image corresponding to a specific person in many surveillance images is called pedestrian re-identification. Person re-identification, also known as pedestrian re-identification, is a technology that uses computer vision technology to determine whether there is a specific pedestrian in an image or video sequence. The current pedestrian re-identification algorithm is often implemented by training a neural network, which has the problems of time-consuming and high cost of training, and low accuracy of the final model.

[0003] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following technical problems in the related art: the current pedestrian re-identification algorithm has the problems of time-consuming and high cost of model training and low accuracy. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure provide a pedestrian re-identification method, device, electronic device and computer-readable storage medium to solve the problems in the prior art that model training is time-consuming, costly and has low accuracy in current pedestrian re-identification algorithms.

[0005] According to a first aspect of an embodiment of the present disclosure, a pedestrian re-identification method is provided, comprising: constructing a pedestrian re-identification model, wherein the pedestrian re-identification model comprises: a pedestrian re-identification branch and a quality regression branch; obtaining a pre-trained model, and initializing the pedestrian re-identification model using the pre-trained model; performing multi-stage training on the initialized pedestrian re-identification model; obtaining a pedestrian data set to be classified, wherein the pedestrian data set comprises: multiple pictures of multiple persons; and classifying the pictures in the pedestrian data set using the pedestrian re-identification model after multi-stage training to obtain one or more pictures of each person.

[0006] According to a second aspect of an embodiment of the present disclosure, a pedestrian re-identification device is provided, comprising: a construction module, configured to construct a pedestrian re-identification model, wherein the pedestrian re-identification model comprises: a pedestrian re-identification branch and a quality regression branch; an initialization module, configured to obtain a pre-trained model, and initialize the pedestrian re-identification model using the pre-trained model; a training module, configured to perform multi-stage training on the initialized pedestrian re-identification model; an acquisition module, configured to obtain a pedestrian data set to be classified, wherein the pedestrian data set comprises: multiple pictures of multiple persons; a classification module, configured to classify the pictures in the pedestrian data set using the pedestrian re-identification model after multi-stage training, and obtain one or more pictures of each person.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0008] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0009] Compared with the prior art, the embodiments of the present disclosure have the following beneficial effects: because the embodiments of the present disclosure construct a pedestrian re-identification model, wherein the pedestrian re-identification model includes: a pedestrian re-identification branch and a quality regression branch; obtain a pre-trained model, and use the pre-trained model to initialize the pedestrian re-identification model; perform multi-stage training on the initialized pedestrian re-identification model; obtain a pedestrian data set to be classified, wherein the pedestrian data set includes: multiple pictures of multiple people; use the pedestrian re-identification model that has undergone multi-stage training to classify the pictures in the pedestrian data set to obtain one or more pictures of each person. Therefore, the above-mentioned technical means can be used to solve the problems of time-consuming, high-cost, and low-precision model training in the current pedestrian re-identification algorithm in the prior art, thereby reducing the time and cost of model training in the pedestrian re-identification algorithm and improving the model accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 is a schematic diagram of an application scenario of an embodiment of the present disclosure;

[0012] Figure 2 is a flowchart of a pedestrian re-identification method provided by an embodiment of the present disclosure;

[0013] Figure 3 is a structural schematic diagram of a pedestrian re-identification device provided by an embodiment of the present disclosure;

[0014] Figure 4 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0015] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.

[0016] A pedestrian re-identification method and device according to an embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0017] Figure 1 1 is a schematic diagram of an application scenario of an embodiment of the present disclosure. The application scenario may include terminal devices 101, 102 and 103, a server 104 and a network 105.

[0018] The terminal devices 101, 102 and 103 can be hardware or software. When the terminal devices 101, 102 and 103 are hardware, they can be various electronic devices with display screens and supporting communication with the server 104, including but not limited to smart phones, tablet computers, laptop portable computers and desktop computers; when the terminal devices 101, 102 and 103 are software, they can be installed in the above electronic devices. The terminal devices 101, 102 and 103 can be implemented as multiple software or software modules, or as a single software or software module, and the embodiments of the present disclosure are not limited to this. Furthermore, various applications can be installed on the terminal devices 101, 102 and 103, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0019] The server 104 may be a server that provides various services, for example, a background server that receives a request sent by a terminal device that establishes a communication connection with the server, and the background server may receive and analyze the request sent by the terminal device, and generate a processing result. The server 104 may be a single server, a server cluster consisting of several servers, or a cloud computing service center, which is not limited in the embodiments of the present disclosure.

[0020] It should be noted that the server 104 can be hardware or software. When the server 104 is hardware, it can be various electronic devices that provide various services for the terminal devices 101, 102, and 103. When the server 104 is software, it can be multiple software or software modules that provide various services for the terminal devices 101, 102, and 103, or it can be a single software or software module that provides various services for the terminal devices 101, 102, and 103, and the embodiments of the present disclosure are not limited to this.

[0021] The network 105 may be a wired network connected by coaxial cables, twisted pairs, and optical fibers, or a wireless network that can interconnect various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), infrared, etc., which is not limited in the embodiments of the present disclosure.

[0022] The user can establish a communication connection with the server 104 via the network 105 through the terminal devices 101, 102 and 103 to receive or send information, etc. It should be noted that the specific types, quantities and combinations of the terminal devices 101, 102 and 103, the server 104 and the network 105 can be adjusted according to the actual needs of the application scenario, and the embodiments of the present disclosure are not limited to this.

[0023] Figure 2 It is a flowchart of a pedestrian re-identification method provided by an embodiment of the present disclosure. Figure 2 The pedestrian re-identification method can be obtained by Figure 1 The computer or server, or the software on the computer or server (the software is a tool for testing the smart helmet) is executed. Figure 2 As shown, the pedestrian re-identification method includes:

[0024] S201, constructing a pedestrian re-identification model, wherein the pedestrian re-identification model includes: a pedestrian re-identification branch and a quality regression branch;

[0025] S202, obtaining a pre-trained model, and initializing a person re-identification model using the pre-trained model;

[0026] S203, performing multi-stage training on the initialized person re-identification model;

[0027] S204, obtaining a pedestrian data set to be classified, wherein the pedestrian data set includes: a plurality of pictures of a plurality of persons;

[0028] S205, using the pedestrian re-identification model that has undergone multi-stage training to classify the images in the pedestrian dataset to obtain one or more images of each person.

[0029] The current pedestrian re-identification algorithm is often implemented by training a neural network, which is equivalent to the pedestrian re-identification branch in the embodiment of the present disclosure. On the basis of the pedestrian re-identification branch, the embodiment of the present disclosure proposes a quality regression branch, and further provides a dynamically changing interval factor. Finally, the optimal interval factor is found through the quality regression branch, thereby improving the model accuracy. It should be noted that if there is only a pedestrian re-identification branch and no quality regression branch, the interval factor is fixed. The embodiment of the present disclosure proposes a corresponding multi-stage training method for the structure of the pedestrian re-identification model including two branches, the pedestrian re-identification branch and the quality regression branch. The multi-stage training method is also different from the existing model training method.

[0030] The pre-trained model is a model that has been trained in the field (pedestrian recognition or re-identification). Using the pre-trained model to initialize the pedestrian re-identification model can make the pedestrian re-identification model have better model parameters before training, and then during training, it can converge in a shorter time and with less cost, thereby achieving the training purpose. It should be noted that the pre-trained model is a model that has been trained in the field of pedestrian recognition or re-identification. The pre-trained model has a strong generalization ability, but it is not necessarily perfectly applicable to each specific scenario, so it is necessary to combine the application scenario to retrain a pedestrian re-identification model. For example, there are differences in clothing and appearance between people in region A and region B, so the model used in region A or region B should be specifically trained with a training set of pictures containing local pedestrian characteristics.

[0031] Using the pedestrian re-identification model that has been trained in multiple stages, it is possible to find all pictures of a specific person from the pedestrian dataset.

[0032] According to the technical solution provided by the embodiment of the present disclosure, a pedestrian re-identification model is constructed, wherein the pedestrian re-identification model includes: a pedestrian re-identification branch and a quality regression branch; a pre-trained model is obtained, and the pedestrian re-identification model is initialized by using the pre-trained model; the initialized pedestrian re-identification model is trained in multiple stages; a pedestrian data set to be classified is obtained, wherein the pedestrian data set includes: multiple pictures of multiple people; the pictures in the pedestrian data set are classified by using the pedestrian re-identification model that has undergone multiple stages of training to obtain one or more pictures of each person. Therefore, the above-mentioned technical means can be adopted to solve the problems of time-consuming, high-cost and low-precision model training in the current pedestrian re-identification algorithm in the prior art, thereby reducing the time-consuming and cost of model training in the pedestrian re-identification algorithm and improving the model accuracy.

[0033] In step S203, the initialized pedestrian re-identification model is subjected to multi-stage training, including three stages of training: the first stage: freezing the quality regression branch, and training the pedestrian re-identification branch using the first loss function of the pedestrian re-identification model, wherein the interval factor in the first loss function is determined by the quality regression branch, and after freezing the quality regression branch, the interval factor is a constant; the second stage: freezing the pedestrian re-identification branch, and training the quality regression branch using the second loss function that determines the interval factor; the third stage: simultaneously training the pedestrian re-identification branch and the quality regression branch using the first loss function, wherein if the quality regression branch is not frozen, the interval factor is a variable.

[0034] The specific method of training a pedestrian re-identification branch or training a quality regression branch is the same as the prior art, such as using the first training data set to train a branch (the branch is a pedestrian re-identification branch or a quality regression branch): inputting samples in the first training data set into the branch, and calculating the loss value using the first loss function; calculating the gradient value based on the loss value, and then using the gradient back propagation algorithm to update the model parameters of the branch.

[0035] The interval factor is a variable, which means that the interval factor changes dynamically. In the process of training the pedestrian re-identification branch and the quality regression branch at the same time, the optimal interval factor will be found, thereby making the first loss function optimal, and thus making the accuracy of the pedestrian re-identification branch the highest. It should be noted that the pedestrian re-identification model that has undergone multi-stage training is used to classify the images in the pedestrian dataset. In fact, the pedestrian re-identification branch in the pedestrian re-identification model is used, and the quality regression branch is only used to assist the training of the pedestrian re-identification branch.

[0036] In the process of multi-stage training of the initialized pedestrian re-identification model, the SGD optimizer can be used. The initial learning rate of the two branches of the pedestrian re-identification model is set to 0.1 (set to 0.001 in the third stage), the momentum of the optimizer is set to 0.9, and the weight decay factor is set to 5e-4; in each stage of training, the learning rate is adjusted to the original 0.1 in the 8th and 12th rounds.

[0037] The first loss function L circle :

[0038]

[0039] α p =1+ms p

[0040]

[0041] Δ p =1-m

[0042] Δ n =m

[0043] Among them, s p Indicates the similarity between the feature vector of the current image and the positive class center vector of the current image, It represents the similarity between the feature vector of the current image and the j-th negative class center vector of the current image, L represents the number of negative classes, γ is the feature scaling factor, m is the interval factor, m is determined by the quality regression branch, if the quality regression branch is frozen, m is a constant, if the quality regression branch is not frozen, m is a variable, and m is equal to the function value of the second loss function.

[0044] At this time, the quality regression branch is frozen, and only the pedestrian recognition branch is trained, and m is a constant.

[0045] For example, the current image belongs to pedestrian No. 1, the positive class center vector is the feature vector of the image of pedestrian No. 1, a negative class center vector is the feature vector of a picture of other pedestrians, and the number of all pictures of other pedestrians is L. Because the disclosed embodiment is for pedestrian re-identification, that is, each pedestrian has at least one picture confirmed to belong to this person, assuming that each pedestrian has only one picture (which can be serial numbered, such as pedestrian No. 1), then the positive class center vector of pedestrian No. 1 is the feature vector of the picture under the name of pedestrian No. 1. The feature vector of the current picture has the highest similarity with the feature vector of the picture of pedestrian No. 1, so it is determined that the current picture belongs to pedestrian No. 1.

[0046] The second loss function m:

[0047]

[0048] Among them, m0 and m1 are constants, q is the output of the quality regression branch, and q min is the minimum output of the quality regression branch, q max is the maximum output of the mass regression branch.

[0049] If the quality regression branch is not frozen, m is equal to the function value of the second loss function, or m represents the second loss function, and m is a variable; in one training, the minimum output and maximum output of the quality regression branch are certain, so in this training, the minimum output and maximum output in the second loss function can be understood as constants.

[0050] Optionally, the pedestrian re-identification branch and the quality regression branch are trained simultaneously using the first loss function, the first loss function L circle for:

[0051]

[0052] α p =1+ms p

[0053]

[0054] Δ p =1-m

[0055] Δ n =m

[0056] At this time, the quality regression branch is not frozen, and m is a variable:

[0057]

[0058] The pedestrian re-identification branch includes the following multiple network layers connected in series: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, and a global pooling and fully connected layer; the number of channels of the first convolutional layer is a first number, and the size of the picture passing through the first convolutional layer becomes one quarter of the original size; the number of channels of the second convolutional layer is a second number, and the size of the picture passing through the second convolutional layer remains unchanged; the number of channels of the third convolutional layer is a third number, and the size of the picture passing through the third convolutional layer becomes half of the original size; the number of channels of the fourth convolutional layer is a fourth number, and the size of the picture passing through the fourth convolutional layer becomes half of the original size; the number of channels of the fifth convolutional layer is a fifth number, and the size of the picture passing through the fifth convolutional layer becomes half of the original size.

[0059] The pedestrian re-identification branch is based on the R50-IBN architecture. For example: the input of the pedestrian re-identification branch is a three-channel RGB image of (384,128), where 384 is the length of the image and 128 is the width of the image. After the first convolutional layer, the length and width of the image are 1 / 4 of the original, and the number of channels is 64; after the second convolutional layer, the length and width of the image remain unchanged, and the number of channels is 128; after the third, fourth, and fifth convolutional layers, the length and width are shortened to 1 / 2 of the original, and the number of channels becomes twice the original; after the GAP (global pooling) and FC (fully connected) layers, the output length is a feature vector of 2048.

[0060] The size of the image in the first convolutional layer becomes one-fourth of the original size, and "originally" refers to the input of the person re-identification branch. Because the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, the global pooling layer, and the fully connected layer are network layers connected in series, the size of the image after the third convolutional layer becomes half of the original size, and "originally" refers to the output of the second convolutional layer; the size of the image after the fourth convolutional layer becomes half of the original size, and "originally" refers to the output of the third convolutional layer; the size of the image after the fifth convolutional layer becomes half of the original size, and "originally" refers to the output of the fourth convolutional layer.

[0061] Optionally, the global pooling and fully connected layers can be a global pooling layer or a fully connected layer, the output of which is the output of the person re-identification branch, and the output of which indicates which person a picture belongs to.

[0062] The input of the quality regression branch is the output of the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer of the pedestrian re-identification branch; the quality regression branch includes the following multiple network layers connected in series: a global pooling layer, a feature concatenation layer, a sixth convolutional layer and a fully connected layer.

[0063] Because the input of the quality regression branch is the output of the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer of the pedestrian re-identification branch, based on the example given in the previous embodiment, the following description is continued: the quality regression branch receives the output of the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer of the pedestrian re-identification branch, and first converts the outputs of the three to the same scale through the global pooling layer, that is, obtains feature maps of 12x4x256 (12 is the length of the feature map, 4 is the width of the feature map, and the number of channels is 256), 12x4x512, and 12x4x1024 respectively; then concatenates the three feature maps to obtain a feature map of 12x4x1792; then passes through two sixth convolutional layers with a step size of 2 and a convolution kernel size of 3x3; continues to pass through a fully connected layer with an input of 1792x3 and an output of 1, that is, obtains a feature value of length 1, which indicates the quality score q of the pedestrian image, and q is related to m.

[0064] Before executing step S202, that is, before obtaining the pre-trained model and initializing the pedestrian re-identification model using the pre-trained model, the method also includes: constructing a pre-trained model, wherein the pre-trained model consists of a first encoder and a second encoder with the same structure; training the first encoder using a third loss function; training the second encoder using a momentum update function, wherein the momentum update function is related to model parameters of the first encoder and the second encoder.

[0065] The specific method of training the first encoder or training the second encoder is the same as the prior art, such as using the second training data set to train the first encoder: inputting samples in the second training data set into the encoder, calculating the loss value using the third loss function; calculating the gradient value based on the loss value, and then using the gradient back propagation algorithm to update the model parameters of the first encoder. Training the second encoder with the second training data set is similar to training the first encoder with the second training data set.

[0066] Before training, a data augmentation operation may be performed on the second training data set, wherein the data augmentation operation includes: image scaling and cropping, random color jittering, random horizontal flipping, and random grayscale conversion.

[0067] The first encoder and the second encoder can be a commonly used encoding neural network model, such as a denoising autoencoder, a stacked autoencoder, and a sparse encoder. The first encoder and the second encoder have the same structure, but the corresponding loss functions are different. The loss function of the first encoder is the third loss function, and the loss function of the second encoder is the momentum update function.

[0068] The third loss function Lq:

[0069]

[0070] Where q is the output of the first encoder, k is the output of the second encoder, and k includes k + and {k -}, k + is the feature vector obtained when the second encoder and the first encoder input the same picture, that is, the feature vector of the positive sample; the set {k -} from the second training data set except k + All other samples except the corresponding samples are used as the feature vectors of negative samples of q; τ is the temperature parameter, which can be set to 0.2.

[0071] Momentum update function f k :

[0072] f k =n·f k +(1-n)f q

[0073] f k is the model parameter of the second encoder, f q is the model parameter of the first encoder, n is a constant, n is less than or equal to 1 and greater than or equal to 0.

[0074] Because the structures of the first encoder and the second encoder are the same, the model parameter categories of the first encoder and the second encoder are the same, but the specific values ​​are different. This is because the training of the two is different.

[0075] It should be noted that when the pre-trained model is used to initialize the pedestrian re-identification model, one of the first encoder and the second encoder in the pre-trained model is used, and the encoder with higher accuracy is used.

[0076] Optionally, in the training of the encoder, the AdamW optimizer can be used to optimize the training, the initial learning rate of the encoder is set to 5e-4, the momentum of AdamW is set to 0.9, and the weight decay factor is set to 5e-4; a total of 100 rounds are trained, and a cosine decay learning rate schedule is used. The parameters of the trained second encoder can be used as the model pre-training parameters for pedestrian re-identification.

[0077] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0078] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0079] Figure 3 Schematic diagram of a pedestrian re-identification device provided by an embodiment of the present disclosure. Figure 3 As shown, the pedestrian re-identification device includes:

[0080] The construction module 301 is configured to construct a pedestrian re-identification model, wherein the pedestrian re-identification model includes: a pedestrian re-identification branch and a quality regression branch;

[0081] An initialization module 302 is configured to obtain a pre-trained model and initialize a person re-identification model using the pre-trained model;

[0082] The training module 303 is configured to perform multi-stage training on the initialized person re-identification model;

[0083] The acquisition module 304 is configured to acquire a pedestrian data set to be classified, wherein the pedestrian data set includes: a plurality of pictures of a plurality of persons;

[0084] The classification module 305 is configured to classify the images in the pedestrian dataset using the pedestrian re-identification model trained in multiple stages to obtain one or more images of each person.

[0085] The current pedestrian re-identification algorithm is often implemented by training a neural network, which is equivalent to the pedestrian re-identification branch in the embodiment of the present disclosure. On the basis of the pedestrian re-identification branch, the embodiment of the present disclosure proposes a quality regression branch, and further provides a dynamically changing interval factor. Finally, the optimal interval factor is found through the quality regression branch, thereby improving the model accuracy. It should be noted that if there is only a pedestrian re-identification branch and no quality regression branch, the interval factor is fixed. The embodiment of the present disclosure proposes a corresponding multi-stage training method for the structure of the pedestrian re-identification model including two branches, the pedestrian re-identification branch and the quality regression branch. The multi-stage training method is also different from the existing model training method.

[0086] The pre-trained model is a model that has been trained in the field (pedestrian recognition or re-identification). Using the pre-trained model to initialize the pedestrian re-identification model can make the pedestrian re-identification model have better model parameters before training, and then during training, it can converge in a shorter time and with less cost, thereby achieving the training purpose. It should be noted that the pre-trained model is a model that has been trained in the field of pedestrian recognition or re-identification. The pre-trained model has a strong generalization ability, but it is not necessarily perfectly applicable to each specific scenario, so it is necessary to combine the application scenario to retrain a pedestrian re-identification model. For example, there are differences in clothing and appearance between people in region A and region B, so the model used in region A or region B should be specifically trained with a training set of pictures containing local pedestrian characteristics.

[0087] Using the pedestrian re-identification model that has been trained in multiple stages, it is possible to find all pictures of a specific person from the pedestrian dataset.

[0088] According to the technical solution provided by the embodiment of the present disclosure, a pedestrian re-identification model is constructed, wherein the pedestrian re-identification model includes: a pedestrian re-identification branch and a quality regression branch; a pre-trained model is obtained, and the pedestrian re-identification model is initialized by using the pre-trained model; the initialized pedestrian re-identification model is trained in multiple stages; a pedestrian data set to be classified is obtained, wherein the pedestrian data set includes: multiple pictures of multiple people; the pictures in the pedestrian data set are classified by using the pedestrian re-identification model that has undergone multiple stages of training to obtain one or more pictures of each person. Therefore, the above-mentioned technical means can be adopted to solve the problems of time-consuming, high-cost and low-precision model training in the current pedestrian re-identification algorithm in the prior art, thereby reducing the time-consuming and cost of model training in the pedestrian re-identification algorithm and improving the model accuracy.

[0089] Optionally, the training module 303 is also configured to freeze the quality regression branch, train the pedestrian re-identification branch using the first loss function of the pedestrian re-identification model, wherein the interval factor in the first loss function is determined by the quality regression branch, and after freezing the quality regression branch, the interval factor is a constant; freeze the pedestrian re-identification branch, and train the quality regression branch using the second loss function that determines the interval factor; and simultaneously train the pedestrian re-identification branch and the quality regression branch using the first loss function, wherein if the quality regression branch is not frozen, the interval factor is a variable.

[0090] The specific method of training a pedestrian re-identification branch or training a quality regression branch is the same as the prior art, such as using the first training data set to train a branch (the branch is a pedestrian re-identification branch or a quality regression branch): inputting samples in the first training data set into the branch, and calculating the loss value using the first loss function; calculating the gradient value based on the loss value, and then using the gradient back propagation algorithm to update the model parameters of the branch.

[0091] The interval factor is a variable, which means that the interval factor changes dynamically. In the process of training the pedestrian re-identification branch and the quality regression branch at the same time, the optimal interval factor will be found, thereby making the first loss function optimal, and thus making the accuracy of the pedestrian re-identification branch the highest. It should be noted that the pedestrian re-identification model that has undergone multi-stage training is used to classify the images in the pedestrian dataset. In fact, the pedestrian re-identification branch in the pedestrian re-identification model is used, and the quality regression branch is only used to assist the training of the pedestrian re-identification branch.

[0092] Optionally, the training module 303 is also configured to use the SGD optimizer during the multi-stage training of the initialized pedestrian re-identification model, with the initial learning rate of the two branches of the pedestrian re-identification model set to 0.1 (set to 0.001 in the third stage), the momentum of the optimizer set to 0.9, and the weight decay factor set to 5e-4; in each stage of training, the learning rate is adjusted to the original 0.1 at the 8th and 12th rounds.

[0093] The first loss function L circle :

[0094]

[0095] α p =1+ms p

[0096]

[0097] Δ p =1-m

[0098] Δ n =m

[0099] Among them, s p Indicates the similarity between the feature vector of the current image and the positive class center vector of the current image, It represents the similarity between the feature vector of the current image and the j-th negative class center vector of the current image, L represents the number of negative classes, γ is the feature scaling factor, m is the interval factor, m is determined by the quality regression branch, if the quality regression branch is frozen, m is a constant, if the quality regression branch is not frozen, m is a variable, and m is equal to the function value of the second loss function.

[0100] At this time, the quality regression branch is frozen, and only the pedestrian recognition branch is trained, and m is a constant.

[0101] For example, the current image belongs to pedestrian No. 1, the positive class center vector is the feature vector of the image of pedestrian No. 1, a negative class center vector is the feature vector of a picture of other pedestrians, and the number of all pictures of other pedestrians is L. Because the disclosed embodiment is for pedestrian re-identification, that is, each pedestrian has at least one picture confirmed to belong to this person, assuming that each pedestrian has only one picture (which can be serial numbered, such as pedestrian No. 1), then the positive class center vector of pedestrian No. 1 is the feature vector of the picture under the name of pedestrian No. 1. The feature vector of the current picture has the highest similarity with the feature vector of the picture of pedestrian No. 1, so it is determined that the current picture belongs to pedestrian No. 1.

[0102] The second loss function m:

[0103]

[0104] Among them, m0 and m1 are constants, q is the output of the quality regression branch, and q min is the minimum output of the quality regression branch, q max is the maximum output of the mass regression branch.

[0105] If the quality regression branch is not frozen, m is equal to the function value of the second loss function, or m represents the second loss function, and m is a variable; in one training, the minimum output and maximum output of the quality regression branch are certain, so in this training, the minimum output and maximum output in the second loss function can be understood as constants.

[0106] Optionally, the training module 303 is further configured to simultaneously train the person re-identification branch and the quality regression branch using a first loss function, wherein the first loss function L circle for:

[0107]

[0108] α p =1+ms p

[0109]

[0110] Δ p =1-m

[0111] Δ n =m

[0112] At this time, the quality regression branch is not frozen, and m is a variable:

[0113]

[0114] The pedestrian re-identification branch includes the following multiple network layers connected in series: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, and a global pooling and fully connected layer; the number of channels of the first convolutional layer is a first number, and the size of the picture passing through the first convolutional layer becomes one quarter of the original size; the number of channels of the second convolutional layer is a second number, and the size of the picture passing through the second convolutional layer remains unchanged; the number of channels of the third convolutional layer is a third number, and the size of the picture passing through the third convolutional layer becomes half of the original size; the number of channels of the fourth convolutional layer is a fourth number, and the size of the picture passing through the fourth convolutional layer becomes half of the original size; the number of channels of the fifth convolutional layer is a fifth number, and the size of the picture passing through the fifth convolutional layer becomes half of the original size.

[0115] The pedestrian re-identification branch is based on the R50-IBN architecture. For example: the input of the pedestrian re-identification branch is a three-channel RGB image of (384,128), where 384 is the length of the image and 128 is the width of the image. After the first convolutional layer, the length and width of the image are 1 / 4 of the original, and the number of channels is 64; after the second convolutional layer, the length and width of the image remain unchanged, and the number of channels is 128; after the third, fourth, and fifth convolutional layers, the length and width are shortened to 1 / 2 of the original, and the number of channels becomes twice the original; after the GAP (global pooling) and FC (fully connected) layers, the output length is a feature vector of 2048.

[0116] The size of the image in the first convolutional layer becomes one-fourth of the original size, and "originally" refers to the input of the person re-identification branch. Because the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, the global pooling layer, and the fully connected layer are network layers connected in series, the size of the image after the third convolutional layer becomes half of the original size, and "originally" refers to the output of the second convolutional layer; the size of the image after the fourth convolutional layer becomes half of the original size, and "originally" refers to the output of the third convolutional layer; the size of the image after the fifth convolutional layer becomes half of the original size, and "originally" refers to the output of the fourth convolutional layer.

[0117] Optionally, the global pooling and fully connected layers can be a global pooling layer or a fully connected layer, the output of which is the output of the person re-identification branch, and the output of which indicates which person a picture belongs to.

[0118] The input of the quality regression branch is the output of the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer of the pedestrian re-identification branch; the quality regression branch includes the following multiple network layers connected in series: a global pooling layer, a feature concatenation layer, a sixth convolutional layer and a fully connected layer.

[0119] Because the input of the quality regression branch is the output of the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer of the pedestrian re-identification branch, based on the example given in the previous embodiment, the following description is continued: the quality regression branch receives the output of the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer of the pedestrian re-identification branch, and first converts the outputs of the three to the same scale through the global pooling layer, that is, obtains feature maps of 12x4x256 (12 is the length of the feature map, 4 is the width of the feature map, and the number of channels is 256), 12x4x512, and 12x4x1024 respectively; then concatenates the three feature maps to obtain a feature map of 12x4x1792; then passes through two sixth convolutional layers with a step size of 2 and a convolution kernel size of 3x3; continues to pass through a fully connected layer with an input of 1792x3 and an output of 1, that is, obtains a feature value of length 1, which indicates the quality score q of the pedestrian image, and q is related to m.

[0120] Optionally, the construction module 301 is also configured to construct a pre-trained model, wherein the pre-trained model consists of a first encoder and a second encoder having the same structure; the first encoder is trained using a third loss function; the second encoder is trained using a momentum update function, wherein the momentum update function is related to model parameters of the first encoder and the second encoder.

[0121] The specific method of training the first encoder or training the second encoder is the same as the prior art, such as using the second training data set to train the first encoder: inputting samples in the second training data set into the encoder, calculating the loss value using the third loss function; calculating the gradient value based on the loss value, and then using the gradient back propagation algorithm to update the model parameters of the first encoder. Training the second encoder with the second training data set is similar to training the first encoder with the second training data set.

[0122] The first encoder and the second encoder can be a commonly used encoding neural network model, such as a denoising autoencoder, a stacked autoencoder, and a sparse encoder. The first encoder and the second encoder have the same structure, but the corresponding loss functions are different. The loss function of the first encoder is the third loss function, and the loss function of the second encoder is the momentum update function.

[0123] The third loss function Lq:

[0124]

[0125] Where q is the output of the first encoder, k is the output of the second encoder, and k includes k + and {k -}, k + is the feature vector obtained when the second encoder and the first encoder input the same picture, that is, the feature vector of the positive sample; the set {k -} from the second training data set except k + All other samples except the corresponding samples are used as the feature vectors of negative samples of q; τ is the temperature parameter, which can be set to 0.2.

[0126] Momentum update function f k :

[0127] f k =n·f k +(1-n)f q

[0128] f k is the model parameter of the second encoder, f q is the model parameter of the first encoder, n is a constant, n is less than or equal to 1 and greater than or equal to 0.

[0129] Because the structures of the first encoder and the second encoder are the same, the model parameter categories of the first encoder and the second encoder are the same, but the specific values ​​are different. This is because the training of the two is different.

[0130] It should be noted that when the pre-trained model is used to initialize the pedestrian re-identification model, one of the first encoder and the second encoder in the pre-trained model is used, and the encoder with higher accuracy is used.

[0131] Optionally, the construction module 301 is further configured to use an AdamW optimizer to optimize the training in the training of the encoder, the initial learning rate of the encoder is set to 5e-4, the momentum of AdamW is set to 0.9, and the weight decay factor is set to 5e-4; a total of 100 rounds of training are performed, and a cosine decay learning rate schedule is used. The parameters of the trained second encoder can be used as model pre-training parameters for pedestrian re-identification.

[0132] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0133] Figure 4 Schematic diagram of an electronic device 4 provided in an embodiment of the present disclosure. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0134] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 The electronic device 4 is merely an example and does not limit the electronic device 4 , and may include more or less components than those shown in the figure, or different components.

[0135] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0136] The memory 402 may be an internal storage unit of the electronic device 4, for example, a hard disk or memory of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. The memory 402 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0137] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0138] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0139] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be included in the protection scope of the present disclosure.

Claims

1. A pedestrian re-identification method, characterized in that: include: Constructing a pedestrian re-identification model, wherein the pedestrian re-identification model includes: a pedestrian re-identification branch and a quality regression branch; Obtaining a pre-trained model, and initializing the person re-identification model using the pre-trained model; Perform multi-stage training on the initialized person re-identification model; Obtain a pedestrian data set to be classified, wherein the pedestrian data set includes: multiple pictures of multiple people; Using the pedestrian re-identification model trained in multiple stages, classify the images in the pedestrian dataset to obtain one or more images of each person; The multi-stage training of the initialized person re-identification model includes: freezing the quality regression branch, and training the person re-identification branch using the first loss function of the person re-identification model, wherein the interval factor in the first loss function is determined by the quality regression branch, and after freezing the quality regression branch, the interval factor is a constant; freezing the person re-identification branch, and training the quality regression branch using the second loss function that determines the interval factor; and simultaneously training the person re-identification branch and the quality regression branch using the first loss function, wherein if the quality regression branch is not frozen, the interval factor is a variable; Among them, the first loss function L circle : a p =1+ms p D p =1-m D n =m Among them, s p Indicates the similarity between the feature vector of the current image and the positive class center vector of the current image, represents the similarity between the feature vector of the current image and the j-th negative class center vector of the current image, L represents the number of negative classes, γ is the feature scaling factor, m is the interval factor, m is determined by the quality regression branch, if the quality regression branch is frozen, m is a constant, if the quality regression branch is not frozen, m is a variable, and m is equal to the function value of the second loss function; Among them, the second loss function m: Wherein, m0 and m1 are constants, q is the output of the quality regression branch, and the output of the quality regression branch represents the quality score of the pedestrian image, q min is the minimum output of the mass regression branch, q max is the maximum output of the mass regression branch.

2. The method according to claim 1, characterized in that The pedestrian re-identification branch includes the following multiple network layers connected in series: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, and a global pooling and fully connected layer; The number of channels of the first convolutional layer is a first number, and the size of the image after passing through the first convolutional layer becomes one quarter of the original size; The number of channels of the second convolutional layer is a second number, and the size of the image passing through the second convolutional layer remains unchanged; The number of channels of the third convolutional layer is a third number, and the size of the image after passing through the third convolutional layer becomes half of the original size; The number of channels of the fourth convolutional layer is a fourth number, and the size of the image after passing through the fourth convolutional layer becomes half of the original size; The number of channels of the fifth convolutional layer is the fifth number, and the size of the image passing through the fifth convolutional layer becomes half of the original size.

3. The method according to claim 1, characterized in that include: The input of the quality regression branch is the output of the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer of the pedestrian re-identification branch; The quality regression branch includes the following multiple network layers connected in series: a global pooling layer, a feature concatenation layer, a sixth convolutional layer, and a fully connected layer.

4. The method according to claim 1, characterized in that Before obtaining the pre-trained model and initializing the pedestrian re-identification model using the pre-trained model, the method further includes: Constructing the pre-training model, wherein the pre-training model is composed of a first encoder and a second encoder having the same structure; Training the first encoder using a third loss function; The second encoder is trained using a momentum update function, wherein the momentum update function is related to model parameters of the first encoder and the second encoder.

5. A pedestrian re-identification device, characterized in that: include: A construction module is configured to construct a pedestrian re-identification model, wherein the pedestrian re-identification model includes: a pedestrian re-identification branch and a quality regression branch; An initialization module is configured to obtain a pre-trained model and initialize the pedestrian re-identification model using the pre-trained model; A training module, configured to perform multi-stage training on the initialized person re-identification model; An acquisition module is configured to acquire a pedestrian data set to be classified, wherein the pedestrian data set includes: a plurality of pictures of a plurality of persons; A classification module is configured to classify the images in the pedestrian dataset using the pedestrian re-identification model trained in multiple stages to obtain one or more images of each person; The training module is further configured to freeze the quality regression branch, train the pedestrian re-identification branch using the first loss function of the pedestrian re-identification model, wherein the interval factor in the first loss function is determined by the quality regression branch, and after freezing the quality regression branch, the interval factor is a constant; freeze the pedestrian re-identification branch, and train the quality regression branch using the second loss function that determines the interval factor; and simultaneously train the pedestrian re-identification branch and the quality regression branch using the first loss function, wherein if the quality regression branch is not frozen, the interval factor is a variable; The first loss function L circle : a p =1+ms p D p =1-m D n =m Among them, s p Indicates the similarity between the feature vector of the current image and the positive class center vector of the current image, represents the similarity between the feature vector of the current image and the j-th negative class center vector of the current image, L represents the number of negative classes, γ is the feature scaling factor, m is the interval factor, m is determined by the quality regression branch, if the quality regression branch is frozen, m is a constant, if the quality regression branch is not frozen, m is a variable, and m is equal to the function value of the second loss function; The second loss function m: Wherein, m0 and m1 are constants, q is the output of the quality regression branch, and the output of the quality regression branch represents the quality score of the pedestrian image, q min is the minimum output of the mass regression branch, q max is the maximum output of the mass regression branch.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Citation Information

Patent Citations

  • Training method of image target recognition model and image recognition method and device

    CN111598190A

  • Model training method, user identification method, system, device and medium

    CN113327621A

  • Training and recognition method and device based on pedestrian re-recognition, medium and equipment

    CN114419678A

  • Pedestrian re-identification method based on target structure relation enhancement

    CN114529942A

  • Automatic Face Recognition

    US20100232657A1