A pedestrian re-identification method based on a generative adversarial network

Abstract

The application discloses a pedestrian re-identification method based on a generative adversarial network, which comprises four models, namely a feature extraction model, a generative model, a pedestrian image discrimination model and a similarity discrimination model, and steps include randomly sampling pictures from a data set, extracting features, outputting the pictures through the generative model, extracting features from the generated pictures, simultaneously using two discrimination models to judge confidence, and calculating a loss function, repeating the training process until the loss function converges, and using the trained model to judge the similarity of pedestrian images. The application indirectly increases the number of data sets by using the generative adversarial network to train the identification model, and uses a learnable model to calculate the similarity of two features, so that more robust information can be extracted from the features, and the interference of clothes and cameras is excluded.

Description

Technical Field

[0001] This invention relates to the field of image recognition research, specifically to a pedestrian re-identification method based on generative adversarial networks. Background Technology

[0002] In the construction of smart cities, more and more intelligent methods are constantly achieving innovative breakthroughs and being applied to various fields of urban construction. With the widespread use of public cameras, pedestrian re-identification technology has become a popular research direction in artificial intelligence, closely related to public safety. However, pedestrian re-identification technology currently faces many limitations, resulting in less than ideal recognition results: large-scale datasets are difficult to create, images of the same person taken by different cameras vary significantly, and the same person can be seen wearing different clothes, etc. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, this invention establishes a generative adversarial network (GAN) framework by using a feature extraction model, a generation model, a pedestrian image discrimination model, and a similarity discrimination model. Each model is trained using this GAN framework, and two discrimination models are trained simultaneously, thereby achieving a more robust pedestrian re-identification effect. The technical solution is as follows:

[0004] A pedestrian re-identification method based on generative adversarial networks includes a feature extraction model E, a generative model G, a pedestrian image discrimination model C, and a similarity discrimination model D.

[0005] The feature extraction model mainly consists of a backbone network, fully connected layers, and activation layers. The backbone network can be any CNN backbone network, preferably ResNet50. The fully connected layers are responsible for mapping the feature maps of the backbone network to feature vectors, preferably with a feature vector length of 512. The activation layer is tanh.

[0006] The generative model consists of fully connected layers, a backbone network, and activation layers. The fully connected layers are responsible for mapping the input to feature maps. The backbone network can be any mapping from a vector to a feature map of image size. The activation layer is tanh. The random vectors follow a Gaussian distribution with a mean of 0 and a variance of 1.

[0007] The pedestrian image discrimination model consists of a backbone network, fully connected layers, and activation layers. The backbone network can be any CNN backbone network, preferably ResNet18. The fully connected layers are responsible for mapping the feature maps obtained by the backbone network to a floating-point number. Finally, a sigmoid activation layer is used to obtain the probability.

[0008] The similarity discrimination model consists of two fully connected layers plus an activation layer. The two fully connected layers map two features into a floating-point number, and finally, a sigmoid is used as the activation layer to obtain the probability.

[0009] This invention includes the following steps:

[0010] Step 1: Randomly sample two IDs from the dataset, denoted as I1 and I2. Randomly sample two images from I1, denoted as M1 and M2. Randomly sample one image from I2, denoted as M3. Use the feature extraction model E to extract features from M1, M2, and M3 respectively, and obtain features f1 = E(M1), f2 = E(M2), and f3 = E(M3).

[0011] Step 2: Sample a random vector from a K-dimensional Gaussian distribution (K is a predefined parameter, preferably K=128), denoted as noise, and feed (f1, noise) into the generation model G to obtain the generated image F=G(concat(f1,noise)).

[0012] Step 3: Use the feature extraction model E to extract features from F, and obtain feature g = E(F). Use the pedestrian image discrimination model C to judge images M1 and F, and obtain confidence scores s1 = C(M1) and s2 = C(F).

[0013] Step 4: Use the similarity discrimination model D to judge (f1, f2), (f1, f3), and (f1, g) respectively, and obtain the confidence t1 = D(concat(f1, f2)), t2 = D(concat(f1, f3)), t3 = D(concat(f1, g)).

[0014] Step 5, calculate the loss function L

[0015] L = -log(s1)

[0016] -log(1s2)

[0017] -log(t1)

[0018] -log(1t2)

[0019] -log(1t3)

[0020] Step 6: Use backpropagation to train the feature extraction model E, the generation model G, the pedestrian image discrimination model C, and the similarity discrimination model D. Then return to step 1 and repeat the process until the loss function L converges.

[0021] Step 7: Input the two pedestrian images P1 and P2 into the trained feature extraction model E, and extract features q1 = E(P1) and q2 = E(P2) respectively.

[0022] Step 8: Calculate the similarity between q1 and q2 using the trained similarity discrimination model D: sim = D(concat(q1,q2))

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows: by using generative adversarial networks to indirectly increase the amount of dataset to train the recognition model; by using a learnable model to calculate the similarity between two features, more robust information can be extracted from the features, eliminating interference from clothing and cameras. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the network structure of the feature extraction model.

[0025] Figure 2 This is a schematic diagram of the network structure for generating the model.

[0026] Figure 3 This is a schematic diagram of the network structure of a pedestrian image discrimination model.

[0027] Figure 4 This is a schematic diagram of the network structure of the similarity discrimination model. Detailed Implementation

[0028] To clarify the technical solution and working principle of the present invention, the embodiments of this disclosure will be described in further detail below. All the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be elaborated upon here.

[0029] The terms "step 1," "step 2," "step 3," and similar descriptions used in the specification and claims of this application are for distinguishing similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those described herein.

[0030] This disclosure provides a person re-identification method based on generative adversarial networks, which includes four models:

[0031] Feature extraction model E extracts features from the image, such as... Figure 1 As shown. The feature extraction model mainly consists of a backbone network, fully connected layers, and activation layers. The backbone network can be any CNN backbone network, preferably ResNet50. The fully connected layers are responsible for mapping the feature maps of the backbone network to feature vectors, preferably with a feature vector length of 512. The activation layer is tanh.

[0032] Generative model G generates images based on extracted features and random vectors, such as Figure 2As shown, the generative model consists of fully connected layers, a backbone network, and activation layers. The fully connected layers are responsible for mapping the input to feature maps. The backbone network can be any mapping from a vector to a feature map of image size. The activation layer is tanh. The random vectors follow a Gaussian distribution with a mean of 0 and a variance of 1.

[0033] Pedestrian image discrimination model C determines whether an image is a pedestrian image, such as... Figure 3 As shown, the model mainly consists of a backbone network, fully connected layers, and activation layers. The backbone network can be any CNN backbone network, preferably ResNet18. The fully connected layers are responsible for mapping the feature maps obtained from the backbone network to a floating-point number. Finally, a sigmoid function is used as the activation layer to obtain the probability.

[0034] Similarity discrimination model D determines whether two features were generated by the same person, such as Figure 4 As shown, the model mainly consists of two fully connected layers plus an activation layer. The two fully connected layers map two features into a floating-point number, and finally, a sigmoid is used as the activation layer to obtain the probability.

[0035] The specific steps of this method are as follows:

[0036] Step 1: Randomly sample two IDs from the dataset, denoted as I1 and I2. Randomly sample two images from I1, denoted as M1 and M2. Randomly sample one image from I2, denoted as M3. Use the feature extraction model E to extract features from M1, M2, and M3 respectively, and obtain features f1 = E(M1), f2 = E(M2), and f3 = E(M3).

[0037] Step 2: Sample a random vector from a K-dimensional Gaussian distribution (K is a predefined parameter, preferably K=128), denoted as noise, and feed (f1, noise) into the generation model G to obtain the generated image F=G(concat(f1,noise)).

[0038] Step 3: Use the feature extraction model E to extract features from F, and obtain feature g = E(F). Use the pedestrian image discrimination model C to judge images M1 and F, and obtain confidence scores s1 = C(M1) and s2 = C(F).

[0039] Step 4: Use the similarity discrimination model D to judge (f1, f2), (f1, f3), and (f1, g) respectively, and obtain the confidence t1 = D(concat(f1, f2)), t2 = D(concat(f1, f3)), and t3 = D(concat(f1, g)).

[0040] Step 5, calculate the loss function L

[0041] L = -log(s1)

[0042] -log(1-s2)

[0043] -log(t1).

[0044] -log(1-t2)

[0045] -log(1-t3)

[0046] Step 6: Use backpropagation to train the feature extraction model E, the generation model G, the pedestrian image discrimination model C, and the similarity discrimination model D. Then return to step 1 and repeat the process until the loss function L converges.

[0047] Step 7: Input the two pedestrian images P1 and P2 into the trained feature extraction model E, and extract features q1 = E(P1) and q2 = E(P2) respectively.

[0048] Step 8: Use the trained similarity discrimination model D to calculate the similarity between q1 and q2: sim = D(concat(q1,q2)).

[0049] The present invention has been described above by way of example. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other situations without improvement or equivalent substitution, are all within the protection scope of the present invention.

Claims

1. A person re-identification method based on generative adversarial networks, comprising a feature extraction model E, a generation model G, a person image discrimination model C, and a similarity discrimination model D, characterized in that, The feature extraction model consists of a backbone network, fully connected layers, and activation layers; the generation model consists of fully connected layers, a backbone network, and activation layers; the pedestrian image discrimination model consists of a backbone network, fully connected layers, and activation layers; and the similarity discrimination model consists of two fully connected layers plus an activation layer. The two fully connected layers map two features to a floating-point number, and finally, a sigmoid function is used as the activation layer to obtain the probability. This method includes the following steps: Step 1: Randomly sample two IDs from the dataset, denoted as I1 and I2; randomly sample two images from I1, denoted as M1 and M2; and randomly sample one image from I2, denoted as M3. The feature extraction model E is used to extract features from M1, M2, and M3 respectively, resulting in features f1 = E(M1), f2 = E(M2), and f3 = E(M3). Step 2: Sample a random vector from the K-dimensional Gaussian distribution, denoted as noise, and feed (f1, noise) into the generative model G to obtain the generated image F = G(concat(f1, noise)); Step 3: Use the feature extraction model E to extract features from F respectively, and obtain feature g = E(F). Use the pedestrian image discrimination model C to judge images M1 and F respectively, and obtain confidence scores s1 = C(M1) and s2 = C(F). Step 4: Use the similarity discrimination model D to judge (f1, f2), (f1, f3), and (f1, g) respectively, and obtain the confidence scores t1 = D(concat(f1, f2)), t2 = D(concat(f1, f3)), and t3 = D(concat(f1, g)). Step 5, calculate the loss function L L = -log(s1) -log(1-s2) -log(t1); -log(1-t2) -log(1-t3) Step 6: Use backpropagation to train the feature extraction model E, the generation model G, the pedestrian image discrimination model C, and the similarity discrimination model D. Then return to Step 1 and repeat the process until the loss function L converges. Step 7: Input the two pedestrian images P1 and P2 into the trained feature extraction model E, and extract features q1 = E(P1) and q2 = E(P2) respectively; Step 8: Use the trained similarity discrimination model D to calculate the similarity between q1 and q2: sim = D(concat(q1,q2)).

2. The person re-identification method based on generative adversarial networks according to claim 1, characterized in that, The feature extraction model uses ResNet50 as the backbone network; the fully connected layer is responsible for mapping the feature maps of the backbone network to feature vectors, the feature vector length is 512, and the activation layer is tanh.

3. The person re-identification method based on generative adversarial networks according to claim 2, characterized in that, In the generative model, the fully connected layer is responsible for mapping the input to the feature map. The backbone network can be any mapping from a vector to a feature map of image size. The activation layer is tanh, and the random vectors follow a Gaussian distribution with a mean of 0 and a variance of 1.

4. The person re-identification method based on generative adversarial networks according to claim 3, characterized in that, The pedestrian image discrimination model uses ResNet18 as the backbone network.

5. The pedestrian re-identification method based on generative adversarial networks according to claim 4, characterized in that, The K-dimensional Gaussian distribution in step 2 has K = 128.

Images