A domain generalization person re-identification method based on dual meta-learning

Abstract

The present invention discloses a domain generalization pedestrian re-identification method based on a dual meta-learning model, S1: inputting an original image into a meta-training model for training, and calculating the loss of the meta-training stage; S2: inputting an image of a different category from the original image input in S1 into a category meta-test model, calculating the loss of the category meta-test stage, and adding the loss to the loss of the meta-training stage as the total loss of the category meta-test stage; S3: inputting the image input in S2 into a style meta-test model, calculating the loss of the style meta-test stage, and adding the loss to the total loss of the category meta-test stage as the total loss. The beneficial effect of the present invention is that by performing the category meta-test stage and the style meta-test stage respectively, the changes of pedestrian categories and image styles in an unknown domain are fully simulated, thereby improving the generalization performance of the model.

Description

Technical Field

[0001] The present invention belongs to the field of pattern recognition and image retrieval in computer vision, and specifically relates to a pedestrian re-identification method using deep learning for domain generalization, in particular a pedestrian re-identification method based on a dual meta-learning model. Background Art

[0002] As an important part of the construction of a three-dimensional social security prevention and control system, the intelligent video surveillance system has played a vital role in maintaining social security and stability. Up to now, the market size of my country's security industry has reached 800 billion yuan, of which the market size of video surveillance equipment has reached more than 80 billion yuan, becoming the world's largest. The resulting massive video surveillance data is an important foundation for building an intelligent security monitoring system and supporting the digital application of security monitoring. Therefore, it is particularly important to analyze massive video data quickly and accurately. Relying entirely on manual video analysis and processing has many disadvantages such as time-consuming and labor-intensive, low accuracy, etc. In contrast, intelligent video analysis technology can complete the automatic analysis of massive monitoring data and has received more and more attention in the field of computer vision. Person Re-Identification (Re-ID), as an important part of intelligent video analysis, has broad application prospects in the fields of intelligent security and criminal investigation.

[0003] Person re-identification uses computer vision technology to retrieve and match target pedestrians in a cross-view multi-camera surveillance network. It can quickly find target objects from massive video or image libraries, providing strong technical support for social public safety. With the rapid development of convolutional neural networks in the field of computer vision, deep learning has become the mainstream method for person re-identification tasks. At present, the research on person re-identification methods usually focuses on the situation where the distribution of training sets and test sets is consistent, such as using supervised methods to train and test on labeled data sets, or using unsupervised methods to train and test on unlabeled data sets. When the training set and the test set come from the same data set, the existing models often show higher performance. However, in actual environments, due to different shooting scenes, lighting intensities, cameras, etc., there are large data distribution differences between different data sets. These distribution differences will cause the existing models to overfit the training set, and lack generalization ability when applied to the target domain (different from the shooting scene of the test set), resulting in a significant decrease in recognition performance. This is also one of the main reasons why most deep learning methods are difficult to promote.

[0004] Domain generalization person re-identification aims to use one or more source domain data to train a general model that can achieve good recognition performance when tested in other target domains. Since domain generalization person re-identification does not require retraining the model using data from the target domain, it is closer to practical applications. Meta-learning is an effective way to solve the domain generalization problem, but existing domain generalization person re-identification methods based on meta-learning mainly simulate the test process from a single aspect such as category or style, ignoring the completely different changes in pedestrian categories and styles in the unseen target domain, resulting in low generalization recognition performance;

[0005] The patent document with announcement number CN111488797A discloses a pedestrian re-identification method, including steps 1, adjusting the size of the original pedestrian image and performing preprocessing; 2, inputting the preprocessed image into the MEMF model for forward propagation, and outputting four feature vectors; 3, calculating the total loss of a training; 4, performing back propagation according to the total loss, updating and saving the network weights; 5, using the network weights as the initial network weights for the next model training, repeating steps 2-4, and saving the model after the last training; 6, adjusting the size of the retrieval image and the image in the image library and normalizing them; 7, inputting the trained model after normalization, and obtaining four equal-dimensional feature expression vectors for each image; 8, cascading the four feature expression vectors of each image as the final feature expression vector; 9, calculating the Euclidean distance between the final feature expression vectors of the retrieval image and the image in the image library, and sorting them from small to large; 10, outputting the corresponding sequence image as the recognition result according to the sorting, however, it ignores the completely different pedestrian categories and style changes in the invisible target domain, resulting in low generalization recognition performance.

[0006] The patent document with announcement number CN 111881714 A discloses an unsupervised cross-domain pedestrian re-identification method, comprising the following steps: using labeled source domain training images for pre-training to obtain baseline network weights, and using them as the baseline network initial weights in the multi-loss optimization learning training process; using unlabeled target domain training images for multi-loss optimization learning training, and based on the baseline network initial weights, performing multiple multi-loss optimization learning training to obtain a baseline network after multi-loss optimization learning training; using unlabeled target domain test images for unsupervised cross-domain pedestrian re-identification testing, and inputting the unlabeled target domain test images into the baseline network after multi-loss optimization learning training for testing to obtain recognition results. However, it also ignores the completely different pedestrian categories and style changes in the invisible target domain, resulting in low generalization recognition performance. Summary of the invention

[0007] The purpose of the present invention is to overcome the above problems and provide a domain generalization pedestrian re-identification method based on a dual meta-learning model. The present invention focuses on comprehensively simulating the changes in pedestrian categories and styles in unknown domains, thereby improving the generalization performance of the model and having a higher recognition accuracy than other methods in the same field.

[0008] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0009] A domain generalization person re-identification method based on dual meta-learning includes the following steps:

[0010] S1: Input the original image into the meta-training model training and calculate the loss of the meta-training stage;

[0011] S2: Input images of different categories from the original images input in S1 into the category meta-test model, calculate the loss of the category meta-test stage, and add it to the loss of the meta-training stage as the total loss of the category meta-test stage;

[0012] S3: Input the image input in S2 into the style meta-test model, calculate the loss of the style meta-test stage, and add it to the total loss of the category meta-test stage as the total loss;

[0013] The baseline networks of the meta-training model, the category meta-test model, and the style meta-test model include an IBN-Net network.

[0014] As an improvement, the preprocessing steps of the original image are also included, including:

[0015] The size of the original pedestrian image for training is adjusted, and the original pedestrian image for training is preprocessed by random horizontal flipping, random cropping, normalization and random erasing to obtain the original image.

[0016] As a further improvement, step S1 also includes: using the network weights pre-trained on the ImageNet image dataset as the initial baseline network weights of the meta-training model;

[0017] Perform back propagation according to the loss in the meta-training phase, update and save the baseline network weights, and use the saved baseline network weights as the initial baseline network weights of the category meta-test model;

[0018] Step S2 also includes: performing back propagation according to the total loss in the category meta-test phase, updating and saving the baseline network weights, and using the saved baseline network weights as the initial baseline network weights of the style meta-test model;

[0019] Step S3 also includes: performing back propagation according to the total loss, updating and saving the baseline network weights, and using the saved baseline network weights as the initial baseline network weights of the meta-training model.

[0020] As further improvements, it also includes:

[0021] S4: Repeat steps S1-S3 to make the total loss approach zero until the network converges, and save the meta-training model after the last training.

[0022] As a further improvement, the S1 step further includes:

[0023] Input the original image of size 3×256×128 into the first layer of the IBN-Net network of the meta-training model, and output the meta-training feature map of size 256×128×64;

[0024] The meta-training feature map of size 256×128×64 is input to the second layer of the IBN-Net network of the meta-training model, and the meta-training feature map of size 512×64×32 is output;

[0025] The meta-training feature map with a size of 512×64×32 is input to the third layer of the IBN-Net network of the meta-training model, and the meta-training feature map with a size of 1024×32×16 is output;

[0026] The meta-training feature map with a size of 1024×32×16 is input to the fourth layer of the IBN-Net network of the meta-training model, and the meta-training feature map with a size of 2048×16×8 is output;

[0027] The meta-trained feature map of size 2048×16×8 is input to the global average pooling layer, which outputs a feature vector of size 2048×1×1;

[0028] A feature vector of size 2048×1×1 is input to a classifier, which consists of a fully connected layer and outputs a feature vector of the same dimension as the number of pedestrian categories in the training dataset.

[0029] As a further improvement, the S2 step further includes:

[0030] Input an image of a different category from the original image input in S1 and having a size of 3×256×128 into the first layer of the IBN-Net network of the category meta-test model, and output a category meta-test feature map of a size of 256×128×64;

[0031] The category meta-test feature map with a size of 256×128×64 is input to the second layer of the IBN-Net network of the category meta-test model, and the category meta-test feature map with a size of 512×64×32 is output;

[0032] The category meta-test feature map with a size of 512×64×32 is input to the third layer of the IBN-Net network of the category meta-test model, and the category meta-test feature map with a size of 1024×32×16 is output;

[0033] The category meta-test feature map with a size of 1024×32×16 is input to the fourth layer of the IBN-Net network of the category meta-test model, and the category meta-test feature map with a size of 2048×16×8 is output;

[0034] The class meta-test feature map of size 2048×16×8 is input to the global average pooling layer, and a feature vector of size 2048×1×1 is output;

[0035] A feature vector of size 2048×1×1 is input into a classifier, which consists of a fully connected layer and outputs a feature vector of the same dimension as the number of pedestrian categories in the training dataset.

[0036] As a further improvement, the S3 step further includes:

[0037] The image input in S2 with a size of 3×256×128 is input to the first layer of the IBN-Net network of the style meta-test model, and the style meta-test feature map with a size of 256×128×64 is output;

[0038] Input the style element test feature map of size 256×128×64 into the style enhancement module based on Fourier transform to obtain the style element test feature map of size 256×128×64 with changed pixel values;

[0039] The style meta-test feature map with a size of 256×128×64 with changed pixel values ​​is input to the second layer of the IBN-Net network, and the style meta-test feature map with a size of 512×64×32 is output;

[0040] Input the style element test feature map of size 512×64×32 into the style enhancement module based on Fourier transform to obtain the style element test feature map of size 512×64×32 with changed pixel values;

[0041] The style meta-test feature map with a size of 512×64×32 is input to the third layer of the IBN-Net network of the style meta-test model, and the style meta-test feature map with a size of 1024×32×16 is output;

[0042] Input the style element test feature map of size 1024×32×16 into the style enhancement module based on Fourier transform to obtain the style element test feature map of size 1024×32×16 with changed pixel values;

[0043] The style meta-test feature map with a size of 1024×32×16 is input to the fourth layer of the IBN-Net network of the style meta-test model, and the style meta-test feature map with a size of 2048×16×8 is output;

[0044] The style meta-test feature map of size 2048×16×8 is input to the global average pooling layer, and a feature vector of size 2048×1×1 is output;

[0045] The feature vector of size 2048×1×1 is input to the classifier, which consists of a fully connected layer and outputs a feature vector with the same dimension as the number of pedestrian categories in the training dataset.

[0046] As a further improvement, the calculation of the loss in the meta-training stage specifically includes: calculating the cross entropy loss value between the feature vector output in step S1 and the number of pedestrian categories in the training data set; calculating the triplet loss value of the feature vector output in step S1 and the number of pedestrian categories in the training data set in a training batch; summing the obtained cross entropy loss value and the triplet loss value as the loss in the meta-training stage;

[0047] The loss of the category meta-test stage specifically includes: calculating the cross entropy loss value between the feature vector output in step S2 and the number of pedestrian categories in the training data set; calculating the triplet loss value of the feature vector output in step S2 and the number of pedestrian categories in the training data set in a training batch; summing the obtained cross entropy loss value and triplet loss value as the loss of the category meta-test stage;

[0048] The loss of the style element testing phase specifically includes: calculating the cross entropy loss value between the feature vector output in step S3 and the number of pedestrian categories in the training data set; calculating the triplet loss value of the feature vector output in step S3 and the number of pedestrian categories in the training data set in a training batch; and summing the obtained cross entropy loss value and the triplet loss value as the loss of the style element testing phase.

[0049] As further improvements, it also includes:

[0050] S5: Adjust the size of the retrieved images and the images in the image library and normalize them.

[0051] S6: The normalized retrieval images and images in the image library are respectively input into the meta-training model after the last training, and each image obtains a feature expression vector as the final feature expression vector of the image.

[0052] S7: Given a retrieval image, calculate the Euclidean distance between the final feature expression vectors of the retrieval image and the image library image, and sort them from small to large.

[0053] S8: According to the sorting order of the Euclidean distance, the corresponding sequence image is output as the recognition result.

[0054] On the other hand, the present invention also provides a domain generalization pedestrian re-identification system based on dual meta-learning, the system comprising at least one processor and a memory storing instructions, which, when executed by at least one processor, implements the method in the above technical solution.

[0055] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method described in the steps of the above technical solution.

[0056] A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method described in the steps of the above technical solution is implemented.

[0057] The beneficial effects of the present invention are:

[0058] The present invention innovatively expands single-stage meta-learning into two-stage meta-learning. By performing the category meta-test stage and the style meta-test stage respectively, it fully simulates the changes in pedestrian categories and image styles in unknown domains and improves the generalization performance of the model. The present invention also proposes a style enhancement network based on Fourier transform, which can randomly change the style of an image without changing the semantic information of the image. The present invention is applied to the fields of surveillance security and criminal investigation, and can be better generalized to different video surveillance data, find the target location, reduce the workload of relevant personnel, and improve the accuracy and efficiency of intelligent video analysis and processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The present invention has the following accompanying drawings:

[0060] Figure 1 are flow charts of some specific embodiments of the present invention;

[0061] Figure 2 are flow charts of some specific embodiments of the present invention;

[0062] Figure 3 It is a flow chart of a style enhancement module based on Fourier transform in a domain generalization person re-identification method based on dual meta-learning of the present invention;

[0063] Figure 4 A system schematic diagram of a domain generalization person re-identification system based on dual meta-learning in the present invention. DETAILED DESCRIPTION

[0064] The following examples further illustrate the content of the present invention, but should not be construed as limiting the present invention. Without departing from the spirit and substance of the present invention, modifications or substitutions made to the methods, steps or conditions of the present invention all fall within the scope of the present invention.

[0065] Embodiment 1:

[0066] like Figure 1 As shown, the present invention provides a domain generalization pedestrian re-identification method based on a dual meta-learning model, which specifically includes the following steps: S1: inputting the original image into the meta-training model training, and calculating the loss of the meta-training stage;

[0067] S2: Input images of different categories from the original images input in S1 into the category meta-test model, calculate the loss of the category meta-test stage, and add it to the loss of the meta-training stage as the total loss of the category meta-test stage;

[0068] It should be noted that each image of the original image has a corresponding pedestrian category, and images of different pedestrian categories are images of different categories from the original image input in S1.

[0069] S3: Input the image input in S2 into the style meta-test model, calculate the loss of the style meta-test stage, and add it to the total loss of the category meta-test stage as the total loss;

[0070] The baseline networks of the meta-training model, the category meta-testing model, and the style meta-testing model include the IBN-Net network.

[0071] It should be noted that by adding a category meta-test model and a style meta-test model to perform the category meta-test stage and the style meta-test stage respectively, the changes in pedestrian categories and image styles in the unknown domain can be fully simulated to improve the generalization performance of the model.

[0072] In some specific embodiments, Figure 2 As shown, the preprocessing steps of the original image are also included, including:

[0073] The size of the original pedestrian image for training is adjusted, and the original pedestrian image for training is preprocessed by random horizontal flipping, random cropping, normalization and random erasing to obtain the original image.

[0074] In some specific embodiments, Figure 2 As shown, step S1 also includes: using the network weights pre-trained on the ImageNet image dataset as the initial baseline network weights of the meta-training model;

[0075] Perform back propagation based on the loss in the meta-training phase, update and save the baseline network weights, and use the saved baseline network weights as the initial baseline network weights for the category meta-test model;

[0076] Step S2 also includes: performing back propagation according to the total loss in the category meta-test phase, updating and saving the baseline network weights, and using the saved baseline network weights as the initial baseline network weights of the style meta-test model;

[0077] Step S3 also includes: performing back propagation according to the total loss, updating and saving the baseline network weights, and using the saved baseline network weights as the initial baseline network weights of the meta-training model.

[0078] In some specific embodiments, it also includes:

[0079] S4: Repeat steps S1-S3 to make the total loss approach zero until the network converges, and save the meta-training model after the last training.

[0080] In some specific embodiments, Figure 2 As shown, step S1 also includes:

[0081] The original image of size 3×256×128 is input to the first layer of the IBN-Net network of the meta-training model, and the meta-training feature map of size 256×128×64 is output;

[0082] The meta-training feature map of size 256×128×64 is input to the second layer of the IBN-Net network of the meta-training model, and the meta-training feature map of size 512×64×32 is output;

[0083] The meta-training feature map with a size of 512×64×32 is input to the third layer of the IBN-Net network of the meta-training model, and the meta-training feature map with a size of 1024×32×16 is output;

[0084] The meta-training feature map with a size of 1024×32×16 is input to the fourth layer of the IBN-Net network of the meta-training model, and the meta-training feature map with a size of 2048×16×8 is output;

[0085] The meta-trained feature map of size 2048×16×8 is input to the global average pooling layer, which outputs a feature vector of size 2048×1×1;

[0086] The feature vector of size 2048×1×1 is input to the classifier, which consists of a fully connected layer and outputs a feature vector with the same dimension as the number of pedestrian categories in the training dataset.

[0087] In some specific embodiments, Figure 2As shown, step S2 also includes:

[0088] Input images of different categories with a size of 3×256×128 and the original image input in S1 into the first layer of the IBN-Net network of the category meta-test model, and output a category meta-test feature map with a size of 256×128×64;

[0089] The category meta-test feature map with a size of 256×128×64 is input to the second layer of the IBN-Net network of the category meta-test model, and the category meta-test feature map with a size of 512×64×32 is output;

[0090] The category meta-test feature map with a size of 512×64×32 is input to the third layer of the IBN-Net network of the category meta-test model, and the category meta-test feature map with a size of 1024×32×16 is output;

[0091] The category meta-test feature map with a size of 1024×32×16 is input to the fourth layer of the IBN-Net network of the category meta-test model, and the category meta-test feature map with a size of 2048×16×8 is output;

[0092] The class meta-test feature map of size 2048×16×8 is input to the global average pooling layer, and a feature vector of size 2048×1×1 is output;

[0093] A feature vector of size 2048×1×1 is input into the classifier, which consists of a fully connected layer and outputs a feature vector of the same dimension as the number of pedestrian categories in the training dataset.

[0094] In some specific embodiments, Figure 2 As shown, step S3 also includes:

[0095] The image input in S2 with a size of 3×256×128 is input to the first layer of the IBN-Net network of the style meta-test model, and the style meta-test feature map with a size of 256×128×64 is output;

[0096] Input the style element test feature map of size 256×128×64 into the style enhancement module based on Fourier transform to obtain the style element test feature map of size 256×128×64 with changed pixel values;

[0097] The style meta-test feature map with a size of 256×128×64 with changed pixel values ​​is input to the second layer of the IBN-Net network, and the style meta-test feature map with a size of 512×64×32 is output;

[0098] Input the style element test feature map of size 512×64×32 into the style enhancement module based on Fourier transform to obtain the style element test feature map of size 512×64×32 with changed pixel values;

[0099] The style meta-test feature map with a size of 512×64×32 is input to the third layer of the IBN-Net network of the style meta-test model, and the style meta-test feature map with a size of 1024×32×16 is output;

[0100] Input the style element test feature map of size 1024×32×16 into the style enhancement module based on Fourier transform to obtain the style element test feature map of size 1024×32×16 with changed pixel values;

[0101] The style meta-test feature map with a size of 1024×32×16 is input to the fourth layer of the IBN-Net network of the style meta-test model, and the style meta-test feature map with a size of 2048×16×8 is output;

[0102] The style meta-test feature map of size 2048×16×8 is input to the global average pooling layer, and a feature vector of size 2048×1×1 is output;

[0103] The feature vector of size 2048×1×1 is input to the classifier, which consists of a fully connected layer and outputs a feature vector with the same dimension as the number of pedestrian categories in the training dataset.

[0104] In some specific embodiments, Figure 2 As shown, calculating the loss of the meta-training stage specifically includes: calculating the cross entropy loss value between the feature vector output in step S1 and the number of pedestrian categories in the training data set; calculating the triplet loss value of the feature vector output in step S1 and the number of pedestrian categories in the training data set in a training batch; summing the obtained cross entropy loss value and triplet loss value as the loss of the meta-training stage;

[0105] The loss of the category meta-test stage specifically includes: calculating the cross entropy loss value between the feature vector output in step S2 and the number of pedestrian categories in the training data set; calculating the triplet loss value of the feature vector output in step S2 and the number of pedestrian categories in the training data set in a training batch; summing the obtained cross entropy loss value and triplet loss value as the loss of the category meta-test stage;

[0106] The loss of the style meta test phase specifically includes: calculating the cross entropy loss value between the feature vector output in step S3 and the number of pedestrian categories in the training data set; calculating the triplet loss value of the feature vector output in step S3 and the number of pedestrian categories in the training data set in a training batch; and summing the obtained cross entropy loss value and triplet loss value as the loss of the style meta test phase.

[0107] In some specific embodiments, Figure 2 and Figure 3 As shown, a style meta-test feature map of size 256×128×64 is input into a style enhancement module based on Fourier transform to obtain a style meta-test feature map of size 256×128×64 with changed pixel values. Specifically, the style meta-test feature map of size 256×128×64 is input into the style enhancement module, and a fast Fourier transform is performed to obtain a phase spectrum P(u, v) and an amplitude spectrum A(u, v). The amplitude spectrum A(u, v) is randomly perturbed to obtain a transformed amplitude spectrum A"(u, v). An inverse fast Fourier transform is performed on the phase spectrum P(u, v) and the transformed amplitude spectrum A"(u, v) to obtain a style meta-test feature map of size 256×128×64 with changed pixel values.

[0108] In some specific embodiments, Figure 2 and Figure 3 As shown, a style element test feature map of size 512×64×32 is input into a style enhancement module based on Fourier transform to obtain a style element test feature map of size 512×64×32 with changed pixel values. Specifically, the style element test feature map of size 512×64×32 is input into the style enhancement module, and a fast Fourier transform is performed to obtain a phase spectrum P(u, v) and an amplitude spectrum A(u, v), and the amplitude spectrum A(u, v) is randomly perturbed to obtain a transformed amplitude spectrum A"(u, v), and an inverse fast Fourier transform is performed on the phase spectrum P(u, v) and the transformed amplitude spectrum A"(u, v) to obtain a style element test feature map of size 512×64×32 with changed pixel values.

[0109] In some specific embodiments, Figure 2 and Figure 3As shown, a style element test feature map of size 1024×32×16 is input into a style enhancement module based on Fourier transform to obtain a style element test feature map of size 1024×32×16 with changed pixel values. Specifically, the style element test feature map of size 1024×32×16 is input into the style enhancement module, and a fast Fourier transform is performed to obtain a phase spectrum P(u, v) and an amplitude spectrum A(u, v), and the amplitude spectrum A(u, v) is randomly perturbed to obtain a transformed amplitude spectrum A"(u, v), and an inverse fast Fourier transform is performed on the phase spectrum P(u, v) and the transformed amplitude spectrum A"(u, v) to obtain a style element test feature map of size 1024×32×16 with changed pixel values.

[0110] It should be noted that the style enhancement network based on Fourier transform can randomly change the style of the image without changing the semantic information of the image.

[0111] In some specific embodiments, Figure 1 As shown, it also includes:

[0112] S5: Adjust the size of the retrieved images and the images in the image library and normalize them.

[0113] S6: The normalized retrieval images and images in the image library are respectively input into the meta-training model after the last training, and each image obtains a feature expression vector as the final feature expression vector of the image.

[0114] S7: Given a retrieval image, calculate the Euclidean distance between the final feature expression vectors of the retrieval image and the image library image, and sort them from small to large.

[0115] S8: According to the sorting order of the Euclidean distance, the corresponding sequence image is output as the recognition result.

[0116] It should be noted that the cross entropy loss value is defined as where y j Represents the real pedestrian category, To predict the pedestrian category, each of them is a vector with the same dimension as the number of pedestrian categories in the training dataset, and N is the number of pedestrians; the triplet loss is defined as Where δ is the distance hyperparameter, d p,i and d n,i are the distances between the positive sample pair and the negative sample pair, respectively, and B is the number of samples in a training batch; the loss in the meta-training phase is L = L CEL +L Tri .

[0117] It should be noted that the output of the global average pooling layer is m, n are the length and width of the meta-training feature map, the category meta-test feature map, or the style meta-test feature map, respectively. i is the pixel value of the corresponding point in the meta-training feature map, the category meta-test feature map, or the style meta-test feature map, with a total of m×n pixel values.

[0118] It should be noted that the Euclidean distance between the final feature expression vectors of the retrieved image and the image library image is Q and G are the final feature expression vectors of the retrieved image and the image library image, respectively. p , g p are the values ​​of the corresponding points in the final feature expression vectors of the retrieved image and the image library image respectively, and P is the dimension of the final feature expression vector.

[0119] Embodiment 2:

[0120] A domain generalization person re-identification method based on a dual meta-learning model is implemented through the following steps:

[0121] The size of the original pedestrian image for training is adjusted to 256×128, and the image is preprocessed by horizontal flipping, random cropping, normalization and random erasing. The pixel value of the image is normalized to [-1.0, 1.0] to obtain the original image.

[0122] The original image is input into the meta-training model for forward propagation, the IBN-Net network is used as the baseline network, the network weights pre-trained on the ImageNet image dataset are used as the initial weights of the baseline network, and a feature vector with the same dimension as the number of pedestrian categories in the training dataset is output. The specific method is as follows:

[0123] Input the image of size 3×256×128 into the first layer of the IBN-Net network of the meta-training model, and output the meta-training feature map of size 256×128×64;

[0124] Input the meta-training feature map of size 256×128×64 into the second layer of the IBN-Net network of the meta-training model, and output the meta-training feature map of size 512×64×32;

[0125] Input the meta-training feature map of size 512×64×32 into the third layer of the IBN-Net network of the meta-training model, and output the meta-training feature map of size 1024×32×16;

[0126] The meta-training feature map with a size of 1024×32×16 is input into the fourth layer of the IBN-Net network of the meta-training model, and the meta-training feature map with a size of 2048×16×8 is output;

[0127] The meta-training feature map of size 2048×16×8 is input into the global average pooling layer, and a feature vector of size 2048×1×1 is output; the output of the global average pooling layer is m, n are the length and width of the meta-training feature map of size 2048×16×8, respectively. i is the pixel value of the corresponding point in the meta-training feature map of size 2048×16×8, with a total of m×n pixel values.

[0128] The feature vector with a size of 2048×1×1 is input into the classifier, which consists of a fully connected layer and outputs a feature vector with the same dimension as the number of pedestrian categories in the training dataset.

[0129] Calculate the cross entropy loss value between the output feature vector with the same dimension as the number of pedestrian categories in the training data set and the pedestrian category; calculate the triplet loss value of the output feature vector with the same dimension as the number of pedestrian categories in the training data set in a training batch, and sum the obtained cross entropy loss value and triplet loss value as the loss of the meta-training stage;

[0130] The cross entropy loss is defined as where y j Represents the real pedestrian category, To predict the pedestrian category, each of them is a vector with the same dimension as the number of pedestrian categories in the training dataset, and N is the number of pedestrians; the triplet loss is defined as Where δ is the distance hyperparameter, d p,i and d n,i are the distances between positive and negative sample pairs, respectively, and B is the number of samples in a training batch; the loss in the meta-training phase is L = L CEL +L Tri .

[0131] Perform backpropagation based on the loss obtained in the meta-training phase to update and save the baseline network weights;

[0132] The network weights saved in the previous steps are used as the initial baseline network weights of the category meta-test model, and the IBN-Net network is used as the baseline network. The images of different categories of the input original image are input into the category meta-test model for forward propagation, and a feature vector with the same dimension as the number of pedestrian categories in the training dataset is output. The purpose is to simulate different pedestrian categories in the unknown domain and improve the generalization of the model to the unknown domain from the perspective of categories. The specific method is:

[0133] Input the image of size 3×256×128 into the first layer of the IBN-Net network of the category meta-test model, and output the category meta-test feature map of size 256×128×64;

[0134] Input the category meta-test feature map of size 256×128×64 into the second layer of the IBN-Net network of the category meta-test model, and output the category meta-test feature map of size 512×64×32;

[0135] Input the category meta-test feature map of size 512×64×32 into the third layer of the IBN-Net network of the category meta-test model, and output the category meta-test feature map of size 1024×32×16;

[0136] Input the category meta-test feature map of size 1024×32×16 into the fourth layer of the IBN-Net network of the category meta-test model, and output the category meta-test feature map of size 2048×16×8;

[0137] The class meta-test feature map of size 2048×16×8 is input into the global average pooling layer, and a feature vector of size 2048×1×1 is output;

[0138] A feature vector of size 2048×1×1 is input into the classifier, which consists of a fully connected layer and outputs a feature vector of the same dimension as the number of pedestrian categories in the training dataset.

[0139] Calculate the cross entropy loss value between the feature vector with the same dimension as the number of pedestrian categories in the training data set outputted in the above step and the pedestrian category; calculate the triplet loss value of the feature vector with the same dimension as the number of pedestrian categories in the training data set outputted in a training batch, sum the obtained cross entropy loss value and triplet loss value as the loss of the category meta-test stage, add it to the loss of the meta-training stage as the total loss of the category meta-test stage, perform back propagation, and update and save the baseline network weights;

[0140] The network weights saved in the previous steps are used as the initial baseline network weights of the style meta-test model, and the IBN-Net network is used as the baseline network. The same image as the input image of the category meta-test model is input into the style meta-test model for forward propagation, and a feature vector with the same dimension as the number of pedestrian categories in the training dataset is output. The purpose is to simulate different pedestrian image styles in unknown domains and improve the generalization of the model to unknown domains from the perspective of style. The specific method is:

[0141] Input an image of size 3×256×128 into the first layer of the IBN-Net network of the style meta-test model, and output a style meta-test feature map of size 256×128×64;

[0142] Input the style element test feature map of size 256×128×64 into the style enhancement module based on Fourier transform to obtain the style element test feature map of size 256×128×64 with changed pixel values;

[0143] The second layer of the IBN-Net network of the style meta test model with a pixel value of 256×128×64 is changed to output a style meta test feature map of size 512×64×32;

[0144] Input the style element test feature map of size 512×64×32 into the style enhancement module based on Fourier transform to obtain the style element test feature map of size 512×64×32 with changed pixel values;

[0145] Input the style meta-test feature map of size 512×64×32 into the third layer of the IBN-Net network of the style meta-test model, and output the style meta-test feature map of size 1024×32×16;

[0146] Input the style element test feature map of size 1024×32×16 into the style enhancement module based on Fourier transform to obtain the style element test feature map of size 1024×32×16 with changed pixel values;

[0147] Input the style meta-test feature map of size 1024×32×16 into the fourth layer of the IBN-Net network of the style meta-test model, and output the style meta-test feature map of size 2048×16×8;

[0148] The style element test feature map of size 2048×16×8 is input into the global average pooling layer, and a feature vector of size 2048×1×1 is output;

[0149] A feature vector of size 2048×1×1 is input into the classifier, which consists of a fully connected layer and outputs a feature vector of the same dimension as the number of pedestrian categories in the training dataset.

[0150] Calculate the cross entropy loss value between the feature vector with the same dimension as the number of pedestrian categories in the training data set outputted in the above step and the pedestrian category; calculate the triplet loss value of the feature vector with the same dimension as the number of pedestrian categories in the training data set outputted in a training batch, sum the obtained cross entropy loss value and triplet loss value as the loss of the style meta-test stage, add it to the total loss of the category meta-test stage as the total loss of the model, perform back propagation, and update and save the baseline network weights;

[0151] The network weights saved in the above steps are used as the initial baseline network weights of the meta-training model. The steps except preprocessing are repeated for a total of 80 iterations, and the meta-training model after the last training is saved.

[0152] The meta-trained model after the last training is used for the test process of pedestrian re-identification, which specifically includes the following steps:

[0153] Read in the search image, resize it to 256×128, and normalize the pixel values ​​to [-1.0, 1.0]. Read in the image from the image library and perform the same processing.

[0154] The normalized retrieval images and images in the image library are respectively input into the multi-level perturbation model after the last training, and each image obtains a feature expression vector.

[0155] The obtained feature expression vectors are concatenated as the final feature expression vector of an image.

[0156] Given a retrieval image, calculate the Euclidean distance between the final feature expression vectors of the retrieval image and the image library image Q and G are the final feature expression vectors of the retrieved image and the image library image, respectively. p , g p are the values ​​of the corresponding points in the final feature expression vectors of the retrieved image and the image library image, respectively, and P is the dimension of the final feature expression vector. Sort the Euclidean distances from small to large.

[0157] Step 14: According to the sorting order of the Euclidean distance, the corresponding sequence image is output as the recognition result.

[0158] In some specific embodiments, Figure 4 As shown, a domain generalization pedestrian re-identification system based on dual meta-learning is provided, and the system includes at least one processor and a memory storing instructions to implement any one of the domain generalization pedestrian re-identification methods based on dual meta-learning in the above technical solutions.

[0159] In some specific embodiments, a computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in the above technical solution.

[0160] In some specific embodiments, a computer program product is provided, the computer program product includes a computer program, and when the computer program is executed by a processor, the method of implementing the steps in the above technical solution is implemented.

[0161] The embodiments and functional operations of the subject matter described in this specification may be implemented in: digital electronic circuitry, tangibly implemented computer software or firmware, computer hardware, including the structures disclosed in this specification and their structural equivalents, or a combination of more than one of the above. The embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on one or more tangible non-transitory program carriers, for execution by a data processing device or to control the operation of a data processing device.

[0162] Alternatively or additionally, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information for transmission to an appropriate receiver device for execution by a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of the above devices.

[0163] A computer program (which may also be referred to or described as a program, software, software application, module, software module, script or code) may be written in any form of programming language, including compiled or interpreted languages ​​or declarative or procedural languages, and the computer program may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that stores other programs or data, for example, one or more scripts stored in the following: in a markup language document; in a single file dedicated to a related program; or in multiple collaborative files, for example, files storing one or more modules, subroutines or code portions. A computer program may be deployed to execute on one or more computers, the computers being located in one place or distributed to multiple locations and interconnected by a communication network.

[0164] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

[0165] Although the present invention has been described in detail above by general description, specific implementation methods and experiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A domain generalization person re-identification method based on dual meta-learning, characterized in that: The steps include: S1: Input the original image into the meta-training model training and calculate the loss of the meta-training stage; S2: Input images of different categories from the original images input in S1 into the category meta-test model, calculate the loss of the category meta-test stage, and add it to the loss of the meta-training stage as the total loss of the category meta-test stage; S3: Input the image input in S2 into the style meta-test model, calculate the loss of the style meta-test stage, and add it to the total loss of the category meta-test stage as the total loss; S4: Repeat steps S1-S3 to make the total loss approach zero until the network converges, and save the meta-training model after the last training; S5: adjusting the size of the retrieved image and the image in the image library, and normalizing them; S6: input the normalized retrieval image and the images in the image library into the meta-training model after the last training respectively, and obtain a feature expression vector for each image as the final feature expression vector of the image; S7: Given a retrieval image, calculate the Euclidean distance between the final feature expression vectors of the retrieval image and the image library image, and sort them from small to large; S8: Outputting the corresponding sequence images as the recognition result according to the sorting order of the Euclidean distance; The step S3 further includes: inputting the image input in S2 with a size of 3×256×128 into the first layer of the IBN-Net network of the style meta test model, and outputting a style meta test feature map with a size of 256×128×64; Input the style element test feature map of size 256×128×64 into the style enhancement module based on Fourier transform to obtain the style element test feature map of size 256×128×64 with changed pixel values; The style meta-test feature map with a size of 256×128×64 with changed pixel values ​​is input to the second layer of the IBN-Net network, and the style meta-test feature map with a size of 512×64×32 is output; Input the style element test feature map of size 512×64×32 into the style enhancement module based on Fourier transform to obtain the style element test feature map of size 512×64×32 with changed pixel values; The style meta-test feature map with a size of 512×64×32 is input to the third layer of the IBN-Net network of the style meta-test model, and the style meta-test feature map with a size of 1024×32×16 is output; Input the style element test feature map of size 1024×32×16 into the style enhancement module based on Fourier transform to obtain the style element test feature map of size 1024×32×16 with changed pixel values; The style meta-test feature map with a size of 1024×32×16 is input to the fourth layer of the IBN-Net network of the style meta-test model, and the style meta-test feature map with a size of 2048×16×8 is output; The style meta-test feature map of size 2048×16×8 is input to the global average pooling layer, and a feature vector of size 2048×1×1 is output; The feature vector of size 2048×1×1 is input to the classifier, which consists of a fully connected layer and outputs a feature vector of the same dimension as the number of pedestrian categories in the training dataset; The Fourier transform-based style enhancement module is as follows: inputting the style element test feature map into the style enhancement module, performing fast Fourier transform to obtain a phase spectrum P(u, v) and an amplitude spectrum A(u, v), performing random perturbation on the amplitude spectrum A(u, v) to obtain a transformed amplitude spectrum A"(u, v), and performing inverse fast Fourier transform on the phase spectrum P(u, v) and the transformed amplitude spectrum A"(u, v) to obtain a style element test feature map with changed pixel values; The baseline network of the meta-training model, the category meta-test model, and the style meta-test model is the IBN-Net network.

2. According to claim 1, a domain generalization pedestrian re-identification method based on dual meta-learning is characterized in that: It also includes the preprocessing steps of the original image, including: The size of the original pedestrian image for training is adjusted, and the original pedestrian image for training is preprocessed by random horizontal flipping, random cropping, normalization and random erasing to obtain the original image.

3. The domain generalization person re-identification method based on dual meta-learning according to claim 1, characterized in that: Step S1 also includes: using the network weights pre-trained on the ImageNet image dataset as the initial baseline network weights of the meta-training model; Perform back propagation according to the loss in the meta-training phase, update and save the baseline network weights, and use the saved baseline network weights as the initial baseline network weights of the category meta-test model; Step S2 also includes: performing back propagation according to the total loss in the category meta-test phase, updating and saving the baseline network weights, and using the saved baseline network weights as the initial baseline network weights of the style meta-test model; Step S3 also includes: performing back propagation according to the total loss, updating and saving the baseline network weights, and using the saved baseline network weights as the initial baseline network weights of the meta-training model.

4. The domain generalization person re-identification method based on dual meta-learning according to claim 3 is characterized in that: The S1 step further includes: Input the original image of size 3×256×128 into the first layer of the IBN-Net network of the meta-training model, and output the meta-training feature map of size 256×128×64; The meta-training feature map of size 256×128×64 is input to the second layer of the IBN-Net network of the meta-training model, and the meta-training feature map of size 512×64×32 is output; The meta-training feature map with a size of 512×64×32 is input to the third layer of the IBN-Net network of the meta-training model, and the meta-training feature map with a size of 1024×32×16 is output; The meta-training feature map with a size of 1024×32×16 is input to the fourth layer of the IBN-Net network of the meta-training model, and the meta-training feature map with a size of 2048×16×8 is output; The meta-trained feature map of size 2048×16×8 is input to the global average pooling layer, which outputs a feature vector of size 2048×1×1; A feature vector of size 2048×1×1 is input to a classifier, which consists of a fully connected layer and outputs a feature vector of the same dimension as the number of pedestrian categories in the training dataset.

5. The domain generalization person re-identification method based on dual meta-learning according to claim 4, characterized in that: The S2 step further includes: Input an image of a different category from the original image input in S1 and having a size of 3×256×128 into the first layer of the IBN-Net network of the category meta-test model, and output a category meta-test feature map of a size of 256×128×64; The category meta-test feature map with a size of 256×128×64 is input to the second layer of the IBN-Net network of the category meta-test model, and the category meta-test feature map with a size of 512×64×32 is output; The category meta-test feature map with a size of 512×64×32 is input to the third layer of the IBN-Net network of the category meta-test model, and the category meta-test feature map with a size of 1024×32×16 is output; The category meta-test feature map with a size of 1024×32×16 is input to the fourth layer of the IBN-Net network of the category meta-test model, and the category meta-test feature map with a size of 2048×16×8 is output; The class meta-test feature map of size 2048×16×8 is input to the global average pooling layer, and a feature vector of size 2048×1×1 is output; A feature vector of size 2048×1×1 is input into a classifier, which consists of a fully connected layer and outputs a feature vector of the same dimension as the number of pedestrian categories in the training dataset.

6. The domain generalization person re-identification method based on dual meta-learning according to claim 3, characterized in that: Also includes: The calculation of the loss in the meta-training stage specifically includes: calculating the cross entropy loss value between the feature vector output in step S1 and the number of pedestrian categories in the training data set; calculating the triplet loss value of the feature vector output in step S1 and the number of pedestrian categories in the training data set in a training batch; summing the obtained cross entropy loss value and triplet loss value as the loss in the meta-training stage; The loss of the category meta-test stage specifically includes: calculating the cross entropy loss value between the feature vector output in step S2 and the number of pedestrian categories in the training data set; calculating the triplet loss value of the feature vector output in step S2 and the number of pedestrian categories in the training data set in a training batch; summing the obtained cross entropy loss value and triplet loss value as the loss of the category meta-test stage; The loss of the style element testing phase specifically includes: calculating the cross entropy loss value between the feature vector output in step S3 and the number of pedestrian categories in the training data set; calculating the triplet loss value of the feature vector output in step S3 and the number of pedestrian categories in the training data set in a training batch; and summing the obtained cross entropy loss value and the triplet loss value as the loss of the style element testing phase.

7. A domain generalization pedestrian re-identification system based on dual meta-learning, the system comprising at least one processor and a memory storing instructions, which, when executed by at least one processor, implements a domain generalization pedestrian re-identification method based on dual meta-learning as described in any one of claims 1-6.

Images