A cross-domain re-identification method based on shallow texture extraction and related equipment

Abstract

The application discloses a cross-domain re-identification method based on shallow texture extraction and related equipment, and adds a re-identification model on the basis of a cycle generative adversarial network; the re-identification model extracts shallow feature maps of generated images and target domain images, uses texture loss for supervision, so that the images generated by the generator are more close to the target domain images in the shallow texture level; the re-identification model extracts deep feature encodings of the images on the source domain, the generated domain and the target domain on the metric branch, uses a metric learning loss function to retain the identity information of the generated images, and simultaneously lets the re-identification model distinguish the differences between the generated images and the target domain images; by the method, pictures closer to the target domain style and more suitable for the re-identification task can be generated; in the re-identification task, an adaptive local feature segmentation method and a local information dynamic matching method are used to improve a multi-granularity feature extraction model, so that the posture mismatch problem can be relieved to a certain extent.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a cross-domain re-identification method, system, terminal and computer-readable storage medium based on shallow texture extraction. Background Art

[0002] With the rapid development of the economy and technology, my country's urbanization process is steadily advancing. The number of surveillance cameras in various scenarios is constantly increasing, which is very conducive to the implementation of intelligent surveillance technology. Using auxiliary feature representation learning to add additional annotation information or training samples to enhance the features learned by neural network models has attracted attention.

[0003] In recent years, numerous methods have been proposed. For example, a deep attribute learning framework has been proposed that combines predicted semantic attribute information to enhance the generalizability and robustness of feature representations in semi-supervised learning. Viewpoint information has been used to enhance features. A multi-camera consistent matching constraint has been proposed, treating each camera as a distinct domain, to obtain globally optimal representations within a deep learning framework. Furthermore, attempts have been made to use GANs for re-identification, improving supervised feature representation learning based on generated pedestrian images. However, none of these methods consider texture information. Given that datasets acquired from different scenes have varying styles, increasing the number of samples and improving model generalization requires transferring the styles of different datasets to generate more training samples. Style is often correlated with the texture characteristics of the samples, and constraining texture features can improve style transfer. In re-identification tasks, however, if the camera viewpoint used to acquire the samples is suboptimal, the effective receptive area and position of the target in the image will vary, resulting in some detection results being overly dominated by background areas.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0005] The main purpose of the present invention is to provide a cross-domain re-identification method, system, terminal and computer-readable storage medium based on shallow texture extraction, aiming to solve the problem in the prior art that in re-identification tasks, if the camera perspective for acquiring samples is not ideal, it will lead to different effective receptive areas and positions of the target on the image, resulting in excessive proportion of background areas in some detection results.

[0006] To achieve the above object, the present invention provides a cross-domain re-identification method based on shallow texture extraction, the cross-domain re-identification method based on shallow texture extraction comprising the following steps:

[0007] Obtain a source domain dataset and a target domain dataset, and input the source domain dataset and the target domain dataset into a cyclic generative adversarial network;

[0008] A source domain image and a target domain image are sampled from the source domain dataset and the target domain dataset, respectively; the cyclic generative adversarial network converts the source domain image into a generated image with a style close to that of the target domain image according to the style of the target domain image; and the generated image, the source domain image, and the target domain image are input into a re-identification model;

[0009] Extracting shallow feature maps and deep feature maps obtained by inputting the generated image, the source domain image, and the target domain image into the re-identification model, respectively, and performing loss calculation;

[0010] Calculate the gradient based on the loss and update the parameters of the recurrent generative adversarial network and the re-identification model;

[0011] Obtaining an updated cyclic generative adversarial network, and using the updated cyclic generative adversarial network to convert the source domain dataset into the style of the target domain dataset;

[0012] The images generated by the updated recurrent generative adversarial network are fed into a multi-granularity feature extraction model for training to obtain a re-identification model with improved cross-domain recognition performance.

[0013] In addition, to achieve the above-mentioned object, the present invention further provides a cross-domain re-identification system based on shallow texture extraction, wherein the cross-domain re-identification system based on shallow texture extraction comprises:

[0014] A data acquisition module is used to acquire a source domain dataset and a target domain dataset, and input the source domain dataset and the target domain dataset into a recurrent generative adversarial network;

[0015] An image generation module is configured to sample a source domain image and a target domain image from the source domain dataset and the target domain dataset, respectively, convert the source domain image into a generated image with a style close to that of the target domain image according to the style of the target domain image, and input the generated image, the source domain image, and the target domain image into a re-identification model;

[0016] A feature extraction module is used to extract shallow feature maps and deep feature maps obtained after the generated image, the source domain image, and the target domain image are input into the re-identification model, and perform loss calculation;

[0017] A parameter updating module, configured to calculate gradients based on the loss and update parameters of the recurrent generative adversarial network and the re-identification model;

[0018] A data conversion module, configured to obtain an updated cyclic generative adversarial network and convert the source domain dataset into the style of the target domain dataset using the updated cyclic generative adversarial network;

[0019] The model training module is configured to input the updated picture generated by the cycle generative adversarial network into the multi-granularity feature extraction model for training, so as to obtain a re-identification model with improved cross-domain recognition performance.

[0020] In addition, to achieve the above object, the present application also provides a terminal, wherein the terminal comprises a memory, a processor, and a cross-domain re-identification method based on shallow texture extraction program stored in the memory and executable on the processor, and the cross-domain re-identification method based on shallow texture extraction program implements the steps of the cross-domain re-identification method based on shallow texture extraction when executed by the processor.

[0021] In addition, to achieve the above object, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a cross-domain re-identification method based on shallow texture extraction program, and the cross-domain re-identification method based on shallow texture extraction program implements the steps of the cross-domain re-identification method based on shallow texture extraction when executed by a processor.

[0022] The present application adds a re-identification model to the cycle generative adversarial network to constrain the shallow texture information of the generated picture and the target picture to be more similar, and this improvement makes the style of the generated picture closer to the style of the target picture than other methods, thereby improving the generalization ability of the model in the re-identification task, and a self-adaptive local feature segmentation method is proposed to alleviate the pose mismatch problem. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a flowchart of the preferred embodiment of the cross-domain re-identification method based on shallow texture extraction of the present application;

[0024] Figure 2 is a schematic diagram of the training framework of the re-identification model supervising the style transfer model in the preferred embodiment of the cross-domain re-identification method based on shallow texture extraction of the present application;

[0025] Figure 3 is a schematic diagram of the multi-granularity feature extraction network structure in the preferred embodiment of the cross-domain re-identification method based on shallow texture extraction of the present application;

[0026] Figure 4 is a schematic diagram of the self-adaptive local feature segmentation realized by using cumulative distribution in the preferred embodiment of the cross-domain re-identification method based on shallow texture extraction of the present application;

[0027] Figure 5 is a schematic diagram of the dynamic matching local information method in the preferred embodiment of the cross-domain re-identification method based on shallow texture extraction of the present application;

[0028] Figure 62 is a schematic diagram of dynamic alignment of local branch features of a multi-granularity feature extraction network in a preferred embodiment of the cross-domain re-identification method based on shallow texture extraction of the present invention;

[0029] Figure 7 2 is a schematic diagram of the principle of a preferred embodiment of the cross-domain re-identification system based on shallow texture extraction of the present invention;

[0030] Figure 8 Schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0032] The cross-domain re-identification method based on shallow texture extraction described in the preferred embodiment of the present invention aims to generate images that are closer to the target domain style and can retain the original identity information, while alleviating the pose mismatch problem in the re-identification task to a certain extent. Figure 1 As shown, the cross-domain re-identification method based on shallow texture extraction includes the following steps:

[0033] Step S10: Obtain a source domain dataset and a target domain dataset, and input the source domain dataset and the target domain dataset into a recurrent generative adversarial network.

[0034] Specifically, the source domain dataset and the target domain dataset are obtained, and k samples are randomly sampled from the source domain dataset and the target domain dataset.

[0035] like Figure 2 As shown in the style transfer model box, the source domain image A and the target domain image B are fed into the style generator, namely the Cycle Generative Adversarial Network (CycleGAN). That is, the k collected samples are input into the Cycle Generative Adversarial Network, and the generative adversarial losses are:

[0036]

[0037] Where G represents the target domain generator of the cyclic generative adversarial network; F represents the source domain generator; D T represents the target domain discriminator; D S represents the source domain discriminator; A represents the source domain image; B represents the target domain image; L adv (G, D T ) represents the generative adversarial loss of the target domain generator G of the recurrent generative adversarial network; Ladv (F, D S ) represents the generative adversarial loss of the source domain generator F of the recurrent generative adversarial network; Indicates that the target domain image B used for calculation in the formula comes from the target domain, that is, it obeys the data distribution of the target domain image B; D T (B) represents the target domain discriminator D T Determine whether the target domain image B is the target domain or not; Indicates that the source domain image A used for calculation in the formula comes from the source domain, that is, it obeys the data distribution of the source domain image A; G(A) represents the image generated after the source domain image A is fed into the target domain generator G of the cyclic generative adversarial network; D S (A) represents the source domain discriminator D S The result of judging whether the source domain image A is the source domain or not is represented by 1 or 0 respectively; D S (F(B)) represents the source domain discriminator D S Determine whether the target domain image B is the source domain or not after being fed into the source domain generator F of the recurrent generative adversarial network. The results are represented by 1 and 0 respectively.

[0038] The cycle consistency loss is expressed as:

[0039]

[0040] Here, F(G(A)) represents the image generated by feeding the source domain image A into the target domain generator G of the cyclic generative adversarial network; G(F(B)) represents the image generated by feeding the target domain image B into the source domain generator F of the cyclic generative adversarial network; F(B) represents the image generated by feeding the target domain image B into the target domain generator G of the cyclic generative adversarial network.

[0041] The intra-domain identity loss is expressed as:

[0042]

[0043] Here, F(A) represents the image generated by feeding the source domain image A into the source domain generator F of the cyclic generative adversarial network; F(B) represents the image generated by feeding the target domain image B into the source domain generator F of the cyclic generative adversarial network.

[0044] The overall loss function is expressed as:

[0045] L CycleGAN (G,F,D S ,D T )=L adv (G,D T )+L adv(F, D S )+λ1L rec (G,F)+λ2L idt (G,F);

[0046] Among them, λ1 and λ2 represent weights, which can be adjusted according to actual conditions and can generally be set to 1.

[0047] Step S20: Sample source domain images and target domain images from the source domain dataset and the target domain dataset, respectively. The cyclic generative adversarial network converts the source domain images into generated images with a style close to that of the target domain images according to the style of the target domain images, and inputs the generated images, the source domain images, and the target domain images into a re-identification model.

[0048] Specifically, if Figure 2 As shown in the re-identification model box, the source domain image A, the generated image A′ and the target domain image B are input into the re-identification model; the re-identification model uses Resnet50 as the feature extraction model backbone network M(·); the features extracted from the source domain image A, the generated image A′ and the target domain image B by the re-identification model are represented as M(A), M(A′) and M(B), respectively.

[0049] Step S30: extracting shallow feature maps and deep feature maps obtained after the generated image, the source domain image, and the target domain image are input into the re-identification model, and performing loss calculation.

[0050] Specifically, the re-identification model extracts and generates the deep feature encoding vectors of image A′, source domain image A, and target domain image B, which are M(A′), M(A), and M(B), respectively.

[0051] Using Triplet loss, the loss of constructing deep feature encoding is:

[0052] L triplet (A,B)=max{||M(G(A))-M(A)||2-||M(G(A))-M(B)||2+m,0};

[0053] Here, M(G(A)) represents the features extracted by the re-ID model from the image G(A) generated by feeding the source domain image A into the target domain generator G of the recurrent generative adversarial network. m represents the threshold, that is, the distance between the features M(G(A)) and M(A) should be at least m smaller than the distance between M(G(A)) and M(B). This value can be adjusted according to the actual situation.

[0054] The shallow texture features extracted by the re-identification model for the generated image A′ and the target domain image B are M j (A′) and M j (B), where Mj (·) represents the feature map of the j-th convolutional layer output extracted by the backbone network of the re-ID model.

[0055] The calculation of shallow style texture loss is expressed as:

[0056]

[0057] Among them, C j H j W j Representation feature M j (G(A)) and M j (B) The length obtained after expanding in width, height, and channel dimensions is similar to the total number of pixels in the image in width, height, and RGB channel dimensions; M j (G(A)) represents the features extracted by the j-th convolutional layer of the re-identification model after the source domain image A is fed into the target domain generator G of the recurrent generative adversarial network.

[0058] Step S40: Calculate the gradient based on the loss and update the parameters of the recurrent generative adversarial network and the re-identification model.

[0059] Specifically, the final loss is expressed as:

[0060]

[0061] Among them, λ3 and λ4 represent weights, which can be adjusted according to actual conditions and can generally be set to 1; L triplet (G,M) represents L triplet (A,B);L texture (G, M) means L texture (A, B).

[0062] According to the loss, the partial derivative of the feature is calculated and the model parameters are updated.

[0063] According to the partial derivatives, the negative direction of the gradient direction is iteratively learned by the gradient descent method, and the parameters of the cyclic generative adversarial network and the re-identification model are updated; it is determined whether the iteration is completed (the value of the loss function will be in a process of decreasing, slightly increasing but decreasing in overall magnitude during the training process. The iteration ends when the value of the loss function decreases and stabilizes, that is, the partial derivative obtained according to the loss function tends to 0). If the iteration is not completed, return to step S10.

[0064] Step S50: Obtain an updated cyclic generative adversarial network, and use the updated cyclic generative adversarial network to convert the source domain dataset into the style of the target domain dataset.

[0065] Specifically, the trained cyclic generative adversarial network is called, and the source domain dataset is input into the trained cyclic generative adversarial network; the generator of the cyclic generative adversarial network converts the source domain dataset into a generated dataset with the style of the target domain dataset.

[0066] Step S60: Send the updated image generated by the recurrent generative adversarial network to the multi-granularity feature extraction model for training to obtain a re-identification model with improved cross-domain recognition performance.

[0067] Specifically, the generated dataset is input into the multi-granularity feature extraction model, such as Figure 3 shown.

[0068] The multi-granularity feature extraction network also uses ResNet-50 as the backbone network. The part with the same structure as ResNet50 is regarded as the Backbone (the part shared by all branches, the main body of the model), and the remaining structure after the global average pooling layer is regarded as the Head (branch).

[0069] For the backbone, the original ResNet-50 network was decomposed into two groups based on the stage. The first group consists of stages 1, 2, and 3, with all branches sharing the convolution kernel parameters of that layer. The second group consists of 4 stages, so that the heads of different branches have the same structure but do not share parameters. For the head, the section after the original global average pooling layer was deleted. For the extracted features, a BNNeck structure consisting of a 1×1 convolutional layer and a BatchNorm layer was added to the head of each branch to reduce the dimensionality of the extracted features, reducing the number of feature channels in each branch from 2048 to 256. This reduces the complexity of calculating the distance matrix during training and inference retrieval.

[0070] The classification loss and metric loss are used to supervise the features extracted by the multi-granularity feature extraction model. For the classification loss, Softmax Loss is used to supervise the extracted local features and global features. For the extracted i-th local or global feature vector x i , the classification loss is expressed as:

[0071]

[0072] Among them, W k represents the weight vector of the kth category; represents the transpose of the weight vector of the kth category; represents y i The transpose of the weight vector of the category; N represents the number of training samples in a batch iteration; C represents the total number of categories in the training set;

[0073] For metric loss, the traditional Triplet Loss loss function L is used. triplet Optimize

[0074] The final loss function used to optimize the re-identification network is expressed as:

[0075] Loss = L classfication +L triplet ;

[0076] For the block division of local features in local branches, an adaptive local feature segmentation method is used. The schematic diagram of the method is shown in Figure 4 As shown, the size of the feature activation map x extracted by the feature extraction model is C×H×W, where C is the number of channels of the feature map, H is the height of the feature map, and W is the width of the feature map; the activation value of the feature map at the spatial coordinate (h, w) in the cth channel is recorded as x h，w，c , h and w represent height and width respectively.

[0077] Define an indicator function I x (h, w, c) indicates whether the output of the neuron at position (h, w) in the same channel plane c is the maximum activation value. The indicator function is defined as follows:

[0078]

[0079] In order to represent the distribution of the maximum activation value height on each channel, the indicator function is used to count the frequency of the maximum activation value height at each height. The distribution function D of the frequency of the maximum activation value height is x (h) is expressed as:

[0080]

[0081] Among them, the distribution function D x (h) is a function of height h. The output of the distribution function represents how many channels have feature maps with a maximum activation value of exactly h.

[0082] In order to make each local feature block occupy the same number of channels, first calculate the distribution function D x The cumulative distribution function H of (h) x (h):

[0083]

[0084] Among them, the cumulative distribution function H x The output of (h) indicates how many channels there are in the feature map, the height of the maximum activation value is less than or equal to the height h, and the range H x (h)∈[0, C]; the inverse function of the cumulative distribution function is defined as The domain of the definition is c∈[0, C], and the range is The sampling points are set on the inverse function of the cumulative distribution , and the segmentation height is obtained; for n s local patches, each segmentation height point h k satisfies the following relationship in the original cumulative distribution function H x (h):

[0085]

[0086] where H x (h k ) represents the number of channels with the maximum activation value between the kth height segmentation point h k and the k-1th height segmentation point h k-1 ; H x (h k +1) represents the number of channels with the maximum activation value between the kth height segmentation point h k and the k-1th height segmentation point h k-1 .

[0087] After adaptively segmenting the local features according to the channel information of the feature map, in order to further improve the robustness of the matching between the feature sequences, a dynamic matching local information method is introduced, and the effect of the dynamic matching local information algorithm is improved in combination with the obtained adaptively segmented local features as shown in Figure 5 .

[0088] For the pedestrian image A and the pedestrian image B in Figure 5 , the global feature activation maps x A and x B are adaptively pooled and segmented, and each image will obtain n s C-dimensional local feature sequences, and the local feature sequences of the two images are respectively denoted as and represent the n s th C-dimensional local feature sequence of the pedestrian image A, and the n s th C-dimensional local feature sequence of the pedestrian image B.

[0089] In order to ignore the deviation caused by the feature scale, it is necessary to calculate the distance matrix between the local feature sequences L A and L B by using a normalization method, and the values are normalized to the interval [0, 1), and the calculation formula is as follows:

[0090]

[0091] Among them, d i，j represents the normalized distance between the i-th local block of pedestrian image A and the j-th local block of pedestrian image B; Represents the j-th C-dimensional local feature sequence of pedestrian image A; represents the jth C-dimensional local feature sequence of the pedestrian image B; D represents the distance matrix, the distance between the local blocks of the two images and other local blocks of each other, the distance matrix formed is recorded as D, and the value of the distance matrix D at the (i, j) position is represented by d i，j In order to obtain the distance after the local feature sequence is aligned, the dynamic programming algorithm is used to search from (1, 1) to (n s , n s ) is the distance after alignment.

[0092] The shortest path search process is expressed as:

[0093]

[0094] Among them, S i,j represents the shortest path distance from (1, 1) to (i, j) in the distance matrix D; S i-1,j represents the shortest path distance from (1, 1) to (i-1, j) in the distance matrix D; S i，j-1 Represents the shortest path distance from (1, 1) to (i, j-1) in the distance matrix D.

[0095] Finally, after feature alignment correction, the distance d between the local feature sequences of the two images is l (A, B) is expressed as:

[0096]

[0097] For the processing of global features and local features, the network structure framework process of the local branch of the multi-granularity feature extraction structure is as follows Figure 6 As shown in the figure, on the global features obtained by the global branch, on the one hand, due to the faster calculation speed of global features, hard examples are first mined based on the global features, so that the local features only calculate the distance of alignment of local features of hard examples; on the other hand, the global features extracted from pedestrian image A and pedestrian image B are respectively denoted as f A and f B , then the distance between global features is expressed as:

[0098] d g (A,B)=||f A -f B ||2;

[0099] In the re-identification reasoning stage, the global feature distance and the local feature distance are weighted to represent each other, and the impact of the two distances on the sample is considered together. Therefore, the distance used in retrieval is:

[0100] d(A,B)=d g (A,B)+λd l (A,B);

[0101] Among them, λ is a hyperparameter used to balance the two distances, which can usually be set to λ = 1; d l (A, B) represents the distance between local features.

[0102] Finally, partial derivatives of the loss with respect to the features are calculated to update the model parameters. Gradient descent is then iteratively learned in the negative direction of the gradient using the partial derivatives. The parameters of the multi-granular feature extraction network are updated. The iteration is then determined to be complete. If not, the generated dataset is fed into the multi-granular feature extraction model.

[0103] In order to verify the proposed learning framework of supervised style transfer using the re-identification model, cross-domain re-identification experiments were conducted from two aspects: deep feature constraints and shallow feature constraints.

[0104] First, Table 1 shows the impact of deep re-ID feature constraints on recurrent generative adversarial network style transfer. By introducing identity supervision from deep re-ID features, the recurrent generative adversarial network-based style transfer framework improves cross-domain performance by 2.5% mAP and 3.0% Rank-1 accuracy in the cross-domain re-ID task using Market1501 as the source data domain. In experiments using DukeMTMC-reid as the source data domain, cross-domain performance improves by 4.8% mAP and 2.6% Rank-1 accuracy. This demonstrates the effectiveness of introducing deep re-ID features to supervise style transfer models.

[0105] method Cross-domain direction Rank-1 Rank-5 Rank-10 Rank-20 mAP Direct cross-domain 32.5 45.8 52.7 58.9 18.3 CycleGAN Market→Duke 40.2 56.1 61.4 67.5 22.2 Deep feature constraints 43.2 58.7 65.5 71.4 24.7 Direct cross-domain 46.8 64.1 70.5 79.5 19.9 CycleGAN Duke→Market 52.0 69.1 75.3 81.2 22.3 Deep feature constraints 54.6 71.4 79.7 84.2 25.3

[0106] Table 1: Experimental results of CycleGAN style transfer using deep feature constraints of the re-identification model

[0107] Furthermore, to explore the impact of style loss at shallow feature layers of varying depths in the re-ID backbone network on the cyclic generative adversarial network style transfer model, shallow feature constraints were added to different stages of the re-ID backbone network to constrain the style between data domains. The results are shown in Table 2.

[0108]

[0109] Table 2: Impact of shallow feature constraints of different stages of the re-ID model on CycleGAN style transfer

[0110] In order to verify the effectiveness of the proposed adaptive local feature segmentation method in improving the local information dynamic matching method, the relevant experimental results are shown in Table 3.

[0111]

[0112]

[0113] Table 3: Experimental results of the adaptive local feature segmentation method to improve the feature extraction model

[0114] Based on the recurrent generative adversarial network, this paper adds a re-identification model to constrain the generated image to be closer to the shallow texture information of the target image. This improvement makes the style of the generated image closer to the style of the target image than other methods, thereby improving the generalization ability of the model in the re-identification task. At the same time, a method of adaptively segmenting local features is proposed to alleviate the pose mismatch problem.

[0115] Furthermore, if Figure 7 As shown, based on the above cross-domain re-identification method based on shallow texture extraction, the present invention also provides a cross-domain re-identification system based on shallow texture extraction, wherein the cross-domain re-identification system based on shallow texture extraction includes:

[0116] A data acquisition module 51 is configured to acquire a source domain dataset and a target domain dataset, and input the source domain dataset and the target domain dataset into a recurrent generative adversarial network;

[0117] An image generation module 52 is configured to sample a source domain image and a target domain image from the source domain dataset and the target domain dataset, respectively, convert the source domain image into a generated image with a style close to that of the target domain image according to the style of the target domain image, and input the generated image, the source domain image, and the target domain image into a re-identification model;

[0118] A feature extraction module 53 is configured to extract shallow feature maps and deep feature maps obtained by inputting the generated image, the source domain image, and the target domain image into the re-identification model, and perform loss calculation;

[0119] A parameter updating module 54 is configured to calculate gradients based on the loss and update parameters of the recurrent generative adversarial network and the re-identification model;

[0120] a data conversion module 55, configured to obtain an updated recurrent generative adversarial network, and convert the source domain dataset into the style of the target domain dataset using the updated recurrent generative adversarial network;

[0121] The model training module 56 is used to send the updated images generated by the recurrent generative adversarial network into the multi-granularity feature extraction model for training to obtain a re-identification model with improved cross-domain recognition performance.

[0122] Furthermore, if Figure 8 As shown, based on the above-mentioned cross-domain re-identification method and system based on shallow texture extraction, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 8 Only some of the components of the terminal are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0123] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Furthermore, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a cross-domain re-identification program 40 based on shallow texture extraction is stored on the memory 20, and the cross-domain re-identification program 40 based on shallow texture extraction can be executed by the processor 10, thereby realizing the cross-domain re-identification method based on shallow texture extraction in the present application.

[0124] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, configured to execute program code or process data stored in the memory 20, such as executing the cross-domain re-identification method based on shallow texture extraction.

[0125] The display 30 can be, in some embodiments, an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. The display 30 is used to display information at the terminal and to display a visualized user interface. The components 10-30 of the terminal communicate with each other through a system bus.

[0126] In an embodiment, the steps of the shallow texture extraction based cross-domain re-identification method are implemented when the processor 10 executes the shallow texture extraction based cross-domain re-identification program 40 in the memory 20.

[0127] The present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a shallow texture extraction based cross-domain re-identification program, and the shallow texture extraction based cross-domain re-identification program, when executed by a processor, implements the steps of the shallow texture extraction based cross-domain re-identification method as described above.

[0128] In summary, the present application provides a shallow texture extraction based cross-domain re-identification method and related equipment, the method comprising: sampling source domain pictures and target domain pictures from the source domain dataset and the target domain dataset respectively, the cycle generative adversarial network converting the source domain pictures into generated pictures with a style close to that of the target domain pictures according to the style of the target domain pictures, and inputting the generated pictures, the source domain pictures and the target domain pictures into a re-identification model; extracting shallow feature maps and deep feature maps of the generated pictures, the source domain pictures and the target domain pictures input into the re-identification model respectively, and performing loss calculation; updating parameters of the cycle generative adversarial network and the re-identification model according to the loss calculation gradient; obtaining an updated cycle generative adversarial network, converting the source domain dataset into the style of the target domain dataset using the updated cycle generative adversarial network; and inputting the pictures generated by the updated cycle generative adversarial network into a multi-granularity feature extraction model for training to obtain a re-identification model with improved cross-domain recognition performance. The present application adds a re-identification model to the cycle generative adversarial network to constrain the shallow texture information of the generated pictures and the target pictures to be closer, which improves the style of the generated pictures to be closer to that of the target pictures compared to other methods, thereby improving the generalization ability of the model in the re-identification task, and proposes a self-adaptive segmentation local feature method to alleviate the pose mismatch problem.

[0129] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal comprising the element.

[0130] Of course, those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium that can be read by a computer. When the program is executed, it can include the processes in the above-described method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0131] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A cross-domain re-identification method based on shallow texture extraction, characterized in that: The cross-domain re-identification method based on shallow texture extraction includes: Obtain a source domain dataset and a target domain dataset, and input the source domain dataset and the target domain dataset into a cyclic generative adversarial network; A source domain image and a target domain image are sampled from the source domain dataset and the target domain dataset, respectively; the cyclic generative adversarial network converts the source domain image into a generated image with a style close to that of the target domain image according to the style of the target domain image; and the generated image, the source domain image, and the target domain image are input into a re-identification model; Extracting shallow feature maps and deep feature maps obtained by inputting the generated image, the source domain image, and the target domain image into the re-identification model, respectively, and performing loss calculation; Calculate the gradient based on the loss and update the parameters of the recurrent generative adversarial network and the re-identification model; Obtaining an updated cyclic generative adversarial network, and using the updated cyclic generative adversarial network to convert the source domain dataset into the style of the target domain dataset; The images generated by the updated recurrent generative adversarial network are fed into a multi-granularity feature extraction model for training to obtain a re-identification model with improved cross-domain recognition performance.

2. The cross-domain re-identification method based on shallow texture extraction according to claim 1 is characterized in that The acquiring of a source domain dataset and a target domain dataset, and inputting the source domain dataset and the target domain dataset into a cyclic generative adversarial network specifically includes: Acquire the source domain dataset and the target domain dataset, and randomly sample k samples from the source domain dataset and the target domain dataset; The collected k samples are input into the cyclic generative adversarial network, and the generative adversarial losses are: Where G represents the target domain generator of the cyclic generative adversarial network; F represents the source domain generator; D T represents the target domain discriminator; D S represents the source domain discriminator; A represents the source domain image; B represents the target domain image; L adv (G, D T ) represents the generative adversarial loss of the target domain generator G of the recurrent generative adversarial network; L adv (F,D S ) represents the generative adversarial loss of the source domain generator F of the recurrent generative adversarial network; Denotes that it obeys the data distribution of the target domain image B; D T (B) represents the target domain discriminator D T Determine whether the target domain image B is the target domain or not; Denotes the data distribution of the source domain image A; G(A) represents the image generated by feeding the source domain image A into the target domain generator G of the cyclic generative adversarial network; D S (A) represents the source domain discriminator D S The result of judging whether the source domain image A is the source domain or not; D S (F(B)) represents the source domain discriminator d S Determine whether the target domain image B is the source domain or not after being fed into the source domain generator F of the recurrent generative adversarial network. The cycle consistency loss is expressed as: Here, F(G(A)) represents the image generated by feeding the source domain image A into the target domain generator G of the cyclic generative adversarial network and the image generated by feeding G(A) into the source domain generator F of the cyclic generative adversarial network; G(F(B)) represents the image generated by feeding the target domain image B into the source domain generator F of the cyclic generative adversarial network and the image generated by feeding F(B) into the target domain generator G of the cyclic generative adversarial network; The intra-domain identity loss is expressed as: Here, F(A) represents the image generated by feeding the source domain image A into the source domain generator F of the recurrent generative adversarial network; F(B) represents the image generated by feeding the target domain image B into the source domain generator F of the recurrent generative adversarial network; The overall loss function is expressed as: L CycleGAN (G,F,D S ,D T )=L adv (G,D T )+L adv (F,D S )+λ1L rec (G,F)+λ2L idt (G,F); Among them, λ1 and λ2 represent weights.

3. The cross-domain re-identification method based on shallow texture extraction according to claim 2 is characterized in that Inputting the generated image, the source domain image, and the target domain image into a re-identification model specifically includes: Input the source domain image A, generated image A′ and target domain image B into the re-identification model; The features extracted by the re-identification model from the source domain image A, generated image A′, and target domain image B are denoted as M(A), M(A′), and M(B), respectively.

4. The cross-domain re-identification method based on shallow texture extraction according to claim 3 is characterized in that The extracting of shallow feature maps and deep feature maps obtained by inputting the generated image, the source domain image, and the target domain image into the re-identification model, and performing loss calculation specifically includes: The deep feature encoding vectors of the generated image A′, source domain image A, and target domain image B extracted by the re-identification model are M(A′), M(A), and M(B), respectively. Using Triplet loss, the loss of constructing deep feature encoding is: L triplet (A,B)=max{||M(G(A))-M(A)||2-||M(G(A))-M(B)||2+m,0}; Where M(G(A)) represents the features extracted by the re-identification model after the source domain image A is fed into the target domain generator G of the recurrent generative adversarial network; m represents the threshold; The shallow texture features extracted by the re-identification model for the generated image A′ and the target domain image B are M j (A′) and M j (B), where M j (·) represents the feature map of the j-th convolutional layer output extracted by the backbone network of the re-ID model; The calculation of shallow style texture loss is expressed as: Among them, C j H j W j Representation feature M j (G(A)) and M j (B) The length obtained after expansion in width, height, and channel dimensions; M j (G(A)) represents the features extracted by the j-th convolutional layer of the re-identification model after the source domain image A is fed into the target domain generator G of the recurrent generative adversarial network.

5. The cross-domain re-identification method based on shallow texture extraction according to claim 4 is characterized in that Calculating the gradient based on the loss and updating the parameters of the cyclic generative adversarial network and the re-identification model specifically include: The final loss is expressed as: L Proposed (G,F,D S ,D T ,M)=L adv (G,D T )+L adv (F,D S )+λ1L rec (G,F)+λ2L idt (G,F)+λ3L triplet (G,M)+λ4L texture (G,M); Among them, λ3 and λ4 represent weights; L triplet (G, M) represents the loss of deep feature encoding; L texture (G, M) represents shallow style texture loss; Take partial derivatives of the loss with respect to the features; According to the partial derivatives, the negative direction of the gradient is iteratively learned by gradient descent method to update the parameters of the cyclic generative adversarial network and the re-identification model.

6. The cross-domain re-identification method based on shallow texture extraction according to claim 5 is characterized in that Obtaining an updated cyclic generative adversarial network and converting the source domain dataset into the style of the target domain dataset using the updated cyclic generative adversarial network specifically includes: Call the trained cyclic generative adversarial network and input the source domain dataset into the trained cyclic generative adversarial network; The generator of the cyclic generative adversarial network transforms the source domain dataset into a generated dataset with the style of the target domain dataset.

7. The cross-domain re-identification method based on shallow texture extraction according to claim 6 is characterized in that The updated recurrent generative adversarial network generates images that are fed into a multi-granularity feature extraction model for training to obtain a re-identification model with improved cross-domain recognition performance, specifically including: Input the generated dataset into the multi-granularity feature extraction model; The features extracted by the multi-granularity feature extraction model are supervised by using classification loss and metric loss. For the extracted i-th local or global feature vector x i , the classification loss is expressed as: Among them, W k represents the weight vector of the kth category; represents the transpose of the weight vector of the kth category; represents y i The transpose of the weight vector of the category; N represents the number of training samples in a batch iteration; C represents the total number of categories in the training set; For metric loss, the traditional Triplet Loss loss function L is used. triplet Optimize; The final loss function used to optimize the re-identification network is expressed as: Loss=L classfication +L triplet ; For the block of local features in the local branch, an adaptive local feature segmentation method is used, and the feature extraction model is used to extract the feature activation map x with a size of C×H×W, where C is the number of channels of the feature map, H is the height of the feature map, and W is the width of the feature map; the activation value of the feature map at the spatial coordinate (h, w) in the cth channel is recorded as x h，w，c ; Define an indicator function I x (h, w, c) indicates whether the output of the neuron at position (h, w) in the same channel plane c is the maximum activation value. The indicator function is defined as follows: The indicator function is used to count the frequency of the maximum activation value of each height, and the distribution function D of the frequency of the maximum activation value height is used. x (h) is expressed as: Among them, the distribution function D x (h) is a function of height h. The output of the distribution function represents how many channels have feature maps with a maximum activation value of exactly h. Make each local feature block occupy the same number of channels, first calculate the distribution function D x The cumulative distribution function H of (h) x (h): Among them, the cumulative distribution function H x The output of (h) indicates how many channels there are in the feature map, the height of the maximum activation value is less than or equal to the height h, and the range H x (h)∈[0, C]; the inverse function of the cumulative distribution function is defined as The domain of is c∈[0, C], and the range is By inverse function of cumulative distribution On the top, set the sampling points at equal intervals to obtain the segmentation height; for n s For a local block, each segmentation height point h k In the original cumulative distribution function H x (h) satisfies the following relationship: Among them, H x (h k ) represents the kth height segmentation point h k and the k-1th height segmentation point h k-1 The number of channels with maximum activation values ​​between them; H x (h k +1) represents the kth height segmentation point h k and the k-1th height segmentation point h k-1 The number of channels with maximum activation values ​​between For pedestrian image A and pedestrian image B, the global feature activation map x is extracted A with x B Perform adaptive pooling segmentation respectively, and each image will get n s C-dimensional local feature sequences, the local feature sequences of the two images are recorded as and represents the nth s C-dimensional local feature sequences, represents the nth s C-dimensional local feature sequences; Ignore the deviation caused by the feature scale and use the normalization method to calculate the local feature sequence L A With L B The distance matrix between them is normalized to the interval [0, 1). The calculation formula is as follows: Among them, d i，j represents the normalized distance between the i-th local block of pedestrian image A and the j-th local block of pedestrian image B; Represents the i-th C-dimensional local feature sequence of pedestrian image A; represents the jth C-dimensional local feature sequence of pedestrian image B; D represents the distance matrix, and the value of distance matrix D at position (i, j) is represented by d i，j Composition; In the distance matrix D, use the dynamic programming algorithm to search from (1, 1) to (n s , n s ) is the distance after alignment; The shortest path search process is expressed as: Among them, S i，j represents the shortest path distance from (1, 1) to (i, j) in the distance matrix D; S i-1，j represents the shortest path distance from (1, 1) to (i-1, j) in the distance matrix D; S i，j-1 represents the shortest path distance from (1, 1) to (i, j-1) in the distance matrix D; After feature alignment correction, the distance d between the local feature sequences of the two images l (A, B) is expressed as: On the basis of the global features obtained by the global branch, hard examples are mined so that the local features only calculate the distance of alignment of the local features of the hard examples. The global features extracted from the pedestrian image A and the pedestrian image B are respectively denoted as f A and f B , then the distance between global features is expressed as: d g (A,B)=||f A -f B ||2; In the re-identification reasoning phase, the global feature distance and the local feature distance are weighted to represent each other, and the impact of the two distances on the sample is considered together. The distance used during retrieval is: d(A,B)=d g (A,B)+λd l (A,B); Among them, λ is a hyperparameter used to balance the two distances; d l (A, B) represents the distance between local features.

8. A cross-domain re-identification system based on shallow texture extraction, characterized in that: The cross-domain re-identification system based on shallow texture extraction includes: A data acquisition module is used to acquire a source domain dataset and a target domain dataset, and input the source domain dataset and the target domain dataset into a recurrent generative adversarial network; An image generation module is configured to sample a source domain image and a target domain image from the source domain dataset and the target domain dataset, respectively, convert the source domain image into a generated image with a style close to that of the target domain image according to the style of the target domain image, and input the generated image, the source domain image, and the target domain image into a re-identification model; A feature extraction module is used to extract shallow feature maps and deep feature maps obtained after the generated image, the source domain image, and the target domain image are input into the re-identification model, and perform loss calculation; A parameter updating module, configured to calculate gradients based on the loss and update parameters of the recurrent generative adversarial network and the re-identification model; A data conversion module, configured to obtain an updated cyclic generative adversarial network and convert the source domain dataset into the style of the target domain dataset using the updated cyclic generative adversarial network; The model training module is used to feed the images generated by the updated recurrent generative adversarial network into the multi-granularity feature extraction model for training to obtain a re-identification model with improved cross-domain recognition performance.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a cross-domain re-identification program based on shallow texture extraction stored in the memory and runnable on the processor. When the cross-domain re-identification program based on shallow texture extraction is executed by the processor, the steps of the cross-domain re-identification method based on shallow texture extraction are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a cross-domain re-identification program based on shallow texture extraction. When the cross-domain re-identification program based on shallow texture extraction is executed by a processor, the steps of the cross-domain re-identification method based on shallow texture extraction according to any one of claims 1 to 7 are implemented.

Images