A cross-domain pedestrian re-identification method based on accurate feature distribution matching and multi-domain information fusion

Abstract

The application relates to a cross-domain pedestrian re-identification method based on accurate feature distribution matching and multi-domain information fusion. The method is used for solving the feature distribution difference between a source domain and a target domain and the domain difference problem existing in multi-source dataset training. A to-be-trained model is created by using a feature extractor, a generalized mean pooling module GeM, a multi-domain information fusion module, a target domain-specific normalization layer DSBN_T and a target hybrid memory HM_T, and the model is trained in combination with a source domain sample set and a target domain sample set to obtain a target person identification model. Then, the model is used for realizing the identification of each target person in an image. Experiments show that, compared with the prior art, the application has superior actual implementation and application performance and can effectively improve the accuracy and efficiency of person identification in an image.

Description

Technical Field

[0001] This invention relates to a cross-domain pedestrian re-identification method based on accurate feature distribution matching and multi-domain information fusion, belonging to the field of image pedestrian recognition technology. Background Technology

[0002] With the emergence of large-scale datasets and improvements in feature extraction and metric learning methods, person re-identification has received considerable attention. Currently, most person re-identification models are trained using limited data from a single source domain, which is unreasonable in practical applications, preventing the full utilization of other labeled datasets. For example, the advantage of auto-step contrastive learning based on clustering lies in its use of hybrid memory to continuously fine-tune pseudo-labels online and its training of the network using contrastive learning strategies, achieving good recognition results in domain adaptation. However, this network is trained using only a single dataset and has not yet considered differences between different domains. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a cross-domain person re-identification method based on accurate feature distribution matching and multi-domain information fusion. In view of the differences in feature distribution between the source domain and the target domain, as well as the domain differences that exist when training multi-source datasets, a novel logical recognition scheme is designed, which can effectively improve the accuracy of person re-identification.

[0004] To solve the aforementioned technical problems, this invention adopts the following technical solution: This invention designs a cross-domain person re-identification method based on accurate feature distribution matching and multi-domain information fusion, based on the source domain sample set S of each sample image containing each known identity label target person corresponding to each shooting angle. i Combining the target domain sample set T of sample images containing each target person but with unknown corresponding identity labels, steps A to B are performed to obtain a target person recognition model for each target person; then, applying the target person recognition model, step C is performed to identify the target person for the image to be analyzed; 1≤i≤I, where I represents the number of shooting angles, S i This represents the source domain sample set corresponding to the i-th shooting angle;

[0005] Step A. Construct a training model that takes the image of the person as input and the identification result of the target person's identity tag as output, and then proceed to Step B;

[0006] Step B. Based on the multi-source domain contrast loss module L HM and sample sets S from each source domain i The corresponding source domain-specific normalization layer DSBN_Si and source hybrid memory HM_Si are based on the sample sets S of each source domain. iAnd the target domain sample set T, to train the model to be trained, and obtain the target person recognition model for each target person;

[0007] Step C. The target person recognition model receives the image of the person to be analyzed, processes it, and obtains the identification result of the target person's identity label in the image of the person to be analyzed.

[0008] As a preferred technical solution of the present invention: in step A, the model to be trained includes a feature extractor, a generalized mean pooling module GeM, a multi-domain information fusion module, a target domain specific normalization layer DSBN_T, and a target hybrid memory HM_T;

[0009] The feature extractor comprises, from its input to its output, a first residual network L0, a second residual network L1, a first precise feature distribution matching module EFDM1, a third residual network L2, a second precise feature distribution matching module EFDM2, a fourth residual network L3, and a fifth residual network L4, all connected in series. The input of the first residual network L0 constitutes the input of the feature extractor, and the output of the fifth residual network L4 constitutes the output of the feature extractor.

[0010] The multi-domain information fusion module, from its input end to its output end, includes a first graph convolutional neural network GCN1, a leaky modified linear unit ReLU, a second graph convolutional neural network GCN2, and a fusion module connected in series. The input end of the first graph convolutional neural network GCN1 constitutes the input end of the multi-domain information fusion module, and the input end of the fusion module is simultaneously connected to the input end of the multi-domain information fusion module. The output end of the fusion module constitutes the output end of the multi-domain information fusion module.

[0011] The input of the feature extractor constitutes the input of the model to be trained. The output of the feature extractor is connected to the input of the generalized mean pooling module GeM. The output of the generalized mean pooling module GeM is connected to the input of the multi-domain information fusion module. The output of the multi-domain information fusion module is connected to the input of the target domain specific normalization layer DSBN_T. The output of the target domain specific normalization layer DSBN_T is connected to the input of the target hybrid memory HM_T. The output of the target hybrid memory HM_T constitutes the output of the model to be trained.

[0012] As a preferred technical solution of the present invention: In step B, the model to be trained and the multi-source domain contrast loss module L are used. HM and sample sets S from each source domain i The combined structures of the source domain-specific normalization layer DSBN_Si and the source hybrid memory HM_Si are as follows, based on the sample sets S of each source domain. iAnd the target domain sample set T, to train the model to be trained, and obtain the target person recognition model for each target person;

[0013] Combined Structure: The output of the multi-domain information fusion module is connected to the input of the target domain-specific normalization layer DSBN_T and the input of each source domain-specific normalization layer DSBN_Si, respectively. The output of the target domain-specific normalization layer DSBN_T is connected to the input of the target hybrid memory HM_T, and the output of each source domain-specific normalization layer DSBN_Si is connected to the input of the corresponding source hybrid memory HM_Si. Simultaneously, the outputs of the target domain-specific normalization layer DSBN_T, the target hybrid memory HM_T, each source domain-specific normalization layer DSBN_Si, and each source hybrid memory HM_Si are all connected to the multi-source domain contrast loss module L. HM The input terminal.

[0014] As a preferred technical solution of the present invention: In step B, according to the sample sets S of each source domain... i The target domain sample set T is used to train the model to be trained in the following data processing manner to obtain the target person recognition model for each target person.

[0015] Step i. Apply feature extractors to each source domain sample set S. i Feature extraction is performed on the target domain sample set T to obtain the sample sets S of each source domain. i The corresponding feature matrices and the feature matrix corresponding to the target domain sample set T Then proceed to step ii;

[0016] Step ii. For each source domain sample set S i Corresponding feature matrix and the feature matrix corresponding to the target domain sample set T The generalized mean pooling module GeM processes and updates the sample sets S of each source domain. i Corresponding feature matrix and the feature matrix corresponding to the target domain sample set T Then proceed to step iii;

[0017] Step iii. For each source domain sample set S i Corresponding feature matrix and the feature matrix corresponding to the target domain sample set T The multi-domain information fusion module processes the data to obtain sample sets S from each source domain. i The corresponding feature matrices and the feature matrix corresponding to the target domain sample set T Then proceed to step iv;

[0018] Step iv. Feature matrices and feature matrix The corresponding feature matrices are obtained by processing the data through the source domain-specific normalization layer DSBN_Si and the target domain-specific normalization layer DSBN_T, respectively. and feature matrix

[0019] Step v. Feature matrices and feature matrix The samples are processed by the corresponding source hybrid memory HM_Si and the target hybrid memory HM_T respectively to obtain the source domain sample sets S output by each source hybrid memory HM_Si. i Source domain identity tags and the feature matrix of all unclustered target domain instances output by the target hybrid memory HM_T from the target domain sample set T. Pseudo-labels for target domain clustering

[0020] Step vi. For the feature matrix Each feature matrix Identity tags of each source domain and features of all unclustered target domain instances in the target domain sample set T Pseudo-labels for target domain clustering Application of multi-source domain contrast loss module L HM Perform loss calculation.

[0021] As a preferred technical solution of the present invention: In step i, for each source domain sample set S respectively i The source domain sample sets S and the target domain sample set T are first processed through a first-layer residual network L0 and a second-layer residual network L1 for feature extraction to obtain each source domain sample set S. i The corresponding characteristic matrices S i The feature matrix T′ corresponding to the target domain sample set T is sent to the first precise feature distribution matching module EFDM1 for processing.

[0022] Next, in the processing of the first precise feature distribution matching module EFDM1, each feature matrix S is processed separately. i And the characteristic matrix T', first sort the rows of the characteristic matrix in ascending order of value to obtain the characteristic matrix S. i " and feature matrix T", and then for each feature matrix S i "The corresponding feature matrices are obtained according to the following formula (1)."

[0023]

[0024] The first precise feature distribution matching module EFDM1 outputs each feature matrix. The feature matrix T″ is processed by the third residual network L2, where <·> indicates stopping the gradient operation, λ~Beta(α,a), α∈(0,∞) are adjustable hyperparameters, and Beta(α,a) represents a beta distribution with shape parameter α.

[0025] Then, in the third residual network L2 processing, each feature matrix is ​​processed separately. Feature extraction is performed to obtain the corresponding feature matrices. The feature matrix T″ is then fed into the second precise feature distribution matching module EFDM2 for processing. In the processing of the second precise feature distribution matching module EFDM2, each feature matrix is ​​processed separately. First, sort the rows of the feature matrix in ascending order of values ​​to obtain each feature matrix. Then, for each feature matrix... The corresponding feature matrices are obtained according to the following formula (2).

[0026]

[0027] The second precise feature distribution matching module EFDM2 outputs each feature matrix. And the characteristic matrix T″;

[0028] Finally, each feature matrix is ​​analyzed separately. The feature matrix T″ is then processed sequentially through the fourth residual network L3 and the fifth residual network L4 for feature extraction, thus updating and obtaining the sample sets S of each source domain. i The corresponding feature matrices and the feature matrix corresponding to the target domain sample set T

[0029] As a preferred embodiment of the present invention: step iii is performed by the multi-domain information fusion module as follows:

[0030] Step iii-1. For each source domain sample set S... i Corresponding feature matrix and the feature matrix corresponding to the target domain sample set T According to the following formula (3):

[0031]

[0032] Obtain each feature matrix The corresponding domain proxy node vectors and feature matrix Corresponding domain proxy node vector N represents the number of instances in the feature matrix. Representation of the characteristic matrix The vector corresponding to the nth instance, w in Representation of the characteristic matrix The preset learnable weighted vector corresponding to the nth instance. Representation of the characteristic matrix The vector corresponding to the nth instance, w n Representation of the characteristic matrix The preset learnable weighted vector corresponding to the nth instance;

[0033] Step iii-2. According to the following formula (4):

[0034]

[0035] Obtain the multi-domain information fusion matrix F (0) , where ∪ represents stacking by columns, and C represents the length of each row of the feature matrix;

[0036] Step iii-3. Construct the adjacency matrix of the first graph convolutional neural network GCN1 and the second graph convolutional neural network GCN2 according to the following formula (5). l = 1, 2, A (1) Let A represent the adjacency matrix of the first graph convolutional neural network GCN1. (2) This represents the adjacency matrix of the second graph's convolutional neural network, GCN2.

[0037]

[0038] Where r and c represent the adjacency matrix A, respectively. (l) Row index and column index, Represents the adjacency matrix A (l) The value in the r-th row and c-th column;

[0039] Step iii-4. Based on the adjacency matrix A of the convolutional neural network GCN1 in the first graph... (1) For the multi-domain information fusion matrix F (0) According to the following formula (6):

[0040]

[0041] Obtain the multi-domain information fusion matrix F (0) The multi-domain information fusion matrix F after processing by the first convolutional neural network GCN1 is (1 ),W 1 This represents the preset learnable weighting matrix corresponding to the first convolutional neural network GCN1;

[0042] Step iii-5. Based on the adjacency matrix A of the convolutional neural network GCN2 in the second figure... (2) For the multi-domain information fusion matrix F( 1 According to the following formula (7):

[0043]

[0044] Obtain the multi-domain information fusion matrix F( 1 The multi-domain information fusion matrix F after processing by the second graph convolutional neural network GCN2 (2) W 2 This represents the preset learnable weighting matrix corresponding to the GCN2 convolutional neural network in the second graph;

[0045] Step iii-6. For the multi-domain information fusion matrix F (0) With multi-domain information fusion matrix F (2) The multi-domain information fusion feature matrix is ​​obtained by the fusion module. And targeting the feature matrix of multi-domain information fusion According to the corresponding source domain sample sets S i The target domain sample set T is decomposed to obtain the sample sets S of each source domain. i The corresponding feature matrices and the feature matrix corresponding to the target domain sample set T

[0046] As a preferred technical solution of the present invention: in step vi, for the feature matrix Each feature matrix Identity tags of each source domain and the feature matrix of all unclustered target domain instances in the target domain sample set T Pseudo-labels for target domain clustering Application of multi-source domain contrast loss module L HM The loss is calculated according to the following formula (9);

[0047]

[0048] Obtain the loss calculation result L HM ,in, f j Let f be the j-th vector. yes The k-th eigenvector in express The m-th eigenvector in the middle, yes The 0th eigenvector in the vector, where <·, ·> represent the inner product between two eigenvectors, z + Indicates with f j The corresponding positive class prototype, σ represents the temperature parameter; n s express The number of eigenvectors in n v express The number of eigenvectors in n c express The number of eigenvectors in the model.

[0049] As a preferred technical solution of the present invention: In step C, the target person recognition model receives the image of the person to be analyzed, and processes it sequentially through a feature extractor, a generalized mean pooling module GeM, a multi-domain information fusion module, a target domain specific normalization layer DSBN_T, and a target hybrid memory HM_T to obtain the target domain clustering pseudo-label corresponding to the image of the person to be analyzed, that is, to obtain the identity label recognition result of the image of the person to be analyzed regarding the target person.

[0050] The cross-domain person re-identification method based on accurate feature distribution matching and multi-domain information fusion described in this invention has the following technical advantages compared with existing technologies:

[0051] This invention presents a cross-domain person re-identification method based on accurate feature distribution matching and multi-domain information fusion. Addressing the differences in feature distribution between the source and target domains, as well as the domain differences encountered during training with multi-source datasets, the method utilizes a feature extractor, a generalized mean pooling module (GeM), a multi-domain information fusion module, a target domain-specific normalization layer (DSBN_T), and a target hybrid memory (HM_T) to create a training model. This model is then trained using both source and target domain sample sets to obtain a target person recognition model. This model is then used to identify individuals in images. Experiments demonstrate that this invention offers superior performance in practical applications compared to existing technologies, effectively improving the accuracy and efficiency of person recognition in images. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the training architecture of the target person recognition model in this invention. Detailed Implementation

[0053] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0054] This invention presents a cross-domain person re-identification method based on precise feature distribution matching and multi-domain information fusion. This method utilizes a source domain sample set S, which contains sample images of target individuals with known identity tags from each shooting angle.i Combining the target domain sample set T of sample images containing each target person but with unknown corresponding identity labels, perform steps A to B as follows to obtain a target person recognition model for each target person, where 1 ≤ i ≤ I, I represents the number of shooting angles, and S i Let represent the source domain sample set corresponding to the i-th shooting angle.

[0055] Step A. Construct a training model that takes the image of the person as input and the identification result of the target person's identity label as output. The specific design is as follows, and then proceed to Step B.

[0056] like Figure 1 As shown, the model to be trained here includes a feature extractor, a generalized mean pooling module GeM, a multi-domain information fusion module, a target domain-specific normalization layer DSBN_T, and a target hybrid memory HM_T.

[0057] The feature extractor comprises, from its input to its output, a first residual network L0, a second residual network L1, a first precise feature distribution matching module EFDM1, a third residual network L2, a second precise feature distribution matching module EFDM2, a fourth residual network L3, and a fifth residual network L4, all connected in series. The input of the first residual network L0 constitutes the input of the feature extractor, and the output of the fifth residual network L4 constitutes the output of the feature extractor.

[0058] The multi-domain information fusion module, from its input to its output, includes, in sequence, a first graph convolutional neural network GCN1, a leaky modified linear unit ReLU, a second graph convolutional neural network GCN2, and a fusion module. The input of the first graph convolutional neural network GCN1 constitutes the input of the multi-domain information fusion module, and the input of the fusion module is simultaneously connected to the input of the multi-domain information fusion module. The output of the fusion module constitutes the output of the multi-domain information fusion module.

[0059] The input of the feature extractor constitutes the input of the model to be trained. The output of the feature extractor is connected to the input of the generalized mean pooling module GeM. The output of the generalized mean pooling module GeM is connected to the input of the multi-domain information fusion module. Here, the generalized mean pooling module GeM is a classic pooling algorithm that can effectively improve the quality of the feature matrix. The output of the multi-domain information fusion module is connected to the input of the target domain specific normalization layer DSBN_T. The output of the target domain specific normalization layer DSBN_T is connected to the input of the target hybrid memory HM_T. The output of the target hybrid memory HM_T constitutes the output of the model to be trained.

[0060] Step B. Based on the model to be trained and the multi-source domain contrastive loss module L HMand sample sets S from each source domain i The combined structures of the source domain-specific normalization layer DSBN_Si and the source hybrid memory HM_Si are as follows, based on the sample sets S of each source domain. i The target domain sample set T is used to train the model to obtain a target person recognition model for each target person.

[0061] like Figure 1 As shown, the combined structure is as follows: the output of the multi-domain information fusion module is connected to the input of the target domain-specific normalization layer DSBN_T and the input of each source domain-specific normalization layer DSBN_Si, respectively. The output of the target domain-specific normalization layer DSBN_T is connected to the input of the target hybrid memory HM_T, and the output of each source domain-specific normalization layer DSBN_Si is connected to the input of the corresponding source hybrid memory HM_Si. Simultaneously, the outputs of the target domain-specific normalization layer DSBN_T, the target hybrid memory HM_T, each source domain-specific normalization layer DSBN_Si, and each source hybrid memory HM_Si are all connected to the multi-source domain contrast loss module L. HM The input terminal.

[0062] Based on the above combined structure, according to the sample sets S of each source domain i The target domain sample set T is used to train the model to be trained in the following data processing manner to obtain the target person recognition model for each target person.

[0063] Step i. Apply feature extractors to each source domain sample set S. i Feature extraction is performed on the target domain sample set T to obtain the sample sets S of each source domain. i The corresponding feature matrices and the feature matrix corresponding to the target domain sample set T Then proceed to step ii.

[0064] In practical applications, the above step i is specifically executed as follows: for each source domain sample set S i The source domain sample sets S and the target domain sample set T are first processed through a first-layer residual network L0 and a second-layer residual network L1 for feature extraction to obtain each source domain sample set S. i The corresponding characteristic matrices S′ i The feature matrix T′ corresponding to the target domain sample set T is sent to the first precise feature distribution matching module EFDM1 for processing.

[0065] Next, in the processing of the first precise feature distribution matching module EFDM1, each feature matrix S′ is processed separately. iAnd the characteristic matrix T′, first sort the rows of the characteristic matrix in ascending order of value to obtain the characteristic matrix S″. i And the feature matrix T″, and then for each feature matrix S″ i The corresponding feature matrices are obtained according to the following formula (1).

[0066]

[0067] The first precise feature distribution matching module EFDM1 outputs each feature matrix. The feature matrix T″ is processed by the third residual network L2, where <·> indicates stopping the gradient operation, λ~Beta(α,a), α∈(0,∞) are adjustable hyperparameters, and Beta(α,a) represents a beta distribution with shape parameter α.

[0068] Then, in the third residual network L2 processing, each feature matrix is ​​processed separately. Feature extraction is performed to obtain the corresponding feature matrices. The feature matrix T″ is then fed into the second precise feature distribution matching module EFDM2 for processing. In the processing of the second precise feature distribution matching module EFDM2, each feature matrix is ​​processed separately. First, sort the rows of the feature matrix in ascending order of values ​​to obtain each feature matrix. Then, for each feature matrix... The corresponding feature matrices are obtained according to the following formula (2).

[0069]

[0070] The second precise feature distribution matching module EFDM2 outputs each feature matrix. And the characteristic matrix T″;

[0071] Finally, each feature matrix is ​​analyzed separately. The feature matrix T″ is then processed sequentially through the fourth residual network L3 and the fifth residual network L4 for feature extraction, thus updating and obtaining the sample sets S of each source domain. i The corresponding feature matrices and the feature matrix corresponding to the target domain sample set T

[0072] Step ii. For each source domain sample set S i Corresponding feature matrix and the feature matrix corresponding to the target domain sample set T The generalized mean pooling module GeM processes and updates the sample sets S of each source domain. i Corresponding feature matrix and the feature matrix corresponding to the target domain sample set T Then proceed to step iii.

[0073] Step iii. For each source domain sample set S i Corresponding feature matrix and the feature matrix corresponding to the target domain sample set T The multi-domain information fusion module processes the data to obtain sample sets S from each source domain. i The corresponding feature matrices and the feature matrix corresponding to the target domain sample set T Then proceed to step iv.

[0074] In practical applications, step iii above is performed by the multi-domain information fusion module as follows:

[0075] Step iii-1. For each source domain sample set S... i Corresponding feature matrix and the feature matrix corresponding to the target domain sample set T According to the following formula (3):

[0076]

[0077] Obtain each feature matrix The corresponding domain proxy node vectors and feature matrix Corresponding domain proxy node vector N represents the number of instances in the feature matrix. Representation of the characteristic matrix The vector corresponding to the nth instance, w in Representation of the characteristic matrix The preset learnable weighted vector corresponding to the nth instance. Representation of the characteristic matrix The vector corresponding to the nth instance, w n Representation of the characteristic matrix The preset learnable weighted vector corresponding to the nth instance;

[0078] Step iii-2. According to the following formula (4):

[0079]

[0080] Obtain the multi-domain information fusion matrix F (0) , where ∪ represents stacking by columns, and C represents the length of each row of the feature matrix;

[0081] Step iii-3. Construct the adjacency matrix of the first graph convolutional neural network GCN1 and the second graph convolutional neural network GCN2 according to the following formula (5). l = 1, 2, A (1) Let A represent the adjacency matrix of the first graph convolutional neural network GCN1. (2) This represents the adjacency matrix of the second graph's convolutional neural network, GCN2.

[0082]

[0083] Where r and c represent the adjacency matrix A, respectively. (l) Row index and column index, Represents the adjacency matrix A (l) The value in the r-th row and c-th column;

[0084] Step iii-4. Based on the adjacency matrix A of the convolutional neural network GCN1 in the first graph... (1) For the multi-domain information fusion matrix F (0) According to the following formula (6):

[0085]

[0086] Obtain the multi-domain information fusion matrix F (0) The multi-domain information fusion matrix F after processing by the first convolutional neural network GCN1 is (1) W 1 This represents the preset learnable weighting matrix corresponding to the convolutional neural network GCN1 in the first image;

[0087] Step iii-5. Based on the adjacency matrix A of the convolutional neural network GCN2 in the second figure... (2) For the multi-domain information fusion matrix F (1) According to the following formula (7):

[0088]

[0089] Obtain the multi-domain information fusion matrix F (1) The multi-domain information fusion matrix F after processing by the second convolutional neural network GCN2 is shown in Figure 2. (2) W 2 This represents the preset learnable weighting matrix corresponding to the GCN2 convolutional neural network in the second graph;

[0090] Step iii-6. For the multi-domain information fusion matrix F (0) With multi-domain information fusion matrix F (2) The multi-domain information fusion feature matrix is ​​obtained by the fusion module. And targeting the feature matrix of multi-domain information fusion According to the corresponding source domain sample sets S iThe target domain sample set T is decomposed to obtain the sample sets S of each source domain. i The corresponding feature matrices and the feature matrix corresponding to the target domain sample set T

[0091] Step iv. Feature matrices and feature matrix The corresponding feature matrices are obtained by processing the data through the source domain-specific normalization layer DSBN_Si and the target domain-specific normalization layer DSBN_T, respectively. and feature matrix

[0092] Step v. Feature matrices and feature matrix The samples are processed by the corresponding source hybrid memory HM_Si and the target hybrid memory HM_T respectively to obtain the source domain sample sets S output by each source hybrid memory HM_Si. i Source domain identity tags and the feature matrix of all unclustered target domain instances output by the target hybrid memory HM_T from the target domain sample set T. Pseudo-labels for target domain clustering

[0093] Step vi. For the feature matrix Each feature matrix Identity tags of each source domain and features of all unclustered target domain instances in the target domain sample set T Pseudo-labels for target domain clustering Application of multi-source domain contrast loss module L HM Perform loss calculation.

[0094] In practical applications, in step vi above, the feature matrix is... Each feature matrix Identity tags of each source domain and the feature matrix of all unclustered target domain instances in the target domain sample set T Pseudo-labels for target domain clustering Application of multi-source domain contrast loss module L HM The loss is calculated according to the following formula (9);

[0095]

[0096] Obtain the loss calculation result L HM ,in, f j Let f be the j-th vector. The k-th eigenvector in express The m-th eigenvector in the middle, yes The 0th eigenvector in the vector, where <·, ·> represent the inner product between two eigenvectors, z + Indicates with f j The corresponding positive class prototype, σ represents the temperature parameter; n s express The number of eigenvectors in n v express The number of eigenvectors in n c express The number of eigenvectors in the model.

[0097] Based on the target person recognition models obtained above, the target person recognition models are further applied, and the following step C is performed to achieve the recognition of the target person for the image to be analyzed.

[0098] Step C. The target person recognition model receives the image of the person to be analyzed and processes it sequentially through the feature extractor, the generalized mean pooling module GeM, the multi-domain information fusion module, the target domain specific normalization layer DSBN_T, and the target hybrid memory HM_T to obtain the target domain cluster pseudo-label corresponding to the image of the person to be analyzed, that is, to obtain the identity label recognition result of the image of the person to be analyzed regarding the target person.

[0099] The aforementioned technical solution, based on precise feature distribution matching and multi-domain information fusion, is a cross-domain person re-identification method. Addressing the differences in feature distribution between the source and target domains, as well as the domain differences that exist during training with multi-source datasets, it utilizes a feature extractor, a generalized mean pooling module (GeM), a multi-domain information fusion module, a target domain-specific normalization layer (DSBN_T), and a target hybrid memory (HM_T) to create a model to be trained. This model is then trained using both source and target domain sample sets to obtain a target person recognition model. This model is then used to identify various target persons within an image. Experiments show that this invention offers superior performance in practical applications compared to existing technologies, effectively improving the accuracy and efficiency of person recognition in images.

[0100] The cross-domain person re-identification method based on accurate feature distribution matching and multi-domain information fusion designed in this invention is applied in practice, and compared with advanced UDA-ReID methods MMT, SpCL, GLT, and M... 3 A comparison of L and UMSDA is shown in Table 1 below:

[0101] Table 1

[0102]

[0103] As can be seen, compared with the sub-optimal domain adaptation method GLT, the proposed method improves mAP and rank-1 by 2.4% and 1.6% respectively when Market1501 is used as the test dataset; and by 4.6% and 3.1% respectively when MSMT17 is used as the test dataset. Furthermore, compared with the sub-optimal multi-source domain model UMSDA, the proposed method outperforms the model by 0.4% and 1.6% in mAP and rank-1 respectively when MSMT17 and CUHK03-NP are used as the source domains and Market1501 is used as the target domain; and by 15.3% and 21.4% respectively when Market1501 and CUHK03-NP are used as the source domains and MSMT17 is used as the target domain, indicating that the proposed method is more adaptable to multi-source domain tasks.

[0104] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A cross-domain person re-identification method based on accurate feature distribution matching and multi-domain information fusion, characterized in that: The source domain sample set is based on the sample images of target individuals with known identity tags corresponding to each shooting angle. The target domain sample set combines sample images containing each target person, but whose corresponding identity tags are unknown. Perform steps A to B to obtain a target person recognition model for each target person; then apply the target person recognition model and perform step C to identify the target person for the image to be analyzed. , The number of shooting angles indicated. Indicates the first Source domain sample set corresponding to each shooting angle; Step A. Construct a training model that takes the image of the person as input and the identification results of the target person's identity tags as output, and then proceed to Step B; The model to be trained includes a feature extractor, a generalized mean pooling module GeM, a multi-domain information fusion module, a target domain-specific normalization layer DSBN_T, and a target hybrid memory HM_T. The feature extractor comprises, from its input to its output, a first residual network L0, a second residual network L1, a first precise feature distribution matching module EFDM1, a third residual network L2, a second precise feature distribution matching module EFDM2, a fourth residual network L3, and a fifth residual network L4, all connected in series. The input of the first residual network L0 constitutes the input of the feature extractor, and the output of the fifth residual network L4 constitutes the output of the feature extractor. The multi-domain information fusion module, from its input end to its output end, includes a first graph convolutional neural network GCN1, a leaky modified linear unit ReLU, a second graph convolutional neural network GCN2, and a fusion module connected in series. The input end of the first graph convolutional neural network GCN1 constitutes the input end of the multi-domain information fusion module, and the input end of the fusion module is simultaneously connected to the input end of the multi-domain information fusion module. The output end of the fusion module constitutes the output end of the multi-domain information fusion module. The input of the feature extractor constitutes the input of the model to be trained. The output of the feature extractor is connected to the input of the generalized mean pooling module GeM. The output of the generalized mean pooling module GeM is connected to the input of the multi-domain information fusion module. The output of the multi-domain information fusion module is connected to the input of the target domain specific normalization layer DSBN_T. The output of the target domain specific normalization layer DSBN_T is connected to the input of the target hybrid memory HM_T. The output of the target hybrid memory HM_T constitutes the output of the model to be trained. Step B. Based on the multi-source domain contrast loss module and sample sets from each source domain The corresponding source domain-specific normalization layers DSBN_S Hybrid Memory HM_S Based on the sample sets of each source domain and target domain sample set Train the model to be trained to obtain a target person recognition model for each target person; Step C. The target person recognition model receives the image of the person to be analyzed, processes it, and obtains the identification result of the target person's identity label in the image of the person to be analyzed.

2. The cross-domain person re-identification method based on accurate feature distribution matching and multi-domain information fusion according to claim 1, characterized in that: In step B, the model to be trained is compared with the multi-source domain contrast loss module. and sample sets from each source domain The corresponding source domain-specific normalization layers DSBN_S Hybrid Memory HM_S The combined structure is as follows, based on the sample sets of each source domain. and target domain sample set Train the model to be trained to obtain a target person recognition model for each target person; Combined structure: The output of the multi-domain information fusion module is respectively connected to the input of the target domain-specific normalization layer DSBN_T and the source domain-specific normalization layers DSBN_S. The input terminal of the target domain-specific normalization layer DSBN_T is connected to the input terminal of the target hybrid memory HM_T, and the output terminal of each source domain-specific normalization layer DSBN_S is connected to the input terminal of the target hybrid memory HM_T. The output terminals are respectively connected to the corresponding source hybrid memory HM_S The input terminal, and simultaneously the output terminal of the target domain-specific normalization layer DSBN_T, the output terminal of the target hybrid memory HM_T, and the source domain-specific normalization layers DSBN_S Output terminal, HM_S hybrid memory from various sources The output terminals are respectively connected to the multi-source domain contrast loss module. The input terminal.

3. The cross-domain person re-identification method based on accurate feature distribution matching and multi-domain information fusion according to claim 2, characterized in that: In step B, based on the sample sets of each source domain and target domain sample set The following data processing method is used to train the model to be trained, thereby obtaining a target person recognition model for each target person. step Feature extractors are applied to each source domain sample set. and target domain sample set Feature extraction is performed to obtain sample sets from each source domain. The corresponding feature matrices and target domain sample set Corresponding feature matrix Then proceed to the next step. ; step For each source domain sample set... Corresponding feature matrix and target domain sample set Corresponding feature matrix The generalized mean pooling module GeM processes and updates the sample sets of each source domain. Corresponding feature matrix and target domain sample set Corresponding feature matrix Then proceed to the next step. ; step For each source domain sample set... Corresponding feature matrix and target domain sample set Corresponding feature matrix The data is processed by a multi-domain information fusion module to obtain sample sets from each source domain. The corresponding feature matrices and target domain sample set Corresponding feature matrix Then proceed to the next step. ; step Each characteristic matrix and feature matrix Each source domain is passed through a specific normalization layer DSBN_S. The target domain-specific normalization layer DSBN_T is then used to process the data to obtain the corresponding feature matrices. and feature matrix ; step Each characteristic matrix and feature matrix Each is transmitted through its corresponding source hybrid memory HM_S The target hybrid memory HM_T is processed to obtain the source hybrid memory HM_S. The corresponding source domain sample sets are output respectively Source domain identity tags and the target domain sample set output by the target hybrid memory HM_T. Feature matrix of all unclustered target domain instances Pseudo-labels for target domain clustering ; step For the feature matrix Each feature matrix Identity tags of each source domain and target domain sample set Features of all unclustered target domain instances Pseudo-labels for target domain clustering Applying a multi-source domain contrast loss module Perform loss calculation.

4. The cross-domain person re-identification method based on accurate feature distribution matching and multi-domain information fusion according to claim 3, characterized in that: The steps In the middle, for each source domain sample set and target domain sample set First, features are extracted by sequentially passing the first residual network L0 and the second residual network L1 to obtain the sample sets of each source domain. The corresponding feature matrices and target domain sample set Corresponding feature matrix The data is then sent to the first precise feature distribution matching module EFDM1 for processing. Next, in the processing of the first precise feature distribution matching module EFDM1, each feature matrix is ​​processed separately. and feature matrix First, sort the rows of the feature matrix in ascending order of values ​​to obtain each feature matrix. and feature matrix Then, for each feature matrix The corresponding feature matrices are obtained according to the following formula (1). ; （1） The first precise feature distribution matching module EFDM1 outputs each feature matrix. and feature matrix The process is then carried out in the third residual network L2, where... This indicates that the gradient operation is stopped. , These are adjustable hyperparameters. Represented as shape parameters beta distribution; Then, in the third residual network L2 processing, each feature matrix is ​​processed separately. Feature extraction is performed to obtain the corresponding feature matrices. And combined with the feature matrix The data is then sent to the second precise feature distribution matching module EFDM2 for processing. In the processing of the second precise feature distribution matching module EFDM2, each feature matrix is ​​processed separately. First, sort the rows of the feature matrix in ascending order of values ​​to obtain each feature matrix. Then, for each feature matrix The corresponding feature matrices are obtained according to the following formula (2). ; （2） The second precise feature distribution matching module EFDM2 outputs each feature matrix. and feature matrix ; Finally, each feature matrix is ​​analyzed separately. and feature matrix The features are extracted sequentially through the fourth residual network L3 and the fifth residual network L4, thus updating and obtaining the sample sets of each source domain. The corresponding feature matrices and target domain sample set Corresponding feature matrix .

5. The cross-domain person re-identification method based on accurate feature distribution matching and multi-domain information fusion according to claim 3, characterized in that: The steps The multi-domain information fusion module performs the following steps: step -1. For each source domain sample set respectively Corresponding feature matrix and target domain sample set Corresponding feature matrix According to the following formula (3): （3） Obtain each feature matrix The corresponding domain proxy node vectors and feature matrix Corresponding domain proxy node vector , This indicates the number of instances in the feature matrix. Representation of the characteristic matrix The Middle The vector corresponding to each instance Representation of the characteristic matrix The Middle The pre-defined learnable weighted vector corresponding to each instance Representation of the characteristic matrix The Middle The vector corresponding to each instance Representation of the characteristic matrix The Middle Preset learnable weighted vectors corresponding to each instance; step -2. According to the following formula (4): （4） Obtaining a multi-domain information fusion matrix ,in This indicates stacking by columns, where C represents the length of each row of the feature matrix; step -3. Construct the adjacency matrix of the first graph convolutional neural network GCN1 and the second graph convolutional neural network GCN2 according to the following formula (5). , , This represents the adjacency matrix of the first graph convolutional neural network GCN1. This represents the adjacency matrix of the second graph's convolutional neural network, GCN2. （5） in, and Represent the adjacency matrix respectively Row index and column index, Representing the adjacency matrix The Middle Line number The value of the column; step -4. Based on the adjacency matrix of the GCN1 convolutional neural network in the first diagram. For multi-domain information fusion matrix According to the following formula (6): （6） Obtaining a multi-domain information fusion matrix The multi-domain information fusion matrix after processing by the GCN1 convolutional neural network in the first image , This represents the preset learnable weighting matrix corresponding to the first convolutional neural network GCN1; step -5. Based on the adjacency matrix of the GCN2 convolutional neural network in the second figure. For multi-domain information fusion matrix According to the following formula (7): （7） Obtaining a multi-domain information fusion matrix The multi-domain information fusion matrix after processing by the GCN2 convolutional neural network in the second graph. , This represents the preset learnable weighting matrix corresponding to the GCN2 convolutional neural network in the second graph; step -6. For multi-domain information fusion matrices Multi-domain information fusion matrix The multi-domain information fusion feature matrix is ​​obtained by the fusion module. And targeting the feature matrix of multi-domain information fusion According to the corresponding source domain sample sets and target domain sample set Decompose the sample sets to obtain the sample sets of each source domain. The corresponding feature matrices and target domain sample set Corresponding feature matrix .

6. The cross-domain person re-identification method based on accurate feature distribution matching and multi-domain information fusion according to claim 3, characterized in that: The steps In the context of the feature matrix Each feature matrix Identity tags of each source domain and target domain sample set Feature matrix of all unclustered target domain instances Pseudo-labels for target domain clustering Applying a multi-source domain contrast loss module The loss is calculated according to the following formula (9); （9） Obtain the loss calculation results ,in, , for The first in A vector, , yes The first in 1 eigenvector express The Middle 1 eigenvector yes The o-th eigenvector in This represents the inner product between two eigenvectors. Indicates and The corresponding positive class prototype, Indicates temperature parameter; , , , express The number of feature vectors in the middle. express The number of eigenvectors in the middle. express The number of eigenvectors in the model.

7. The cross-domain person re-identification method based on accurate feature distribution matching and multi-domain information fusion according to claim 1, characterized in that: In step C, the target person recognition model receives the image of the person to be analyzed and processes it sequentially through a feature extractor, a generalized mean pooling module GeM, a multi-domain information fusion module, a target domain specific normalization layer DSBN_T, and a target hybrid memory HM_T to obtain the target domain clustering pseudo-labels corresponding to the image of the person to be analyzed, that is, to obtain the identity label recognition result of the image of the person to be analyzed regarding the target person.

Images