Camera invariant feature fused domain generalization pedestrian re-identification method

Abstract

The invention discloses a domain generalization pedestrian re-identification method fusing camera invariant features. The method comprises the following steps: acquiring a pedestrian image data set to be re-identified; designing a normalization mitigation module based on an instance normalization module and a channel attention mechanism, adopting a ResNet-50 backbone network, and inserting the normalization mitigation module in ResNet-50 to construct a domain style normalization module of the domain generalization pedestrian re-identification model; a part spatial relation feature extraction module and a fine discrimination mode mining module are designed, and a domain generalization pedestrian re-identification model is constructed based on the domain style normalization module; and taking the pedestrian image data set to be re-identified as input of the trained domain generalization pedestrian re-identification model to obtain a pedestrian re-identification result of the pedestrian image data set to be re-identified. According to the method, the inherent space proportional relation of different parts of the pedestrian is constructed, the camera invariant features are extracted, and the fine discrimination information in the image is deeply explored, so that the robustness of the figure appearance features in the complex and changeable environment is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a domain generalized pedestrian re-identification method integrating camera invariant features. Background Art

[0002] Person re-identification refers to retrieving a specific person from a large number of pedestrian images collected by different cameras. It is an important sub-topic in the field of computer vision after face recognition. In recent years, with the improvement of monitoring equipment and the improvement of people's safety awareness, more public places, especially those with large traffic, have begun to pay attention to the application of pedestrian re-identification. The task of pedestrian re-identification has been widely studied in ordinary cameras at fixed positions and has achieved great results. However, the existing research on the appearance feature extraction of pedestrian re-identification with domain generalization mainly extracts domain-invariant features by feature decoupling to reduce the distribution differences between different data domains. However, feature decoupling is essentially a more complex problem than domain generalization, and is limited by the distribution, source and number of training data domains, resulting in deficiencies in extracting image visual features. The appearance of a person is prone to change across camera perspectives, but the spatial proportional relationship between different parts can remain consistent, which provides a new perspective for distinguishing pedestrians of different body shapes. In addition, subtle distinguishability information, such as glasses, shoes, bags and clothing length, has not been fully utilized. Existing methods focus on extracting domain-invariant features, which limits the distinguishing ability of features.

[0003] The core of person re-identification technology lies in effectively extracting the appearance features of people. Although existing supervised learning methods have achieved remarkable results in obtaining stable and effective appearance features by improving network structures and optimizing loss functions, these traditional methods are difficult to adapt to the needs of domain generalization person re-identification due to the unknown nature of the target domain and the huge differences between the source domain and the target domain in terms of camera style, posture, and perspective. In addition, the current domain generalization person re-identification methods mainly focus on feature decoupling learning in extracting appearance features. Although this method is effective, it is limited by the diversity of source domain data, and decoupling learning is inherently more challenging and less interpretable than domain generalization.

[0004] Therefore, how to effectively extract and utilize pedestrian appearance features to reduce the distribution differences between different data domains and improve the robustness of human appearance features in complex and changing environments. Summary of the invention

[0005] In view of the above-mentioned deficiencies of the prior art, the present invention provides a domain generalized pedestrian re-identification method integrating camera invariant features, which constructs the inherent spatial proportional relationship of different parts of pedestrians, extracts camera invariant features, and deeply explores the subtle discriminant information in the image, thereby enhancing the robustness of the appearance features of people in complex and changing environments.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A method for domain generalization person re-identification by integrating camera invariant features comprises the following steps:

[0008] Obtain a dataset of pedestrian images to be re-identified;

[0009] A normalization mitigation module based on an instance normalization module and a channel attention mechanism is designed, ResNet-50 is used as the backbone network of a domain generalized person re-identification model, and the normalization mitigation module is inserted into the backbone network ResNet-50 to construct a domain style normalization module of the domain generalized person re-identification model;

[0010] Design a part spatial relationship feature extraction module and a subtle discrimination pattern mining module, and build a domain generalization person re-identification model based on the domain style normalization module;

[0011] The pedestrian image dataset to be re-identified is used as the input of the trained domain generalization pedestrian re-identification model to obtain the pedestrian re-identification result of the pedestrian image dataset to be re-identified.

[0012] As a preferred solution, the trained domain generalized pedestrian re-identification model uses the input pedestrian image dataset to be re-identified as the input of the domain style normalization module, extracts domain-invariant appearance features with distinguishability, and then extracts features of the domain-invariant appearance features through the part spatial relationship feature extraction module, so as to obtain stable spatial relationship features between pedestrian parts under different camera styles, and splices the output of the domain style normalization module with the output of the part spatial relationship feature extraction model to obtain fused features. Finally, the fused features are extracted through the subtle discriminant pattern mining module to obtain subtle discriminant features, thereby obtaining the pedestrian re-identification result as the output of the domain generalized pedestrian re-identification model.

[0013] As a preferred solution, the domain style normalization module includes a backbone network ResNet-50 and a first normalization relief module and a second normalization relief module respectively inserted into convolution block 3 and convolution block 4 in the backbone network ResNet-50; wherein the backbone network ResNet-50 includes 5 cascaded convolution blocks, and the first normalization relief module has the same structure as the second normalization relief module;

[0014] Among them, the pedestrian image dataset to be re-identified as the input of the domain generalization pedestrian re-identification model is used as the input of convolution block 1, the output of each convolution block is used as the input of the next convolution block in the domain generalization pedestrian re-identification model, the output of convolution block 3 is used as the input of the first normalization relief module, the output of the first normalization relief module is used as the input of convolution block 4, the output of convolution block 4 is used as the input of the second normalization relief module, the output of the second normalization relief module is used as the input of convolution block 5, and the output of convolution block 5 is used as the overall output of the domain style normalization module.

[0015] As a preferred solution, the processing process of the normalization mitigation module is:

[0016]

[0017] f boost =λ*CA(X)⊙X;

[0018] F=IN(X)⊙(1-λ*CA(X))+f boost ;

[0019] Where IN(·) is instance normalization, is the Nth dimension of the input feature map X, n is an auxiliary value, γ and β are hyperparameters that can be learned during training, E(·) and Var(·) are the mean and variance of the feature map, respectively, and f boost is the enhanced feature, CA(·) is the channel attention, λ is the hyperparameter that controls the intensity of channel attention, ⊙ is the element-based multiplication, and F is the feature map obtained after the normalization mitigation module.

[0020] As a preferred solution, the part spatial relationship feature extraction module includes two 3D projection convolution units and one 2D projection convolution unit; the 3D projection convolution unit includes a cascaded PT3D module and a three-dimensional convolution block 4, and the 2D projection convolution unit includes a cascaded three-dimensional convolution block 5 and a PT2D module;

[0021] Among them, the output of the first normalization relief module in the domain style normalization module is used as the input of the first 3D projection convolution unit, and the output of the second normalization relief module in the domain style normalization module is used as the input of the second 3D projection convolution unit. The output of the first normalization relief module and the output of the second normalization relief module are added and fused, and transmitted to the 2D projection convolution unit. The output of the 2D projection convolution unit is spliced ​​with the output of the convolution block 5 in the domain style normalization module to obtain the fused feature.

[0022] As a preferred solution, the processing process of the part spatial relationship feature extraction module includes the following steps:

[0023] The input feature map is projected onto the 3D feature space using the PT3D unit, and two 3D convolution kernels of 3×3×3 are stacked using the 3D convolution block 4_1 to perform a convolution operation on the 3D feature map, thereby extracting the part relationship features of different channel feature maps; then, the feature map output by the second normalization relief module in the domain style normalization module is used as an additional input to the part space relationship feature extraction module, and additional part relationship features are extracted through the PT3D unit and the 3D convolution block 4_2; then, the relationship features of the two branches are fused in an additive manner, and the fused features are extracted again using the 3D convolution block 5, and then, the obtained 3D part relationship feature map is projected into the 2D space through the PT2D unit to obtain a 2D feature map with part relationship, and finally, the obtained 2D feature map with part relationship is spliced ​​with the output feature map of the convolution block 5 in the domain style normalization module to obtain a fused feature.

[0024] As a preferred solution, the processing process of the subtle discrimination pattern mining module includes the following steps:

[0025] The input fusion feature is reintegrated through a 1×1 convolutional layer, and the integrated feature map is normalized through a Sigmoid function to obtain a pattern feature map. At the same time, a separation loss function is constructed to ensure the diversity of the pattern feature map. The pattern feature map is optimized and updated with the goal of minimizing the separation loss function. Subsequently, the fusion feature map is divided into l patterns using the pattern feature map. Finally, the global average pooling operation is used to extract the features of each pattern, thereby obtaining subtle discriminant features.

[0026] The processing process is expressed by the following formula:

[0027]

[0028] Where P k M k and The result after element-wise multiplication, M k is the pattern feature map, is the fusion feature, T is the transposition operation, p k is the feature of each pattern extracted by global average pooling, and f is the subtle discriminant feature.

[0029] As a preferred solution, the separation loss function is:

[0030]

[0031] Where, L sep is the separation loss function, (M T M) ij MT The element of the M matrix at the i-th row and j-th column.

[0032] As a preferred solution, the domain generalization person re-identification model is trained in the following manner:

[0033] Pedestrian images with pre-labeled pedestrian features are used as training samples to form a training sample set, which is input into the domain generalized pedestrian re-identification model, and a central loss function is constructed to learn the relationship between different pedestrians. The model parameters of the domain generalized pedestrian re-identification model are optimized and updated with the goal of minimizing the central loss function, and then the domain generalized pedestrian re-identification model is trained.

[0034] As a preferred solution, the center loss function is:

[0035]

[0036] Where, L Center is the center loss function, N ID is the total number of different pedestrian identities in the current batch, is the feature mean of the pedestrians labeled qi in the current batch, P is the number of pedestrian identities in the current batch, ρ is the minimum interval between different centers, is the feature mean of the pedestrians with label qk in the current batch, is the feature mean of the pedestrians with label qj in the current batch.

[0037] Compared with the prior art, the present invention has the following technical effects:

[0038] (1) Based on instance normalization (IN) and channel attention mechanism, the present invention designs a normalization alleviating module (NAM) to extract domain-invariant appearance features with distinguishability while retaining the discriminative information of different persons. This design avoids the loss of discriminative identity information caused by excessive use of the IN module, thereby reducing the style differences between different data domains while enhancing the model's ability to recognize individual features, thereby improving the accuracy and robustness of pedestrian re-identification.

[0039] (2) The present invention utilizes three-dimensional convolution operation to capture the spatial relationship features between different parts of pedestrians, and converts the two-dimensional feature map into a three-dimensional feature space through a PT3D projection operation, so that the three-dimensional convolution can process and extract features. In addition, multiple convolution branches are used to extract relationship features of different levels and different parts, which increases the diversity of features and the model's ability to express pedestrian appearance features. In this way, by extracting the spatial relationship features of parts, the camera-style-invariant pedestrian part relationship features are obtained to adapt to the camera style changes of unknown target domains. At the same time, the present invention aims at domain generalization pedestrian re-identification, and distinguishes pedestrians with similar appearances through a subtle discriminant model mining module, extracts subtle discriminant features with distinguishability and diversity, and further enhances the expression ability of image features in domain generalization scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0041] Figure 1 The present invention discloses a flow chart of a method for domain generalization pedestrian re-identification integrating camera invariant features;

[0042] Figure 2 A schematic flow chart of a method for domain generalization person re-identification integrating camera invariant features according to the present invention;

[0043] Figure 3 This is a schematic diagram of the structure of the normalized mitigation module of the present invention;

[0044] Figure 4 This is a schematic diagram of the PT3D unit structure of the present invention;

[0045] Figure 5 This is a schematic diagram of the PT2D unit structure of the present invention;

[0046] Figure 6 This is an example diagram of the relationship between pedestrian parts according to an embodiment of the present invention;

[0047] Figure 7 This is an example diagram of subtle discrimination pattern mining according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0049] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0050] When it comes to extracting pedestrian appearance features, although the existing supervised learning methods have achieved remarkable results in obtaining stable and effective appearance features by improving the network structure and optimizing the loss function, these traditional methods are difficult to adapt to the needs of domain generalization pedestrian re-identification due to the unknown nature of the target domain and the huge differences between the source domain and the target domain in terms of camera style, posture, and perspective. In addition, the current domain generalization pedestrian re-identification methods mainly focus on feature decoupling learning in extracting appearance features. Although this method is effective, it is limited by the diversity of source domain data, and decoupling learning is inherently more challenging and less interpretable than domain generalization.

[0051] Therefore, in response to the above problems and shortcomings, the present invention constructs the inherent spatial proportional relationship of different parts of pedestrians, extracts camera invariant features, and deeply explores the subtle discriminant information in the image. A domain generalized pedestrian re-identification method integrating camera invariant features is proposed, aiming to enhance the robustness of the appearance features of people in complex and changing environments.

[0052] Specifically, the present invention proposes a domain generalization pedestrian re-identification method that integrates camera invariant features, such as Figure 1 As shown, the method includes:

[0053] Obtain a dataset of pedestrian images to be re-identified;

[0054] A normalization mitigation module based on an instance normalization module and a channel attention mechanism is designed, ResNet-50 is used as the backbone network of a domain generalized person re-identification model, and the normalization mitigation module is inserted into the backbone network ResNet-50 to construct a domain style normalization module of the domain generalized person re-identification model;

[0055] Design a part spatial relationship feature extraction module and a subtle discrimination pattern mining module, and build a domain generalization person re-identification model based on the domain style normalization module;

[0056] The pedestrian image dataset to be re-identified is used as the input of the trained domain generalization pedestrian re-identification model to obtain the pedestrian re-identification result of the pedestrian image dataset to be re-identified.

[0057] The method for domain generalization person re-identification by integrating camera invariant features of the present invention is described in more detail below.

[0058] 1. Domain Generalization Person Re-ID Model

[0059] like Figure 2 As shown in the figure, the present invention proposes a method for domain generalized pedestrian re-identification that integrates camera invariant features. The method first constructs a domain style normalization module to extract distinguishable domain invariant appearance features; then, through the pedestrian part spatial relationship feature extraction module, the stable spatial relationship features between pedestrian parts under different camera styles are captured; finally, by constructing a subtle discriminant pattern mining module, the discriminability of pedestrian appearance features in domain generalization scenarios is further improved.

[0060] Specifically, the trained domain generalized pedestrian re-identification model takes the input pedestrian image dataset to be re-identified as the input of the domain style normalization module, extracts domain-invariant appearance features with distinguishability, and then extracts features of the domain-invariant appearance features through the part spatial relationship feature extraction module, so as to obtain stable spatial relationship features between pedestrian parts under different camera styles, and splices the output of the domain style normalization module with the output of the part spatial relationship feature extraction model to obtain fused features. Finally, the fused features are extracted through the subtle discriminant pattern mining module to obtain subtle discriminant features, thereby obtaining the pedestrian re-identification result as the output of the domain generalized pedestrian re-identification model.

[0061] They are introduced in detail below.

[0062] 1.1 Domain Style Normalization Module

[0063] In the study of domain generalization person re-identification, instance normalization (IN) is usually used to eliminate style deviations in different data domains to obtain domain-invariant appearance features with distinguishability. However, the use of the IN module may lead to the loss of discriminative information, thereby affecting the model performance. The present invention designs a normalization mitigation module (NAM) based on the instance normalization module and the channel attention mechanism, in which the channel attention uses global average pooling and global maximum pooling to extract aggregated features to improve the style normalization effect, and in order to avoid the loss of discriminative identity information caused by excessive use of the IN module, the present invention uses ResNet-50 as the backbone network of the domain generalization person re-identification model, and inserts the normalization mitigation module into the backbone network ResNet-50, thereby reducing the domain style differences while retaining the distinguishable identity information to the maximum extent.

[0064] Specifically, the domain style normalization module of the present invention includes a backbone network ResNet-50 and a first normalization relief module and a second normalization relief module respectively inserted into convolution block 3 and convolution block 4 in the backbone network ResNet-50; wherein the backbone network ResNet-50 includes 5 cascaded convolution blocks, and the first normalization relief module has the same structure as the second normalization relief module;

[0065] Among them, the pedestrian image dataset to be re-identified as the input of the domain generalization pedestrian re-identification model is used as the input of convolution block 1, the output of each convolution block is used as the input of the next convolution block in the domain generalization pedestrian re-identification model, the output of convolution block 3 is used as the input of the first normalization relief module, the output of the first normalization relief module is used as the input of convolution block 4, the output of convolution block 4 is used as the input of the second normalization relief module, the output of the second normalization relief module is used as the input of convolution block 5, and the output of convolution block 5 is used as the overall output of the domain style normalization module.

[0066] like Figure 3 As shown in the figure, it is a schematic diagram of the normalized mitigation module structure, and its processing process is as follows:

[0067]

[0068] f boost =λ*CA(X)⊙X;

[0069] F=IN(X)⊙(1-λ*CA(X))+f boost ;

[0070] Where IN(·) is instance normalization, is the Nth dimension of the input feature map X, n is an auxiliary value, γ and β are hyperparameters that can be learned during training, E(·) and Var(·) are the mean and variance of the feature map, respectively, and f boost is the enhanced feature, CA(·) is the channel attention, λ is the hyperparameter that controls the intensity of channel attention, ⊙ is the element-based multiplication, and F is the feature map obtained after the normalization mitigation module.

[0071] 1.2. Part Spatial Relationship Feature Extraction Module

[0072] Pedestrian appearance feature extraction is easily affected by camera style changes, and the camera style of the unknown target domain cannot be known in advance. Therefore, for domain generalization pedestrian re-identification, extracting pedestrian appearance features that remain unchanged by the image camera style is a very challenging task. In this regard, the present invention extracts pedestrian part relationship features to describe the corresponding relationship between different parts of pedestrians in the image space, in order to adapt to the image style changes in the unknown data domain.

[0073] Specifically, the part spatial relationship feature extraction module (PRFE) includes two 3D projection convolution units and one 2D projection convolution unit; the 3D projection convolution unit includes a cascaded PT3D module and a three-dimensional convolution block 4, and the 2D projection convolution unit includes a cascaded three-dimensional convolution block 5 and a PT2D module;

[0074] Among them, the output of the first normalization relief module in the domain style normalization module is used as the input of the first 3D projection convolution unit, and the output of the second normalization relief module in the domain style normalization module is used as the input of the second 3D projection convolution unit. The output of the first normalization relief module and the output of the second normalization relief module are added and fused, and transmitted to the 2D projection convolution unit. The output of the 2D projection convolution unit is spliced ​​with the output of the convolution block 5 in the domain style normalization module to obtain the fused feature.

[0075] Specifically, it is known that different channels of the pedestrian appearance feature map describe the distinguishable information of different parts respectively. Then, the part relationship feature can be obtained by constructing the relationship between different channels of the appearance feature map. Then, the processing process of its part space relationship feature extraction module is as follows: the input feature map is projected into the 3D feature space using the PT3D unit, such as Figure 4As shown, a three-dimensional convolution block 4_1 is used, and two three-dimensional convolution kernels of 3×3×3 size are stacked to perform a convolution operation on the 3D feature map, and the part relationship features of different channel feature maps are extracted; then, the feature map output by the second normalization relief module in the domain style normalization module is used as an additional input of the part space relationship feature extraction module, and the additional part relationship features are extracted through the PT3D unit and the three-dimensional convolution block 4_2; then, the relationship features of the two branches are fused in an additive manner, and the three-dimensional convolution block 5 is used to extract the relationship features of the fused features again, and then, as shown in Figure 5 The PT2D unit shown projects the obtained three-dimensional part relationship feature map into a two-dimensional space to obtain a two-dimensional feature map with part relationship. Finally, the obtained two-dimensional feature map with part relationship is spliced ​​with the output feature map of convolution block 5 in the domain style normalization module to obtain a fused feature.

[0076] When this embodiment is specifically applied, Figure 6 As shown, Figure 6 (a) and Figure 6 (b) for the same pedestrian, Figure 6 (c) and Figure 6 (d) is another pedestrian. It is obvious that the appearance of pedestrians in the image is easily affected by the changes in the image camera style, such as Figure 6 (a) and Figure 6 (b) They belong to the same pedestrian, but have large appearance differences. However, through further observation, it can be found that the spatial correspondence between different parts has hardly changed, such as Figure 6 (a) and Figure 6 (b) The distance between the waist and the knee and the distance between the knee and the ankle. The ratio of the two distances is almost the same in the two images, which shows that the spatial relationship of the parts is not affected by the change of image style. In addition, the introduction of part relationship features can also further distinguish different pedestrians with similar appearances. Figure 6 (a) and Figure 6 (c) The two pedestrian images have similar appearances, but the spatial relationships of their parts are significantly different, such as the distance from head to shoulder and the distance from shoulder to elbow, the ratios of which are significantly different in the two images.

[0077] 1.3. Subtle discrimination pattern mining module

[0078] Person re-identification is, in a sense, a fine-grained recognition problem, that is, different categories with similar appearances are distinguished by subtle differences. In the field of person re-identification, efficient subtle discrimination pattern mining is of great significance for distinguishing pedestrians with similar appearances. Therefore, the subtle discrimination pattern mining module proposed in the present invention takes the fusion features obtained in the part spatial relationship feature extraction module (PAM) as the input of the module, reintegrates the input fusion features into the feature map through a 1×1 convolutional layer, and normalizes the integrated feature map between 0 and 1 through the Sigmoid function, and obtains the pattern feature map M=[M1,M2,…,M l ], then, the fused feature map is segmented into l patterns using the pattern feature map, and finally, the features of each pattern are extracted using the global average pooling operation to obtain subtle discriminative features;

[0079] The processing process is expressed by the following formula:

[0080]

[0081]

[0082] Where P k M k and The result after element-wise multiplication, M k is the pattern feature map, is the fusion feature, T is the transposition operation, p k is the feature of each pattern extracted by global average pooling, and f is the subtle discriminant feature.

[0083] In order to ensure the diversity of the pattern feature graph M, the present invention optimizes and updates the pattern feature graph by separating the loss function to minimize the separation loss function, so that each feature graph focuses on different patterns. The separation loss function is:

[0084]

[0085] Where, L sep is the separation loss function, (M T M) ij M T The element of the M matrix at the i-th row and j-th column.

[0086] When this embodiment is specifically applied, Figure 7 As shown, Figure 7The three images in the middle left, middle, and right represent three different pedestrians, but they have similar appearances, such as white T-shirts, gray shorts, and body shapes. Careful observation shows that there are subtle differences in hair color and shoes between the left image and the middle and right images, respectively, and that the right image has a unique pattern on its T-shirt. These subtle differences can be used to distinguish different pedestrians. In the field of supervised person re-identification, many studies have been conducted to enhance the robustness of pedestrian appearance features by extracting discriminative subtle information, but there are few studies in the field of domain generalization pedestrian re-identification. Therefore, the present invention conducts in-depth research on the problem of mining subtle discriminative patterns in domain generalization pedestrian re-identification, extracts subtle discriminative features that are distinguishable and diverse, and further enhances the expressive power of image features in domain generalization scenarios.

[0087] 2. Training of Domain Generalization Person Re-ID Model

[0088] In training the domain generalization person re-identification model, the present invention adopts the following method:

[0089] Pedestrian images with pre-labeled pedestrian features are used as training samples to form a training sample set, which is input into the domain generalized pedestrian re-identification model, and a central loss function is constructed to learn the relationship between different pedestrians. The model parameters of the domain generalized pedestrian re-identification model are optimized and updated with the goal of minimizing the central loss function, and then the domain generalized pedestrian re-identification model is trained.

[0090] Among them, the center loss function is:

[0091]

[0092] Where, L Center is the center loss function, N ID is the total number of different pedestrian identities in the current batch, is the feature mean of the pedestrians labeled qi in the current batch, P is the number of pedestrian identities in the current batch, ρ is the minimum interval between different centers, is the feature mean of the pedestrians with label qk in the current batch, is the feature mean of the pedestrians with label qj in the current batch.

[0093] 3. Example

[0094] 3.1 Comparison with State-of-the-Art Methods

[0095] In order to verify the domain generalized pedestrian re-identification method that integrates camera invariant features proposed in this embodiment, experiments are conducted using single-source DG-ReID and multi-source DG-ReID in the embodiment.

[0096] 3.1.1 Single-source DG-ReID

[0097] Since DG-ReID uses different evaluation schemes, this embodiment specifically selects three widely used schemes for experiments. For single-source domain experiments, this embodiment selects Market-1501, DukeMTMC-reID, and MSMT17 datasets because they are the most commonly used benchmarks. For ease of understanding, this embodiment simplifies them as M, D, and MS, respectively. The sub-experiments in the single-source domain setting include M→D and D→M scenarios. To ensure standardization between methods, only the training part of each dataset is used during training, and only the testing part is used during testing. The results shown in Table 1 show that the proposed method trained with Market1501 achieves better results than the state-of-the-art, achieving 35.3% mAP and 57.6% Rank-1 on DukeMTMC-relD. Under the D→M scheme, the Rank-1 accuracy of our method is improved by 1.3% and the mAP is improved by 0.8% compared to the previously highly recommended methods (e.g., MDA)

[39] . It is worth noting that SNR also focuses on extracting useful and useless features using channel attention, but we creatively add an additional branch to SE-Net and use global max pooling (GMP) to capture the most significant features in each channel to improve regularization. Therefore, under the D→M protocol, PFIEN outperforms SNR by 4.9% in both Rank-1 and mAP.

[0098] Table 1. Performance (%) compared with the state-of-the-art on the single-source DG-RelD task.

[0099]

[0100] 3.1.2 Multi-source DG-ReID

[0101] According to the standardized evaluation protocol of DG-ReID, this embodiment uses the leave-one-out method to divide the above four datasets into training and test sets. Specifically, three datasets are selected as training data, while one dataset is reserved for testing purposes. As shown in Table 2, this embodiment evaluates our method on these four datasets using ResNet-50 as the backbone network, showing superior performance on two evaluation metrics compared to other state-of-the-art methods. For example, in the D+MS+M→>C3 protocol, PFIEN improves the state-of-the-art by 1.6% and 3.1% in Rank-1 accuracy and mAP. It is worth noting that even with ResNet50-IBN as the backbone network, the method of this embodiment consistently outperforms other advanced methods overall. The state-of-the-art CINorm improves the IN layer and uses an instance-level dynamic recalibration module to obtain rich normalized features. However, this method ignores the extraction of fine-grained information features for human images.

[0102] Table 2 compares with the state-of-the-art domain generalization methods on four large-scale Person RelD benchmarks: Market-1501(M), DukeMTMC RelD(D), CUHK03(C3), and MSMT17(MS), and quantitatively evaluates the performance by mean average precision (mAP) and Rank-1.

[0103]

[0104] On average, the Rank-1 and mAP results dropped by 1.5% and 2.2%, respectively. The above experiments clearly demonstrate the superior performance of the method in this embodiment on various indicators, surpassing the current state-of-the-art and significantly outperforming the baseline. These findings undoubtedly highlight the effectiveness and superiority of the method in this embodiment, and prove its potential to make breakthrough progress in this field.

[0105] 3.2 Ablation Experiment

[0106] This example uses ablation experiments to explore the effectiveness of each component in our proposed method in the DG-RelD task, using ResNet-50 as the backbone network. Therefore, this example sets up four groups of experiments:

[0107] 1) Baseline model without any modules; 2) Only NAM module is added; 3) Both NAM and PAM modules are added; 4) All modules are added, including NAM, PAM and PRFE. The experimental results are shown in Table 3, where “x” indicates that no module is added and “√” indicates that a module is added.

[0108] Table 3 Ablation study of effectiveness of main components

[0109]

[0110] 3.2.1 Effectiveness of NAM

[0111] As can be seen from Table 4, integrating the NAM module into the second set of experiments leads to significant improvements in all three metrics compared to the first set of experiments. Specifically, the Rank-1 accuracy is significantly improved by 1.5% and the mAP accuracy under the D→M protocol is improved by 0.5%. These findings convincingly demonstrate that the IN layer in this embodiment, under the guidance of channel attention, effectively enhances the model's ability to adapt to style changes in unfamiliar test images from different domains.

[0112] 3.2.2 Effectiveness of PRFE

[0113] The PRFE module not only focuses on converting feature maps into 3D representations and enhancing the model’s ability to identify relationships between human body parts through 3D convolution, but also incorporates the outputs of Conv Block 3 and Conv Block 4 from the backbone network to facilitate learning global human knowledge. The effectiveness of this module can be demonstrated by comparing the experimental results of Group 2 and Group 3 in Table 3. It is worth noting that under the D→M protocol, all other indicators show significant improvements: Rank-1 accuracy is improved by 0.8% and mAP accuracy is improved by 1.5%.

[0114] 3.2.3 Effectiveness of PAM

[0115] The goal of the PAM module in this embodiment is to enhance the robustness of human appearance feature recognition by extracting discriminative fine-grained information and learning distinguishable detail features, thereby improving the visual feature representation ability in domain generalization scenarios. Under the MD protocol in Table 3, Group 3's performance on Rank-1 dropped by 2.1% and its performance on mAP dropped by 1.9% compared to Group 4. These findings provide strong validation for the effectiveness of the adopted method and support its application in a variety of fields.

[0116] 3.2.4 Effectiveness of Channel Attention (CA) in NAM

[0117] In addition, we conducted ablation experiments in NAM to enhance the channel attention of CA by combining MS→M and M→MS countermeasures. The results in Table 4 describe the Rank-1 accuracy data, where NAM (*) represents the replacement of improved attention with conventional SE-Net. The channel attention module of this embodiment adopts a strategy that combines global average pooling and global maximum pooling to effectively integrate the global information in the feature map. The overall distribution characteristics are captured by using global average pooling, and the global maximum pooling is used to identify salient features.

[0118] Table 4 Results of ablation experiments under three schemes

[0119]

[0120] 4. Overview

[0121] Based on instance normalization and channel attention mechanism, this paper designs a normalization mitigation module to extract domain-invariant appearance features with distinguishability while retaining the discriminative information of different people. Through this design, the loss of discriminative identity information caused by excessive use of the IN module is avoided, thereby reducing the style differences between different data domains while enhancing the model's ability to recognize individual features, thereby improving the accuracy and robustness of pedestrian re-identification. The present invention utilizes three-dimensional convolution operation to capture the spatial relationship features between different parts of pedestrians, and converts the two-dimensional feature map into a three-dimensional feature space through a PT3D projection operation, so that the three-dimensional convolution can process and extract features. In addition, multiple convolution branches are used to extract relationship features of different levels and different parts, which increases the diversity of features and the model's ability to express pedestrian appearance features. In this way, by extracting the spatial relationship features of parts, the camera-style-invariant pedestrian part relationship features are obtained to adapt to the camera style changes of unknown target domains. At the same time, the present invention aims at domain generalized pedestrian re-identification, and distinguishes pedestrians with similar appearances through a subtle discriminant model mining module, extracts subtle discriminant features with distinguishability and diversity, and further enhances the expression ability of image features in domain generalization scenarios.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described with reference to the preferred embodiments of the present invention, it should be understood by those skilled in the art that various changes may be made in form and details without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A method for domain generalization person re-identification by integrating camera invariant features, characterized in that: The steps include: Obtain a dataset of pedestrian images to be re-identified; A normalization mitigation module based on an instance normalization module and a channel attention mechanism is designed, ResNet-50 is used as the backbone network of a domain generalized person re-identification model, and the normalization mitigation module is inserted into the backbone network ResNet-50 to construct a domain style normalization module of the domain generalized person re-identification model; Design a part spatial relationship feature extraction module and a subtle discrimination pattern mining module, and build a domain generalization person re-identification model based on the domain style normalization module; The pedestrian image dataset to be re-identified is used as the input of the trained domain generalization pedestrian re-identification model to obtain the pedestrian re-identification result of the pedestrian image dataset to be re-identified.

2. The method for domain generalization person re-identification by integrating camera invariant features according to claim 1, characterized in that: The trained domain generalized pedestrian re-identification model uses the input pedestrian image dataset to be re-identified as the input of the domain style normalization module, extracts domain-invariant appearance features with distinguishability, and then extracts features of the domain-invariant appearance features through the part spatial relationship feature extraction module, so as to obtain stable spatial relationship features between pedestrian parts under different camera styles, and splices the output of the domain style normalization module with the output of the part spatial relationship feature extraction model to obtain fused features. Finally, the fused features are extracted through the subtle discriminant pattern mining module to obtain subtle discriminant features, thereby obtaining the pedestrian re-identification result as the output of the domain generalized pedestrian re-identification model.

3. The method for domain generalization person re-identification by integrating camera invariant features according to claim 1, characterized in that: The domain style normalization module includes a backbone network ResNet-50 and a first normalization relief module and a second normalization relief module respectively inserted into convolution block 3 and convolution block 4 in the backbone network ResNet-50; wherein the backbone network ResNet-50 includes five cascaded convolution blocks, and the first normalization relief module has the same structure as the second normalization relief module; Among them, the pedestrian image dataset to be re-identified as the input of the domain generalization pedestrian re-identification model is used as the input of convolution block 1, the output of each convolution block is used as the input of the next convolution block in the domain generalization pedestrian re-identification model, the output of convolution block 3 is used as the input of the first normalization relief module, the output of the first normalization relief module is used as the input of convolution block 4, the output of convolution block 4 is used as the input of the second normalization relief module, the output of the second normalization relief module is used as the input of convolution block 5, and the output of convolution block 5 is used as the overall output of the domain style normalization module.

4. The method for domain generalization person re-identification by integrating camera invariant features according to claim 3, characterized in that: The processing process of the normalization mitigation module is: f boost =λ*CA(X)⊙X; F=IN(X)⊙(1-λ*CA(X))+f boost ; Where IN(·) is instance normalization, is the Nth dimension of the input feature map X, n is an auxiliary value, γ and β are hyperparameters that can be learned during training, E(·) and Var(·) are the mean and variance of the feature map, respectively, and f boost is the enhanced feature, CA(·) is the channel attention, λ is the hyperparameter that controls the intensity of channel attention, ⊙ is the element-based multiplication, and F is the feature map obtained after the normalization mitigation module.

5. The method for domain generalization person re-identification by integrating camera invariant features according to claim 1, characterized in that: The part spatial relationship feature extraction module includes two 3D projection convolution units and one 2D projection convolution unit; the 3D projection convolution unit includes a cascaded PT3D module and a three-dimensional convolution block 4, and the 2D projection convolution unit includes a cascaded three-dimensional convolution block 5 and a PT2D module; Among them, the output of the first normalization relief module in the domain style normalization module is used as the input of the first 3D projection convolution unit, and the output of the second normalization relief module in the domain style normalization module is used as the input of the second 3D projection convolution unit. The output of the first normalization relief module and the output of the second normalization relief module are added and fused, and transmitted to the 2D projection convolution unit. The output of the 2D projection convolution unit is spliced ​​with the output of the convolution block 5 in the domain style normalization module to obtain the fused feature.

6. The method for domain generalization person re-identification by integrating camera invariant features according to claim 5, characterized in that: The processing process of the part spatial relationship feature extraction module includes the following steps: The input feature map is projected onto the 3D feature space using the PT3D unit, and a three-dimensional convolution block 4_1 is used to stack two 3D convolution kernels of size 3×3×3, thereby performing a convolution operation on the 3D feature map and extracting the part relationship features of feature maps of different channels; Then, the feature map output by the second normalization relief module in the domain style normalization module is used as an additional input of the part space relationship feature extraction module, and the additional part relationship features are extracted through the PT3D unit and the three-dimensional convolution block 4_2; then, the relationship features of the two branches are fused in an additive manner, and the three-dimensional convolution block 5 is used to extract the relationship features of the fused features again, and then, the obtained three-dimensional part relationship feature map is projected into the two-dimensional space through the PT2D unit to obtain a two-dimensional feature map with part relationship, and finally, the obtained two-dimensional feature map with part relationship is spliced ​​with the output feature map of the convolution block 5 in the domain style normalization module to obtain the fused feature.

7. The method for domain generalization person re-identification by integrating camera invariant features according to claim 1, characterized in that: The processing process of the subtle discrimination pattern mining module includes the following steps: The input fusion feature is reintegrated through a 1×1 convolutional layer, and the integrated feature map is normalized through a Sigmoid function to obtain a pattern feature map. At the same time, a separation loss function is constructed to ensure the diversity of the pattern feature map. The pattern feature map is optimized and updated with the goal of minimizing the separation loss function. Subsequently, the fusion feature map is divided into l patterns using the pattern feature map. Finally, the global average pooling operation is used to extract the features of each pattern, thereby obtaining subtle discriminant features. The processing process is expressed by the following formula: Where P k M k and The result after element-wise multiplication, M k is the pattern feature map, is the fusion feature, T is the transposition operation, p k is the feature of each pattern extracted by global average pooling, and f is the subtle discriminant feature.

8. The method for domain generalization person re-identification by integrating camera invariant features according to claim 7, characterized in that: The separation loss function is: Where, L sep is the separation loss function, (M T M) ij M T The element of the M matrix at the i-th row and j-th column.

9. The method for domain generalization person re-identification by integrating camera invariant features according to claim 1, characterized in that: The domain generalized person re-identification model is trained as follows: Pedestrian images with pre-labeled pedestrian features are used as training samples to form a training sample set, which is input into the domain generalized pedestrian re-identification model, and a central loss function is constructed to learn the relationship between different pedestrians. The model parameters of the domain generalized pedestrian re-identification model are optimized and updated with the goal of minimizing the central loss function, and then the domain generalized pedestrian re-identification model is trained.

10. The method for domain generalization person re-identification by integrating camera invariant features according to claim 9, characterized in that: The center loss function is: Where, L Center is the center loss function, N ID is the total number of different pedestrian identities in the current batch, is the feature mean of the pedestrians labeled qi in the current batch, P is the number of pedestrian identities in the current batch, ρ is the minimum interval between different centers, is the feature mean of the pedestrians with label qk in the current batch, is the feature mean of the pedestrians with label qj in the current batch.

Images