Style distillation cross-modal pedestrian re-identification method and system based on identity compensation

Abstract

The invention discloses a style distillation cross-modal pedestrian re-identification method and system based on identity compensation, and the method comprises the steps: obtaining a visible light image and an infrared image with a plurality of pedestrian identities, carrying out the image data preprocessing, and obtaining a preprocessed visible light image and a preprocessed infrared image; based on a double-flow residual network ResNet-50, introducing a style knowledge distillation module and an identity difference amplification module, and constructing an identity compensation style distillation network model; and performing cross-modal style distillation pedestrian re-identification on the preprocessed visible light image and the preprocessed infrared image based on an identity compensation style distillation network model to obtain a cross-modal pedestrian re-identification result. According to the method, style consistency can be forcibly kept among different modes, identity-related semantic information is reserved, and the pedestrian re-identification precision is improved. The style distillation cross-modal pedestrian re-identification method and system based on identity compensation can be widely applied to the technical field of pedestrian re-identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pedestrian re-identification, in particular to a style distillation cross-modal pedestrian re-identification method and system based on identity compensation. BACKGROUND

[0002] Visible-infrared person re-identification aims to match pedestrian images in multiple non-overlapping camera views and different modes. Today's advanced monitoring systems can automatically switch from visible light mode to infrared mode when night falls, thus ensuring sufficient trainable data. However, due to the different wavelength ranges of visible light and infrared images, there is a big difference between the two, and there is a big cross-modal difference in visible-infrared person re-identification. Compared with traditional single-modal person re-identification, the difference between the same person in visible-infrared person re-identification is much greater. This makes the efficiency of the previous single-modal method based on visible light images greatly reduced in the visible-infrared person re-identification task.

[0003] The visible-infrared person re-identification method in the related art usually aims to project heterogeneous modal images into a shared feature space and align at the feature level, so as to learn a modal-invariant representation. Although this kind of method has achieved effective results to some extent, there are still significant challenges when facing intra-modal changes and inter-modal differences, which seriously affect the robustness of cross-modal matching in actual scenes. In order to alleviate the modal difference, some researches introduce a modal style alignment mechanism, aiming to reduce the gap between the modes. Although these methods help to narrow the modal difference, there are often two key problems: first, although style alignment helps to unify the modes, it may inadvertently suppress the identity discrimination clues, thus weakening the discriminability of the learned features; second, identity-independent change factors may mask the fine-grained identity features used to distinguish similar pedestrians. These disturbances limit the ability of the model to capture subtle identity differences, resulting in identity distribution collapse and reducing retrieval accuracy. SUMMARY

[0004] In order to solve the above technical problems, the purpose of the present application is to provide a style distillation cross-modal pedestrian re-identification method and system based on identity compensation, which can force style consistency between different modes and retain identity-related semantic information, thereby improving the accuracy of pedestrian re-identification.

[0005] The first technical solution adopted by the present application is: a style distillation cross-modal pedestrian re-identification method based on identity compensation, comprising the following steps:

[0006] Obtain visible light images and infrared images with a plurality of pedestrian identities and perform image data preprocessing to obtain preprocessed visible light images and preprocessed infrared images;

[0007] Based on the dual-stream residual network ResNet-50, a style knowledge distillation module and an identity difference amplification module are introduced to construct an identity compensation style distillation network model.

[0008] Based on the identity compensation style distillation network model, cross-modal style distillation pedestrian re-identification is performed on the preprocessed visible light image and the preprocessed infrared image to obtain a cross-modal pedestrian re-identification result.

[0009] Further, the identity compensation style distillation network model is used to perform cross-modal style distillation pedestrian re-identification on the preprocessed visible light image and the preprocessed infrared image to obtain a cross-modal pedestrian re-identification result, which specifically includes:

[0010] The preprocessed visible light image and the preprocessed infrared image are input into the identity compensation style distillation network model.

[0011] The feature extractor based on the identity compensation style distillation network model is used to perform low-level feature extraction processing on the preprocessed visible light image and the preprocessed infrared image to obtain visible light modality specific features and infrared modality specific features.

[0012] The dual-stream residual network ResNet-50 based on the identity compensation style distillation network model is used to perform high-level semantic feature extraction on the visible light modality specific features and the infrared modality specific features to obtain visible light modality shared features and infrared modality shared features.

[0013] The style knowledge distillation module based on the identity compensation style distillation network model is used to perform modality style compensation processing on the visible light modality shared features and the infrared modality shared features to obtain compensated visible light modality shared features and compensated infrared modality shared features.

[0014] The identity difference amplification module based on the identity compensation style distillation network model is used to perform different identity feature amplification processing on the compensated visible light modality shared features and the compensated infrared modality shared features to obtain amplified visible light modality shared features and amplified infrared modality shared features.

[0015] The classifier based on the identity compensation style distillation network model is used to perform classification and recognition on the amplified visible light modality shared features and the amplified infrared modality shared features to obtain a cross-modal pedestrian re-identification result.

[0016] Further, the dual-stream residual network ResNet-50 based on the identity compensation style distillation network model is used to perform high-level semantic feature extraction on the visible light modality specific features and the infrared modality specific features to obtain visible light modality shared features and infrared modality shared features, which specifically includes:

[0017] input the visible light modality specific features and the infrared modality specific features into a double-flow residual network ResNet-50 of the identity compensation style distillation network model;

[0018] Based on the first layer residual block of the double-flow residual network ResNet-50, the visible light modality specific features and the infrared modality specific features are subjected to modality sharing feature extraction processing to obtain first visible light modality sharing features and first infrared modality sharing features;

[0019] Based on the second layer residual block of the double-flow residual network ResNet-50, the first visible light modality sharing features and the first infrared modality sharing features are subjected to modality sharing feature extraction processing to obtain second visible light modality sharing features and second infrared modality sharing features;

[0020] Based on the third layer residual block of the double-flow residual network ResNet-50, the second visible light modality sharing features and the second infrared modality sharing features are subjected to modality sharing feature extraction processing to obtain third visible light modality sharing features and third infrared modality sharing features;

[0021] Based on the fourth layer residual block of the double-flow residual network ResNet-50, the third visible light modality sharing features and the third infrared modality sharing features are subjected to modality sharing feature extraction processing to obtain fourth visible light modality sharing features and fourth infrared modality sharing features;

[0022] Based on the global average pooling layer and the batch normalization layer of the double-flow residual network ResNet-50, the fourth visible light modality sharing features and the fourth infrared modality sharing features are subjected to feature dimension adjustment processing to obtain visible light modality sharing features and infrared modality sharing features.

[0023] Further, the style knowledge distillation module based on the identity compensation style distillation network model subjects the visible light modality sharing features and the infrared modality sharing features to modality style compensation processing to obtain compensated visible light modality sharing features and compensated infrared modality sharing features, and the step specifically includes:

[0024] input the visible light modality sharing features and the infrared modality sharing features into the style knowledge distillation module of the identity compensation style distillation network model;

[0025] Based on the style difference reduction module of the style knowledge distillation module, the visible light modality sharing features and the infrared modality sharing features are subjected to style difference reduction processing to obtain style-consistent visible light modality sharing features and style-consistent infrared modality sharing features;

[0026] The identity knowledge compensation module based on the style knowledge distillation module compensates the style-consistent visible light modality shared features and the style-consistent infrared modality shared features to obtain compensated visible light modality shared features and compensated infrared modality shared features.

[0027] Further, the style difference reduction module based on the style knowledge distillation module specifically comprises the following steps:

[0028] The style difference reduction module based on the style knowledge distillation module calculates the Euclidean distance between the class center of the visible light modality shared features and the infrared modality shared features sample, and defines the visible light modality shared features and the infrared modality shared features corresponding to the maximum Euclidean distance as the visible-infrared feature pair with the maximum modality difference.

[0029] The mean and the standard deviation of the visible-infrared feature pair with the maximum modality difference are obtained, and the style attribute of the visible-infrared feature pair with the maximum modality difference is determined.

[0030] The style difference reduction loss function is constructed based on the style attribute of the visible-infrared feature pair with the maximum modality difference, and the minimization processing is performed to obtain the style-consistent visible light modality shared features and the style-consistent infrared modality shared features.

[0031] Further, the identity knowledge compensation module based on the style knowledge distillation module specifically comprises the following steps:

[0032] The identity knowledge compensation module based on the style knowledge distillation module performs style filtering on the style-consistent visible light modality shared features and the style-consistent infrared modality shared features through instance normalization to obtain visible light normalized features and infrared normalized features.

[0033] Difference calculation is performed on the style-consistent visible light modality shared features and the visible light normalized features, and difference calculation is performed on the style-consistent infrared modality shared features and the infrared normalized features to obtain visible light residual features and infrared residual features.

[0034] Identity perception information extraction processing is performed on the visible light residual features and the infrared residual features through the attention channel mask mechanism to obtain visible light identity features and infrared identity features.

[0035] The visible light identity feature is fused with the visible light normalized feature, and the infrared identity feature is fused with the infrared normalized feature to obtain a visible light identity enhanced feature and an infrared identity enhanced feature;

[0036] The intra-modal identity knowledge compensation loss function and the inter-modal identity knowledge compensation loss function are introduced to perform feature alignment and identity information complementation on the visible light identity enhanced feature and the infrared identity enhanced feature, to obtain a compensated visible light modal shared feature and a compensated infrared modal shared feature.

[0037] Further, the loss function of the style knowledge distillation module includes a style difference reduction loss function and an identity knowledge compensation loss function, wherein:

[0038] The expression of the style difference reduction loss function is specifically as follows:

[0039]

[0040] In the above formula, Lsd represents the style difference reduction loss function, L represents the total number of stages, and l represents the lth stage, μvl represents the mean of the visible light feature of the lth stage, μir represents the mean of the infrared feature of the lth stage, σvl represents the standard deviation of the visible light feature of the lth stage, σir represents the standard deviation of the infrared feature of the lth stage.

[0041] The expression of the identity knowledge compensation loss function is specifically as follows:

[0042]

[0043] In the above formula, Lid represents the identity knowledge compensation loss function, μid represents the intra-modal identity knowledge compensation loss function, μid represents the inter-modal identity knowledge compensation loss function, L represents the total number of stages, and l represents the lth stage, μid represents the identity feature of the visible light feature of the lth stage, μid represents the identity feature of the visible light modal center of the lth stage, μid represents the identity feature of the infrared modal center of the lth stage, μid represents the identity feature of the infrared modal center of the lth stage.

[0044] Further, the identity difference amplification module based on the identity compensation style distillation network model performs different identity feature amplification processing on the compensated visible light modal shared features and the compensated infrared modal shared features, to obtain amplified visible light modal shared features and amplified infrared modal shared features, and the step specifically comprises:

[0045] The identity difference amplification module based on the identity compensation style distillation network model acquires the modal shared center of the compensated visible light modal shared features and the compensated infrared modal shared features;

[0046] The maximum Euclidean distance between the modal shared center and the compensated visible light modal shared features and the compensated infrared modal shared features is defined;

[0047] According to the maximum Euclidean distance, the identity category discrimination region of the compensated visible light modal shared features and the compensated infrared modal shared features is determined, and difficult negative samples are mined;

[0048] The difficult negative samples and the modal shared center are subjected to instance normalization, and the difficult negative samples are pushed away from the current modal shared center, to obtain the amplified visible light modal shared features and the amplified infrared modal shared features.

[0049] Further, the expression of the loss function of the identity difference amplification module is specifically as follows:

[0050]

[0051] In the above formula, L represents the loss function of the identity difference amplification module, L represents the total number of stages, and l represents the lth stage, represents the instance-normalized modal shared center of the lth stage, represents the instance-normalized negative sample of the lth stage.

[0052] The second technical solution adopted by the present application is: a style distillation cross-modal pedestrian re-identification system based on identity compensation, comprising:

[0053] The first module is used for acquiring visible light images and infrared images with a plurality of pedestrian identities and performing image data preprocessing, to obtain preprocessed visible light images and preprocessed infrared images;

[0054] The second module is used for introducing a style knowledge distillation module and an identity difference amplification module based on a double-flow residual network ResNet-50, to construct an identity compensation style distillation network model;

[0055] The third module is configured to perform cross-modal style distillation pedestrian re-identification on the preprocessed visible light image and the preprocessed infrared image based on the identity compensation style distillation network model to obtain a cross-modal pedestrian re-identification result.

[0056] The method and system have the following advantages: the visible light image and the infrared image with a plurality of pedestrian identities are obtained and preprocessed to obtain the preprocessed visible light image and the preprocessed infrared image, the identity compensation style distillation network model is constructed based on the double-stream residual network ResNet-50, the style knowledge distillation module and the identity difference amplification module are introduced, the style knowledge distillation module is designed, the identity related information is retained while the style consistency is achieved through knowledge compensation, the identity difference amplification module is introduced to enhance the ability to capture fine-grained identity differences, thereby improving the overall discriminability of the modal invariant feature, and finally the cross-modal style distillation pedestrian re-identification is performed on the preprocessed visible light image and the preprocessed infrared image based on the identity compensation style distillation network model to obtain a cross-modal pedestrian re-identification result, which can force the style consistency between different modalities, retain the identity related semantic information, and enhance the perception ability of the model to fine-grained identity differences, thereby improving the overall discriminability of the modal invariant feature. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a step flow chart of the identity compensation based style distillation cross-modal pedestrian re-identification method of the present application;

[0058] Figure 2 is a structural block diagram of the identity compensation based style distillation cross-modal pedestrian re-identification system of the present application;

[0059] Figure 3 is a schematic diagram of the identity compensation style distillation network model provided by the embodiment of the present application;

[0060] Figure 4 is a schematic diagram of the style knowledge distillation module provided by the embodiment of the present application;

[0061] Figure 5 is a schematic diagram of the identity difference amplification module provided by the embodiment of the present application. DETAILED DESCRIPTION

[0062] The present application will be further described in detail below in combination with the drawings and specific embodiments. For the step numbers in the following embodiments, only the order of the steps is set for the purpose of facilitating the description, and the order of the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0063] Due to the main challenge of visible-infrared person re-identification is that the modal difference between visible light image and infrared image is large, it is urgent to find an intermediate modal data that can balance the two, or a method that can extract modal invariant features. Recently, many advanced methods have appeared in the field of visible-infrared person re-identification. Among them, the knowledge distillation method is a technology to improve the performance of the model while keeping the network unchanged. The student is supervised by the hard label and soft label output by the teacher. According to the type of knowledge transfer, the current research is mainly divided into: probability-based method, feature-based method and relationship-based method. While self-knowledge distillation is more like a training technique to improve model performance, rather than a method to compress or accelerate the model. Unlike traditional knowledge distillation, self-knowledge distillation uses the model's own intermediate features to improve itself. This no longer requires the introduction of a complex teacher model, but focuses on optimizing the existing network architecture to achieve better learning results. At present, self-knowledge distillation is mainly divided into data augmentation-based methods and auxiliary network-based methods. Data augmentation-based methods encourage consistent predictions for various distorted versions of single instances or instance pairs from the same class. Auxiliary network-based methods add additional branches to the classifier network, and through knowledge transfer, these branches are prompted to produce similar outputs. However, the complexity of these auxiliary networks is usually the same as or lower than the main classifier, so it is difficult to generate fine-grained knowledge. There are also a lot of work that applies the idea of knowledge distillation to the visible-infrared person re-identification task for interactive learning of different modalities. For example, interactive learning between the two modalities enhances invariant feature learning, uses multiple classifiers to reduce logits-level modal differences, and through four classifiers, the RGB and IR knowledge is distilled to each other to achieve shared representations. However, they only learn semantic-level knowledge.

[0064] Based on this, the embodiment of the present application performs feature-level knowledge distillation on the modal features of each stage, directly uses modal self-features for knowledge transfer to reduce cross-modal differences, and focuses on learning discriminative knowledge to aggregate more identity discriminative clues to improve model retrieval performance, without the need to introduce additional enhanced images or auxiliary branches.

[0065] Reference Figure 1 The present application provides a style distillation cross-modal pedestrian re-identification method based on identity compensation, which comprises the following steps:

[0066] S100, obtain visible light images and infrared images with several pedestrian identities and perform image data preprocessing to obtain preprocessed visible light images and preprocessed infrared images;

[0067] In this embodiment, in each small batch, we randomly select 4 visible light images and 4 infrared images of 6 identities for training. We first adjust the size of all input images to 3x384x144, and use random horizontal flip, random grayscale and random erasing techniques for data augmentation during the training phase. We set the initial learning rate to 0.01, and then increase it to 0.1 after 10 epochs by the warm-up strategy; then, the learning rate is decayed to 0.01 at 20 epochs; at 60 and 120 epochs, it is further decayed to 0.001 and 0.0001, respectively, until a total of 150 epochs. The SGD optimizer is used for training, with the momentum term set to 0.9.

[0068] S200, based on the dual-stream residual network ResNet-50, a style knowledge distillation module and an identity difference amplification module are introduced to construct an identity compensation style distillation network model;

[0069] In this embodiment, as shown in Figure 3 , the implementation details of the knowledge compensation style distillation network we proposed are shown, and the backbone network uses the dual-stream residual network ResNet-50. In the training process, we use cross-entropy loss and triplet loss as the basic loss, denoted as In order to facilitate the end-to-end optimization of the proposed identity compensation style distillation network, the overall loss function combines the basic loss and the style difference reduction loss identity knowledge compensation loss and identity difference amplification loss The total target is defined as:

[0070]

[0071] In the above formula, represents the loss function of the identity compensation style distillation network model, represents the basic loss function, represents the style difference reduction loss function, represents the identity knowledge compensation loss function, represents the identity difference amplification loss function, and λ1, λ2 represent the hyperparameters for balancing the contribution of the corresponding auxiliary loss.

[0072] S300, based on the identity compensation style distillation network model, cross-modal style distillation pedestrian re-identification is performed on the preprocessed visible light image and the preprocessed infrared image to obtain a cross-modal pedestrian re-identification result.

[0073] S310, input the preprocessed visible light image and the preprocessed infrared image to the identity compensation style distillation network model;

[0074] S320, based on the identity compensation style distillation network model feature extractor, the preprocessed visible light image and the preprocessed infrared image are subjected to low-level feature extraction processing, and visible light modal specific features and infrared modal specific features are obtained;

[0075] In this embodiment, after the data preprocessing is completed, the 4 visible light images and the 4 infrared images of the 6 identities are respectively input into two weight-unshared feature extractors Conv1 and Conv2 to extract visible light modal specific features and infrared modal specific features, respectively. The batch size (B), the number of channels (C), the height (H) and the width (W) of the extracted features are [48, 64, 96, 36] in turn.

[0076] S330, based on the identity compensation style distillation network model double-flow residual network ResNet-50, the visible light modal specific features and the infrared modal specific features are subjected to high-level semantic feature extraction, and visible light modal shared features and infrared modal shared features are obtained.

[0077] Specifically, the visible light modal specific features and the infrared modal specific features are input into the double-flow residual network ResNet-50 of the identity compensation style distillation network model; based on the first layer residual block of the double-flow residual network ResNet-50, the visible light modal specific features and the infrared modal specific features are subjected to modal shared feature extraction processing to obtain first visible light modal shared features and first infrared modal shared features; based on the second layer residual block of the double-flow residual network ResNet-50, the first visible light modal shared features and the first infrared modal shared features are subjected to modal shared feature extraction processing to obtain second visible light modal shared features and second infrared modal shared features; based on the third layer residual block of the double-flow residual network ResNet-50, the second visible light modal shared features and the second infrared modal shared features are subjected to modal shared feature extraction processing to obtain third visible light modal shared features and third infrared modal shared features; based on the fourth layer residual block of the double-flow residual network ResNet-50, the third visible light modal shared features and the third infrared modal shared features are subjected to modal shared feature extraction processing to obtain fourth visible light modal shared features and fourth infrared modal shared features; based on the global average pooling layer and the batch normalization layer of the double-flow residual network ResNet-50, the fourth visible light modal shared features and the fourth infrared modal shared features are subjected to feature dimension adjustment processing to obtain visible light modal shared features and infrared modal shared features.

[0078] In the embodiment, the extracted modal specific features are simultaneously input into a double-stream residual network ResNet-50 sharing weights, and first-stage modal shared features are extracted through a first residual block, and the feature dimension is adjusted to [48, 256, 96, 36]; second-stage modal shared features are extracted through a second residual block, and the feature dimension is adjusted to [48, 512, 48, 18]; third-stage modal shared features are extracted through a third residual block, and the feature dimension is adjusted to [48, 1024, 24, 9]; fourth-stage modal shared features are extracted through a fourth residual block, and the feature dimension is adjusted to [48, 2048, 24, 9]; finally, the modal shared features sequentially pass through a global average pooling layer (GAP) and a batch normalization layer (BN), and the feature dimensions are sequentially adjusted to [48, 2048, 1, 1] and [48, 2048]; in the training stage, all modal shared features passing through the global average pooling layer (GAP) are input into the triplet loss, and all modal shared features after the batch normalization layer (BN) are also input into the classifier to obtain the classification probability, which is input into the classification loss, and the identity compensation style distillation network is optimized by minimizing different loss functions.

[0079] In S340, the style knowledge distillation module based on the identity compensation style distillation network model is used to perform modal style compensation processing on the visible light modal shared features and the infrared modal shared features to obtain compensated visible light modal shared features and compensated infrared modal shared features.

[0080] Specifically, the visible light modal shared features and the infrared modal shared features are input into the style knowledge distillation module of the identity compensation style distillation network model; the style difference reduction module based on the style knowledge distillation module is used to perform style difference reduction processing on the visible light modal shared features and the infrared modal shared features to obtain style-consistent visible light modal shared features and style-consistent infrared modal shared features; the identity knowledge compensation module based on the style knowledge distillation module is used to perform identity knowledge compensation processing on the style-consistent visible light modal shared features and the style-consistent infrared modal shared features to obtain compensated visible light modal shared features and compensated infrared modal shared features.

[0081] In this embodiment, for visible light and infrared image inputs, the first few layers of convolutional layers are independently set to extract low-level features specific to the modalities; subsequently, the residual blocks from stage one to stage four are shared by two branches to generate high-level semantic representations shared by the modalities. The knowledge compensation style distillation framework consists of two core modules: style knowledge distillation and identity difference amplification. These two modules work together to reduce the difference between modalities, preserve identity-related semantic information, and enhance the model's ability to perceive fine-grained class differences. Among them, the style knowledge distillation module minimizes the statistical difference (mean and standard deviation) between visible light and infrared feature pairs to achieve modal style consistency; at the same time, to avoid the loss of identity semantics during style alignment, the style knowledge distillation also introduces a knowledge injection mechanism for compensation.

[0082] Due to the fundamental difference in imaging mechanism, visible and thermal infrared images have significant differences in appearance style, resulting in large modal style differences. To alleviate this problem, we propose style knowledge distillation, which aims to align the style across modalities while preserving identity-related semantics. Style knowledge distillation contains two complementary components: style difference reduction and identity knowledge compensation. The style difference reduction module performs style distillation between visible light and infrared features to alleviate the modality-induced gap, while the identity knowledge compensation module compensates for the degradation of identity-related features caused by alignment by injecting identity-related information.

[0083] Specifically, this module first excavates the feature pair with the largest inter-modal difference and its corresponding modal center feature, and normalizes these features by instance; then, channel attention mechanism is applied to the residual information between the initial features and the normalized features to extract the remaining identity clues; finally, by fusing the normalized features and the extracted identity information, the complete identity representation is restored, thus achieving effective knowledge compensation. The structure of the style knowledge distillation module is shown in Figure 4

[0084] Among them, for the style difference reduction module of the style knowledge distillation module, the Euclidean distance between the class center of the visible light modal shared feature and the infrared modal shared feature sample is calculated, and the visible light modal shared feature and the infrared modal shared feature corresponding to the maximum Euclidean distance are defined as the visible-infrared feature pair with the largest modal difference; the mean and standard deviation values of the visible-infrared feature pair with the largest modal difference are obtained to determine the style attribute of the visible-infrared feature pair with the largest modal difference; based on the style attribute of the visible-infrared feature pair with the largest modal difference, a style difference reduction loss function is constructed and minimized to obtain the style-consistent visible light modal shared feature and the style-consistent infrared modal shared feature.

[0085] ​In this embodiment, to mine the most style-discriminative feature pairs, we compute the Euclidean distance between the class center of one modality and each feature sample of the corresponding identity from the other modality. The feature pair with the largest distance is defined as the most style-discriminative visible-infrared feature pair:

[0086]

[0087] where D(·) denotes the Euclidean distance, represents the modality center feature of class p, N p is the number of samples of class p. j and k represent the indices of identity samples from the visible and infrared modalities, respectively.

[0088] Following the above strategy, we obtain all the most style-discriminative visible-infrared feature pairs within a mini-batch, defined as:

[0089]

[0090] where P is the number of identities in the mini-batch.

[0091] The statistical properties of a feature usually reflect its feature style. Here, we employ the mean and standard deviation to represent the style properties of a feature f m ∈R C×H×W :

[0092]

[0093] To ensure the style consistency between the most style-discriminative visible-infrared pairs, we minimize their statistical property differences at all stages of the network. Given the mean and standard deviation of the feature of the l-th stage of the backbone network, the overall style discrepancy reduction loss is represented as:

[0094]

[0095] where L is the total number of stages. The style discrepancy reduction loss achieves consistent style representation between the visible and infrared modalities by encouraging the network to focus on the most difficult cross-modality sample pairs.

[0096] Further, for the identity knowledge compensation module based on the style knowledge distillation module, the style consistent visible light modality shared features and the style consistent infrared modality shared features are filtered by instance normalization to obtain visible light normalized features and infrared normalized features; the style consistent visible light modality shared features and the visible light normalized features are calculated by difference, and the style consistent infrared modality shared features and the infrared normalized features are calculated by difference to obtain visible light residual features and infrared residual features; the visible light residual features and the infrared residual features are subjected to identity perception information extraction processing by an attention channel mask mechanism to obtain visible light identity features and infrared identity features; the visible light identity features and the visible light normalized features are fused, and the infrared identity features and the infrared normalized features are fused to obtain visible light identity enhanced features and infrared identity enhanced features; the visible light identity enhanced features and the infrared identity enhanced features are subjected to feature alignment and identity information complementation by introducing an intra-modality identity knowledge compensation loss function and an inter-modality identity knowledge compensation loss function to obtain compensated visible light modality shared features and compensated infrared modality shared features.

[0097] In this embodiment, although the style difference reduction module effectively reduces the cross-modality gap, it may inadvertently cause the loss of identity discriminative information. To solve this problem, we propose an identity knowledge compensation module that restores the degraded identity semantics by injecting complementary identity cues from the modality center into the corresponding feature representation. Specifically, to separate the identity-related features from the style-related information, we use instance normalization to filter out the style components from f m and c m , thereby generating normalized features and

[0098]

[0099] where ∈ is a small constant to avoid numerical instability, and E[·], Var[·] represent the mean and variance along each channel, respectively. The residual between the original features and the normalized features can be represented as:

[0100]

[0101] This residual feature mainly captures the style attributes, but due to incomplete decoupling, it still contains valuable identity cues. To extract these cues, we use an attention channel mask and following the CBAM mechanism:

[0102]

[0103] where denotes element-wise multiplication, and denote the identity-aware information extracted from the residual feature and respectively.

[0104] By fusing the normalized feature with these extracted components, we construct the identity-enhanced feature:

[0105]

[0106] To enhance the degraded identity semantics in the feature, we design an intra-modal identity knowledge compensation loss to align the identity-related information between f m and c m :

[0107]

[0108] Therefore, the richer identity semantics contained in f are injected into f , achieving effective identity compensation.

[0109] In addition, to promote the complementary identity information between modal centers, we introduce an inter-modal identity knowledge compensation loss:

[0110]

[0111] This achieves the knowledge interaction and sharing between the visible light center and the infrared center .

[0112] By combining the intra-modal and inter-modal identity knowledge compensation losses, the total identity knowledge compensation loss is denoted as:

[0113]

[0114] In this way, the style difference reduction module effectively eliminates the cross-modal style difference, while the identity knowledge compensation module compensates for the degraded identity discriminative semantics in the style alignment process. Together, they enable style knowledge distillation to achieve cross-modal consistent style alignment and strengthen the identity-related representation of the modal-invariant feature.

[0115] S350, based on the identity compensation style distillation network model, the identity difference amplification module is used to perform different identity feature amplification processing on the compensated visible light modal shared feature and the compensated infrared modal shared feature, to obtain amplified visible light modal shared feature and amplified infrared modal shared feature;

[0116] Specifically, the identity difference amplification module based on the identity compensation style distillation network model obtains a modal sharing center of the compensated visible light modal shared feature and the compensated infrared modal shared feature; a maximum Euclidean distance between the modal sharing center and the compensated visible light modal shared feature and the compensated infrared modal shared feature is defined; identity category discrimination regions of the compensated visible light modal shared feature and the compensated infrared modal shared feature are determined according to the maximum Euclidean distance, and difficult negative samples are mined; instance normalization is performed on the difficult negative samples and the modal sharing center, and the difficult negative samples are pushed away from the current modal sharing center, to obtain amplified visible light modal shared features and amplified infrared modal shared features.

[0117] In the embodiment, the identity difference amplification module further optimizes the learned features by amplifying the subtle differences between different identities, thereby improving the discrimination ability of the model in the shared feature space.

[0118] In cross-modal pedestrian re-identification, a large number of intra-class differences and limited inter-class differences bring great challenges to identity recognition. Relying only on cross-modal alignment often cannot capture subtle inter-class differences, especially between visually similar individuals, thereby causing potential identity confusion. To solve this limitation, we propose an identity difference amplification module that explicitly enhances the model's ability to distinguish subtle identity differences. The module constructs a discrimination region around each identity center and actively mines difficult-to-distinguish negative samples from different identities but close to the center. By amplifying the differences between these difficult-to-distinguish negative examples and their corresponding identity centers, it encourages the network to suppress identity-irrelevant noise while emphasizing identity-specific features. The structure of the identity difference amplification module is shown in Figure 5

[0119] First, the module constructs a different discrimination region for each identity, and each region is expected to contain only features belonging to its corresponding class. To enhance inter-class separability, the overlap between these regions is minimized, thereby improving the uniqueness of the identity discriminative representation. For a given identity class p, we calculate its modal sharing center by averaging all features from the visible light and infrared modal which serves as a robust representation of identity class p:

[0120]

[0121] where N p is the number of samples of class p. Given the shared identity center the discrimination region radius R p of identity p is defined as the maximum Euclidean distance between the center and all intra-class features from the two modalities:

[0122] ​

[0123] To enhance the separability between classes, we mine hard negative samples within the discrimination region of each identity p These hard negative samples are defined as:

[0124]

[0125] Where q represents any identity different from p. These samples are located within the discrimination region of identity p but belong to different classes. Therefore, they are crucial as hard negative samples for learning the subtle differences between classes and enhancing the discriminative ability of the model.

[0126] To suppress the style-related influence and preserve the identity-related information, we normalize the identity center shared by the modalities And the mined hard negative samples By instance normalization:

[0127]

[0128] Based on the above modal normalization features, we propose an identity difference amplification loss to push the hard negative samples away from the current identity center, and the specific formula is:

[0129]

[0130] Where L represents the total number of stages in the backbone network. This loss encourages the model to better capture the fine-grained identity differences and improve the separability between classes, thereby enhancing the overall identity discriminative ability.

[0131] S360, the classifier based on the identity compensation style distillation network model, classifies and identifies the amplified visible light modality shared features and the amplified infrared modality shared features, and obtains the cross-modality pedestrian re-identification result.

[0132] The identity compensation style distillation network proposed by the embodiment of the application performs feature-level knowledge distillation on the modal features of each stage, forces the style consistency between different modalities, preserves the identity-related semantic information, and enhances the perception ability of the model to fine-grained identity differences, thereby improving the overall discriminative ability of the modal invariant features.

[0133] On the one hand, the style knowledge distillation module takes into account the dual goals of style alignment and identity preservation. It performs cross-modal style distillation by mining visible-infrared feature pairs with the largest style difference and minimizing their differences in mean and standard deviation statistics; at the same time, it compensates for the possible weakening of identity semantics by injecting identity-related knowledge contained in the modality center into the aligned features, thereby enhancing semantic consistency and preserving the integrity of discriminative features. On the other hand, the identity difference amplification module aims to improve the model's ability to model fine-grained identity differences. The module defines a discriminative region around a given identity anchor point and locates difficult negative samples in this region. By amplifying the small differences between these negative samples and the identity center, it encourages the model to further refine identity-related clues while suppressing the interference of identity-independent information. Experiments show that the identity-compensated style distillation network outperforms the state-of-the-art methods on three mainstream benchmark datasets.

[0134] In summary, the embodiments of the present application preserve identity semantic information and enhance the model's ability to capture subtle identity differences on the basis of consistent cross-modal style, thereby effectively improving the overall discriminability of modality-invariant features. The knowledge-compensated style distillation network we propose contains two core components: on the one hand, we introduce the style knowledge distillation module, which has both style alignment and identity information preservation purposes. It includes two complementary parts: style difference reduction and identity knowledge compensation. Style difference reduction minimizes the mean and standard deviation statistical differences of the largest style difference visible-infrared feature pairs to achieve cross-modal style distillation; while identity knowledge compensation effectively compensates for the loss of semantic information caused by style difference reduction by injecting identity-related knowledge from the modality center, thereby enhancing semantic consistency and preserving the integrity of discriminative features. On the other hand, we propose the identity difference amplification module to improve the model's ability to model fine-grained identity differences. Specifically, the identity difference amplification module defines a discriminative region around a certain identity center and locates difficult negative samples within the region. By amplifying the subtle differences between these negative samples and the identity center, the module encourages the model to further refine identity features while suppressing the interference of identity-independent information.

[0135] Therefore, the embodiments of the present application have the following advantages compared to the prior art:

[0136] 1) An identity-compensated style distillation network is proposed, which preserves identity semantics and amplifies inter-class subtle differences during style alignment, thereby enhancing the discriminability of modality-invariant features.

[0137] 2) A style knowledge distillation module is designed to achieve style consistency while preserving identity-related information through knowledge compensation.

[0138] 3) Introduce identity difference amplification module to enhance the ability to capture fine-grained identity nuances, thereby improving the overall discriminability of modal invariant features.

[0139] Referring to Figure 2 The identity compensation-based style distillation cross-modal pedestrian re-identification system comprises:

[0140] A first module 201 is configured to acquire visible light images and infrared images with a plurality of pedestrian identities and perform image data preprocessing to obtain preprocessed visible light images and preprocessed infrared images.

[0141] A second module 202 is configured to introduce a style knowledge distillation module and an identity difference amplification module based on a dual-stream residual network ResNet-50 to construct an identity compensation style distillation network model.

[0142] A third module 203 is configured to perform cross-modal style distillation pedestrian re-identification on the preprocessed visible light images and the preprocessed infrared images based on the identity compensation style distillation network model to obtain a cross-modal pedestrian re-identification result.

[0143] The content in the method embodiments is applicable to the system embodiments, the system embodiments specifically implement the same functions as the method embodiments, and achieve the same beneficial effects as the method embodiments.

[0144] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments described above. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application. These equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A style distillation-based cross-modal person re-identification method with identity compensation, characterized in that, Includes the following steps: Visible light and infrared images with several pedestrian identities are acquired and image data preprocessed to obtain preprocessed visible light and infrared images. Based on the two-stream residual network ResNet-50, a style knowledge distillation module and an identity difference amplification module are introduced to construct an identity-compensated style distillation network model; Based on the identity-compensated style distillation network model, cross-modal style distillation pedestrian re-identification is performed on preprocessed visible light images and preprocessed infrared images to obtain cross-modal pedestrian re-identification results.

2. The style distillation cross-modal person re-identification method based on identity compensation according to claim 1, characterized in that, The step of performing cross-modal style distillation pedestrian re-identification on the preprocessed visible light image and the preprocessed infrared image based on the identity-compensated style distillation network model to obtain the cross-modal pedestrian re-identification result specifically includes: The preprocessed visible light image and the preprocessed infrared image are input into the identity compensation style distillation network model; A feature extractor based on an identity-compensated style distillation network model performs low-level feature extraction on preprocessed visible light and infrared images to obtain visible light mode-specific features and infrared mode-specific features. The ResNet-50 dual-stream residual network based on the identity-compensated style distillation network model performs high-level semantic feature extraction on visible light modal-specific features and infrared modal-specific features to obtain visible light modal shared features and infrared modal shared features. The style knowledge distillation module based on the identity-compensated style distillation network model performs modal style compensation processing on visible light modal shared features and infrared modal shared features to obtain compensated visible light modal shared features and compensated infrared modal shared features. Based on the identity difference amplification module of the identity compensation style distillation network model, the compensated visible light modal sharing features and the compensated infrared modal sharing features are amplified by different identity features to obtain amplified visible light modal sharing features and amplified infrared modal sharing features. A classifier based on an identity-compensated style distillation network model is used to classify and identify the magnified visible light modal shared features and the magnified infrared modal shared features, thereby obtaining cross-modal pedestrian re-identification results.

3. The style distillation cross-modal person re-identification method based on identity compensation according to claim 2, characterized in that, The ResNet-50 dual-stream residual network based on the identity-compensated style distillation network model performs high-level semantic feature extraction on visible light modality-specific features and infrared modality-specific features to obtain visible light modality-shared features and infrared modality-shared features. This step specifically includes: Visible light modal-specific features and infrared modal-specific features are input into the ResNet-50 two-stream residual network of the identity compensation style distillation network model; Based on the first layer residual block of the dual-stream residual network ResNet-50, modal shared feature extraction processing is performed on the specific features of the visible light mode and the specific features of the infrared mode to obtain the first visible light mode shared feature and the first infrared mode shared feature. Based on the second layer residual block of the dual-stream residual network ResNet-50, modal sharing feature extraction processing is performed on the first visible light mode sharing feature and the first infrared mode sharing feature to obtain the second visible light mode sharing feature and the second infrared mode sharing feature; Based on the third layer residual block of the dual-stream residual network ResNet-50, modal sharing feature extraction processing is performed on the second visible light mode sharing feature and the second infrared mode sharing feature to obtain the third visible light mode sharing feature and the third infrared mode sharing feature. Based on the fourth layer residual block of the dual-stream residual network ResNet-50, modal sharing feature extraction processing is performed on the third visible light mode sharing feature and the third infrared mode sharing feature to obtain the fourth visible light mode sharing feature and the fourth infrared mode sharing feature. Based on the global average pooling layer and batch normalization layer of the dual-stream residual network ResNet-50, the feature dimension adjustment processing of the fourth visible light mode shared features and the fourth infrared mode shared features is performed to obtain the visible light mode shared features and the infrared mode shared features.

4. The style distillation cross-modal person re-identification method based on identity compensation according to claim 3, characterized in that, The style knowledge distillation module based on the identity-compensated style distillation network model performs modal style compensation processing on the visible light modal shared features and the infrared modal shared features to obtain the compensated visible light modal shared features and the compensated infrared modal shared features. This step specifically includes: Visible light modal shared features and infrared modal shared features are input into the style knowledge distillation module of the identity compensation style distillation network model; The style difference reduction module based on the style knowledge distillation module performs style difference reduction processing on visible light modal shared features and infrared modal shared features to obtain visible light modal shared features and infrared modal shared features with consistent style. The identity knowledge compensation module based on the style knowledge distillation module performs identity knowledge compensation processing on the style-consistent visible light modal shared features and style-consistent infrared modal shared features to obtain compensated visible light modal shared features and compensated infrared modal shared features.

5. The style distillation cross-modal person re-identification method based on identity compensation according to claim 4, characterized in that, The style difference reduction module of the style knowledge distillation module performs style difference reduction processing on visible light modal shared features and infrared modal shared features to obtain style-consistent visible light modal shared features and style-consistent infrared modal shared features. This step specifically includes: The style difference reduction module based on the style knowledge distillation module calculates the Euclidean distance between the class center of the visible light modal shared feature and the infrared modal shared feature sample. The visible light modal shared feature and the infrared modal shared feature corresponding to the largest Euclidean distance are defined as the visible-infrared feature pair with the largest modal difference. Obtain the mean and standard deviation of the visible-infrared feature pairs with the greatest modal differences, and determine the style attributes of the visible-infrared feature pairs with the greatest modal differences; Based on the style attributes of the visible-infrared feature pairs with the greatest modal differences, a style difference reduction loss function is constructed and minimized to obtain style-consistent visible light modal shared features and style-consistent infrared modal shared features.

6. The style distillation cross-modal person re-identification method based on identity compensation according to claim 5, characterized in that, The identity knowledge compensation module based on the style knowledge distillation module performs identity knowledge compensation processing on style-consistent visible light modal shared features and style-consistent infrared modal shared features to obtain compensated visible light modal shared features and compensated infrared modal shared features. This step specifically includes: The identity knowledge compensation module based on the style knowledge distillation module performs style filtering on the style-consistent visible light modal shared features and style-consistent infrared modal shared features through instance normalization, and obtains visible light normalized features and infrared normalized features. The differences between the shared features and normalized features of visible light modes with consistent style, and the differences between the shared features and normalized features of infrared modes with consistent style, are calculated to obtain the visible light residual features and the infrared residual features. By using the attention channel masking mechanism, the visible light residual features and infrared residual features are processed to extract identity perception information, thus obtaining visible light identity features and infrared identity features. By fusing visible light identity features with visible light normalized features and infrared identity features with infrared normalized features, we obtain visible light enhanced identity features and infrared enhanced identity features. By introducing intramodal identity knowledge compensation loss function and intermodal identity knowledge compensation loss function, feature alignment and identity information complementarity are performed on visible light identity enhancement features and infrared identity enhancement features, respectively, to obtain compensated visible light modal shared features and compensated infrared modal shared features.

7. The style distillation cross-modal person re-identification method based on identity compensation according to claim 6, characterized in that, The loss function of the style knowledge distillation module includes a style difference reduction loss function and an identity knowledge compensation loss function, wherein: The expression for the style difference reduction loss function is as follows: In the above formula, This represents the style difference reduction loss function, where L represents the total number of stages, and l represents the l-th stage. This represents the mean of the visible light characteristics in stage l. This represents the mean value of the infrared features in stage l. This represents the standard deviation of the visible light characteristics in stage l. The standard deviation of the infrared characteristics in stage l; The specific expression for the identity knowledge compensation loss function is as follows: In the above formula, This represents the identity knowledge compensation loss function. This represents the intramodal identity knowledge compensation loss function. Let l represent the intermodal identity knowledge compensation loss function, l represent the total number of stages, and l represent the l-th stage. This indicates the identity features of the visible light characteristics in stage l. This indicates the identity characteristics of the visible light mode center in stage l. This indicates the identity characteristics of the infrared mode center in stage l. This represents the identity characteristics of the infrared mode center in stage l.

8. The style distillation cross-modal person re-identification method based on identity compensation according to claim 7, characterized in that, The identity difference amplification module based on the identity-compensated style distillation network model performs different identity feature amplification processing on the compensated visible light modal shared features and the compensated infrared modal shared features to obtain amplified visible light modal shared features and amplified infrared modal shared features. This step specifically includes: Based on the identity difference amplification module of the identity compensation style distillation network model, the modality sharing center of the compensated visible light modal sharing features and the compensated infrared modal sharing features is obtained; Define the modal sharing center and the maximum Euclidean distance between the compensated visible light modal sharing feature and the compensated infrared modal sharing feature; The identity category discrimination region of the compensated visible light modal sharing feature and the compensated infrared modal sharing feature is determined based on the maximum Euclidean distance, and difficult negative samples are mined. Instance normalization is performed on the difficult negative samples and the modal sharing center, and the difficult negative samples are pushed away from the current modal sharing center to obtain the amplified visible light modal sharing features and the amplified infrared modal sharing features.

9. The style distillation cross-modal person re-identification method based on identity compensation according to claim 8, characterized in that, The specific expression for the loss function of the identity difference amplification module is as follows: In the above formula, This represents the loss function of the identity difference amplification module, where L represents the total number of stages and l represents the l-th stage. This represents the instance-normalized modal sharing center in phase l. This represents the instance-normalized negative sample in stage l.

10. A style distillation cross-modal pedestrian re-identification system based on identity compensation, characterized in that, Includes the following modules: The first module is used to acquire visible light images and infrared images with several pedestrian identities and perform image data preprocessing to obtain preprocessed visible light images and preprocessed infrared images. The second module is used to construct an identity-compensated style distillation network model based on the two-stream residual network ResNet-50 by introducing a style knowledge distillation module and an identity difference amplification module. The third module is used to perform cross-modal style distillation pedestrian re-identification on preprocessed visible light images and preprocessed infrared images based on the identity compensation style distillation network model, and obtain cross-modal pedestrian re-identification results.

Images