Visible-infrared pedestrian re-identification method based on diffusion model prior transfer

Abstract

The application provides a visible light-infrared pedestrian re-identification method based on diffusion model prior migration, and the steps comprise: step S01. Constructing a conditional diffusion model and training; step S02. The target pedestrian identity label, target modal label and target posture skeleton graph are jointly input into the trained conditional diffusion model, an RGB-IR paired sample is generated to expand the extended training set; step S03. The input pedestrian image is input into the trained conditional diffusion model, the intermediate feature is extracted as multi-scale diffusion prior information, and after feature adaptation processing, the diffusion prior feature is generated by aggregation; step S04. Double-channel joint training is performed on the ReID network; step S05. The image to be identified is input into the trained ReID network, and the matching result is output. The application can improve the cross-modal identity recognition accuracy and the complex scene robustness, and can also alleviate the posture repetition and insufficient structure coverage problem.

Description

Technical Field

[0001] This invention relates to the field of pedestrian re-identification technology, and in particular to a visible light-infrared pedestrian re-identification method based on diffusion model prior transfer. Background Technology

[0002] Visible-Infrared Person Re-identification (VI-ReID) aims to achieve cross-modal identity matching between visible light and near-infrared cameras. Its core objective is to learn a unified and discriminative representation of pedestrian identity features under different imaging mechanisms and modal differences, and it is widely used in smart security, nighttime surveillance, and cross-camera identity retrieval. In VI-ReID tasks, visible light images typically contain rich color information (such as clothing color) and fine-grained texture features, while near-infrared images mainly record intensity or reflection responses in the near-infrared band, usually lacking the color information found in visible light images. Their texture and signal-to-noise ratio also differ significantly from visible light images. Especially at night, under occlusion, and in complex backgrounds, the appearance differences of the same person in different modalities are further amplified, making cross-modal feature alignment a key problem in VI-ReID tasks.

[0003] To address the aforementioned cross-modal feature alignment issues, traditional VI-ReID methods primarily rely on discriminative feature learning. These methods reduce feature differences between visible light and infrared images through modal alignment, shared feature extraction, cross-modal Transformers, and contrastive learning, thereby improving cross-modal identity matching capabilities. However, because infrared images lack color and texture details, and visible light images are easily affected by lighting, occlusion, and environmental changes, the aforementioned feature-learning-based traditional VI-ReID methods still suffer from unstable identity features, insufficient cross-modal consistency, and weak model generalization ability in complex scenarios.

[0004] With the significant progress made by diffusion models in image generation, some studies have begun to explore their application in VI-ReID tasks, aiming to alleviate the problem of insufficient training data by generating cross-modal pseudo-samples. These methods typically utilize pre-trained diffusion models to learn the modal mapping relationship between visible light and infrared images, and generate corresponding modal pedestrian images to expand the training dataset, enhance the model's adaptability to different modalities and complex scenes, thereby improving cross-modal pedestrian re-identification performance.

[0005] However, most existing diffusion enhancement methods remain at the "pixel-level data augmentation" stage, meaning they merely use the diffusion model as an image generation tool to generate more RGB or IR samples, failing to utilize the large-scale visual semantic prior information and identity structure prior information inherent in the diffusion model. Therefore, performance improvements in these methods often rely primarily on increasing the number of generated images, failing to achieve effective knowledge transfer from the generative model to the discriminative ReID network. Furthermore, existing diffusion enhancement methods typically employ random sampling generation strategies, lacking explicit control mechanisms for cross-modal identity structures. This results in significant deficiencies in the generated samples' pose distribution, structural diversity, and identity consistency, limiting the model's robustness to difficult poses and complex scenes. Summary of the Invention

[0006] The purpose of this invention is to provide a visible light-infrared pedestrian re-identification method based on diffusion model prior transfer, which can effectively transfer the cross-modal semantic knowledge contained in the existing diffusion model to the discriminative ReID network, improve the accuracy of cross-modal identity recognition and robustness in complex scenarios, and alleviate the problems of pose repetition and insufficient structural coverage.

[0007] To achieve the above objectives, embodiments of the present invention provide a visible-infrared pedestrian re-identification method based on diffusion model prior transfer, comprising the following steps: Step S01. Conditional diffusion model training: Construct a conditional diffusion model, which is used to generate RGB-IR image pairs under specified identity conditions, modal conditions and attitude conditions. Use visible light images and infrared images with identity labels as the initial training set to train the conditional diffusion model to obtain the trained conditional diffusion model. Step S02. Diffusion Enhancement Sample Generation: Input the target pedestrian identity tag, target modality tag, and target pose skeleton map into the trained conditional diffusion model to generate RGB-IR paired samples to expand the initial training set and form an extended training set; Step S03. Diffusion Prior Feature Extraction: Input the pedestrian images of the extended training set into the trained conditional diffusion model, extract the intermediate features output by multiple intermediate layers inside the conditional diffusion model as multi-scale diffusion prior information, and aggregate the multi-scale diffusion prior information after feature adaptation processing to generate diffusion prior features. Step S04. Dual-channel joint training: Construct a ReID network and perform dual-channel joint training on the ReID network. The pixel-level enhancement channel is trained using the extended training set according to the basic optimization objective, which includes identity classification loss and metric learning loss. The feature-level prior transfer channel aligns the diffusion prior features with the features extracted by the ReID network to achieve the transfer of semantic prior knowledge from the diffusion model to the ReID network. Step S05. Cross-modal pedestrian re-identification: During the deployment phase, the RGB or IR image to be identified is input into the pedestrian identification model to obtain query features. The pedestrian identification model is the feature extraction network in the trained ReID network. The query features are compared with the library features corresponding to the candidate image library images to calculate and sort similarity, and the candidate pedestrian images and / or candidate identity matching results that match the RGB or IR image to be identified are output.

[0008] Optionally, the conditional diffusion model includes a U-Net backbone network, a pose condition control module, and a conditional embedding module. The input of the pose condition control module is connected to a human pose skeleton map, and the output is connected to the connection positions of the downsampling layer, intermediate layer, and upsampling layer of the U-Net backbone network, respectively, to control the human pose of the generated image. The conditional embedding module is used to encode pedestrian identity tags and modal tags into identity tokens and modal identifiers, respectively, and input them into the U-Net backbone network through a cross-attention mechanism to constrain the identity attributes and modal attributes of the generated image.

[0009] Optionally, step S01 includes: An identity identifier is configured for each pedestrian in the initial training set, and the identity identifier is encoded into an identity token as an identity condition. A modal identifier is configured for each image in the initial training set as a modal condition. The identity token is used to encode the identity features of the corresponding pedestrian to control the pedestrian identity. The modal identifier includes a visible light modal identifier and a near-infrared modal identifier to control the modal attributes of the RGB / IR image. The identity token and the modal identifier are used as conditions to input into the constructed conditional diffusion model to constrain the identity attributes and modal attributes of the generated image. The identity token, the modal identifier, and the diffusion model are jointly trained until the conditional diffusion model has the ability to control identity and modality. After completing the identity token training, the backbone network parameters of the diffusion model are frozen, and a posture control branch is constructed. The posture control branch is trained using a posture skeleton graph to obtain a conditional diffusion model with identity control, modal control, and posture control capabilities. The posture skeleton graph is used to control pedestrian posture.

[0010] Optionally, training the posture control branch using the posture skeleton map includes: inputting the posture skeleton map into the posture control branch; processing the posture skeleton map to generate posture control features that match the resolution and channel dimensions of each intermediate layer of the conditional diffusion model; superimposing the posture control features onto the connection positions of the downsampling layer, intermediate layer, and upsampling layer corresponding to the backbone network in the conditional diffusion model; updating the parameters of the posture control branch during training by using the difference between predicted noise and actual added noise as the loss, so that the posture control branch learns the correspondence between the posture skeleton map and the target human posture.

[0011] Optionally, step S02 includes: selecting a target pose skeleton map from the pose training data, combining the identity token with the visible light modal identifier and the infrared modal identifier to form a text prompt word, inputting the text prompt word into a text encoder to obtain a text condition vector containing the same identity information and different modal information, outputting the text condition vector to the U-Net backbone network in the conditional diffusion model to control the person's identity and target modality in the generated image, and inputting the target pose skeleton map into the pose condition control module in the conditional diffusion model to guide the generated image to follow the target pose.

[0012] Optionally, step S03 includes: Step S301. Input encoding and condition construction: Encode the pedestrian images in the extended training set into the latent space to obtain the initial latent space representation, and fill the identity and modality identifiers of the pedestrian images into the preset text prompt template and pass them through the text encoder to obtain the conditional embedding representation, which is used to provide identity attribute constraints and modality attribute constraints to the conditional diffusion model in the prior extraction process; Step S302. Construction of intermediate diffusion states: Gaussian noise is added to the initial latent space representation according to the preset intermediate diffusion time step to construct the corresponding intermediate diffusion states; Step S303. Diffusion prior information extraction: Input the intermediate diffusion state and the conditional embedding representation into the U-Net network in the conditional diffusion model, perform one forward propagation of the U-Net network under noise conditions, without performing iterative back diffusion sampling, extract the intermediate feature maps output by multiple upsampling stages of the U-Net network, and use them as multi-scale diffusion prior information; Step S304. Prior Feature Adaptation: Perform feature adaptation processing on the extracted multi-scale diffusion prior information to generate the final diffusion prior features. The feature adaptation processing includes multi-scale hierarchical adaptation processing and spectral detail enhancement processing.

[0013] Optionally, in step S304, the multi-scale hierarchical adaptation process includes: Channel mapping and scale alignment are performed on the features at different levels in the diffusion prior information to obtain the features at each level after preliminary processing. After the initial processing, local spatial information is extracted by deep convolution of each level of features, and upsampling and fusion are performed step by step according to the hierarchical order to obtain fused multi-scale features; The spectral detail enhancement processing includes: A Laplacian high-pass convolution operation is performed on the fused multi-scale features to extract high-frequency information from the features; The extracted high-frequency information is aggregated with the fused multi-scale features to obtain the adapted diffusion prior features.

[0014] Optionally, in step S04, the following joint optimization objective is used: in, To diffuse prior alignment loss, The weighting coefficients are for the diffusion prior alignment loss. The basic optimization objective for the ReID network on an expanded training set is... It includes at least identity classification loss and loss used to constrain distance relationships in the feature space; Diffusion Prior Alignment Loss To constrain the consistency between diffusion features and ReID features in order to achieve the transfer of diffusion priors to the ReID network, it is defined as: in, For diffusion characteristics that have undergone prior adaptation, Features that have undergone projection mapping Indicates the number of samples in the batch. Indicates the sample number. This represents the mean square error.

[0015] Optionally, the basic optimization objective Defined as: in, Indicates loss in identity classification. The triplet loss is used to constrain distance relationships in the feature space. This represents the auxiliary loss term corresponding to the ReID network used; the identity classification loss uses cross-entropy loss, and the triplet loss is defined as: in, , and These represent the features of anchor points, positive samples with the same identity, and negative samples with different identities, respectively. This is the interval threshold. Represents the distance function. This represents the positive part of the function.

[0016] An electronic device includes a processor and a memory, the memory being used to store a computer program, and the processor being used to execute the computer program to perform the method described above.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention constructs a dual-channel joint training framework of pixel-level enhancement and feature-level prior transfer to jointly train the ReID network. In the pixel-level enhancement channel, an RGB-IR pairing sample with consistent identity and aligned pose is generated through a pose-controllable conditional diffusion model to expand the training set. In the feature-level prior transfer channel, multi-scale diffusion prior information is extracted from the intermediate layer of the diffusion model to generate diffusion prior features, which constrain the feature learning of the ReID network. Through the collaborative optimization of the two channels, not only can pose-controllable and identity-consistent RGB-IR pairing samples be generated, but the cross-modal identity prior contained in the diffusion model can also be effectively transferred to the ReID network, thereby improving the cross-modal identity recognition accuracy and robustness in complex scenarios.

[0018] 2. This invention decouples and controls identity conditions, modal conditions, and pose conditions, and uses identity tokens, modal identifiers, and pose skeleton diagrams to jointly constrain the diffusion generation process. The pre-learned identity token is used as the identity condition, eliminating the need to input reference pedestrian images in each generation process. At the same time, by setting visible light modal identifiers and infrared modal identifiers as constraints, the modality of the generated image can be switched, thereby generating RGB-IR image pairs with consistent identities and pose correspondences. This reduces identity offset and pose deviation between cross-modal training samples and alleviates the problems of pose repetition and insufficient structural coverage in traditional schemes.

[0019] 3. The present invention further extracts multi-scale feature maps of the backbone network of the diffusion model as multi-scale diffusion prior information, and then performs feature adaptation processing of feature filtering, fusion and enhancement processing to effectively suppress modality-related low-frequency information and enhance identity discrimination details such as human body edges, clothing textures and local accessories. Attached Figure Description

[0020] The invention will now be described in more detail with reference to embodiments and the accompanying drawings. Figure 1 This is a schematic diagram illustrating the implementation process of a visible light-infrared pedestrian re-identification method based on diffusion model prior transfer, according to an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the process for generating attitude-controllable RGB-IR image pairs in one embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of the process for implementing dual-channel joint optimization training in one embodiment of the present invention. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0024] Existing diffusion-based VI-ReID methods mainly suffer from the following problems: 1. The diffusion model has insufficient prior utilization efficiency.

[0025] Most existing methods only utilize diffusion models as image generation tools to augment training samples, without fully exploring the identity consistency semantic priors and cross-modal structural information inherent in the intermediate layers of diffusion models. In reality, diffusion models have already learned rich multi-scale visual representations and modal robust semantic features during large-scale data pre-training, but existing methods lack effective prior transfer mechanisms, preventing the effective transfer of this potential knowledge to ReID networks and thus limiting the performance gains of diffusion models in discriminative tasks.

[0026] 2. Insufficient pose coverage of generated samples.

[0027] Existing methods largely rely on random noise sampling for generation, lacking explicit human pose control mechanisms. Consequently, the generated samples exhibit significant biases in pose distribution and structural diversity, easily generating a large number of samples with similar poses or repetitive structures, making it difficult to effectively cover complex human poses such as bending over, turning around, side occlusion, and cross-viewpoint changes. In cross-camera deployment scenarios, the same identity is often accompanied by significant action and structural changes, and existing random generation methods struggle to learn stable identity structural representations, resulting in insufficient robustness of the model to difficult poses and complex scenes.

[0028] The aforementioned issues have prevented the full potential of diffusion models in VI-ReID tasks from being realized.

[0029] This invention provides a visible light-infrared pedestrian re-identification method (P2P-ReID) based on diffusion model prior transfer. It constructs a dual-channel joint training framework of "pixel-level enhancement + feature-level prior transfer" to jointly train the ReID (pedestrian re-identification) network. In the pixel-level enhancement channel, a pose-controllable conditional diffusion model generates RGB-IR paired samples with consistent identity and pose alignment to expand the training set. In the feature-level prior transfer channel, multi-scale diffusion prior information from the intermediate layers of the diffusion model is extracted to generate diffusion prior features, constraining the feature learning of the ReID network. Through the collaborative optimization of the two channels, not only can pose-controllable and identity-consistent RGB-IR paired samples be generated, but the cross-modal identity priors inherent in the diffusion model can also be effectively transferred to the ReID network. This extends the diffusion model from a traditional image generation tool to a provider of cross-modal identity priors, enabling the effective transfer of cross-modal semantic knowledge inherent in the diffusion model to the discriminative ReID network, thereby improving the accuracy of cross-modal identity recognition and robustness in complex scenarios.

[0030] The specific implementation steps of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0031] like Figure 1 As shown, the steps of the visible light-infrared pedestrian re-identification method based on diffusion model prior transfer in this embodiment include: Step S01. Conditional Diffusion Model Training: Construct a conditional diffusion model, which is used to generate RGB-IR image pairs under specified identity, modality, and pose constraints. Use visible light and infrared images with identity labels as the initial training set to train the conditional diffusion model and obtain the trained conditional diffusion model.

[0032] As an optional implementation, the conditional diffusion model includes a U-Net backbone network, a pose condition control module ControlNet, and a conditional embedding module. The input of the pose condition control module is connected to the human pose skeleton map, and the output is connected to the connection positions of the downsampling layer, intermediate layer, and upsampling layer of the U-Net backbone network, respectively, to control the human pose of the generated image. The conditional embedding module is used to encode pedestrian identity tags and modal tags into identity tokens and modal identifiers, respectively, and input them into the U-Net backbone network through a cross-attention mechanism to constrain the identity attributes and modal attributes of the generated image.

[0033] As an optional implementation, the steps for constructing and training the conditional diffusion model include: Step S101. Construct an identity condition module for generating corresponding identity tokens for different identities and a modal condition module for generating modal identifiers. The identity token is used to encode the identity features of the corresponding pedestrian to control the pedestrian's identity. The modal identifiers include visible light modal identifiers and near-infrared modal identifiers to control the modal attributes of RGB / IR images. The identity tokens and modal identifiers are used as conditional inputs to the constructed conditional diffusion model to constrain the identity attributes and modal attributes of the generated image. The identity tokens, modal identifiers, and diffusion model are jointly trained until the conditional diffusion model has the ability to control identity and modality.

[0034] Specifically, visible light and infrared pedestrian images with identification labels can be obtained from publicly available datasets. Human pose estimation algorithms such as OpenPose can be used to extract human key points from the visible light pedestrian images, and a human pose skeleton map can be generated based on the spatial connectivity between the key points. Furthermore, poor-quality samples can be removed through methods such as image sharpness detection and key point integrity detection to construct a pose training dataset containing representative poses such as walking, turning, and turning.

[0035] Before training the pose control network, an identity encoding mechanism and a modality encoding mechanism are first constructed for conditional control of the diffusion model. Specifically, for each pedestrian identity in the training dataset, a learnable identity identifier is introduced. The model was then optimized using a textual inversion method. During training, the main parameters of the pre-trained diffusion model were kept frozen, and only the text embeddings corresponding to the identity identifiers were updated to encode the visual features and appearance attributes related to the target pedestrian's identity. After optimization, each identity corresponds to an independent identity token. All identity tokens together form the Identity Token Bank, which is used to provide identity condition constraints during subsequent generation processes.

[0036] At the same time, modal identifiers are introduced. As modal conditions, where These represent the visible light mode and the near-infrared mode, respectively. The modal identifier is mapped to the corresponding modal conditional embedding through a text encoder, used to describe the sensor modal attribute to which the generated image belongs. During the diffusion model generation process, the identity token is... With modal identifiers The corresponding conditional embeddings a common input text conditional encoding module, and constrain the generation process through a conditional interaction mechanism in the diffusion model. Here, identity conditions ensure consistency between the generated result and the target identity, while modality conditions control the cross-modal attributes of the generated image. By jointly optimizing identity and modality conditions, the diffusion model can achieve controllable RGB / IR cross-modal pedestrian image generation while maintaining identity consistency, providing a foundation for cross-modal generation with identity preservation capabilities for subsequent pose control network training.

[0037] Step S102. After completing the identity token training, freeze the backbone network parameters of the diffusion model and construct the attitude control branch ControlNet. Use the attitude skeleton graph to train the attitude control branch to obtain a conditional diffusion model with identity control, modal control and attitude control capabilities.

[0038] Specifically, after completing the identity token training, the parameters of the diffusion model backbone network are frozen, and a pose control branch ControlNet is built on it. The ControlNet is trained using the selected pedestrian images and their corresponding pose skeleton diagrams, so that it learns the mapping relationship between the human pose structure and the pose skeleton diagram, thereby achieving precise control of the human pose in the generated image.

[0039] As an optional implementation, training the posture control branch using the posture skeleton diagram includes: inputting the posture skeleton diagram into the posture conditional control module ControlNet; the posture conditional control module ControlNet processes the posture skeleton diagram to generate posture control features that match the resolution and channel dimensions of each intermediate layer of the conditional diffusion model; superimposing the posture control features onto the connection positions of the downsampling layer, intermediate layer, and upsampling layer corresponding to the backbone network in the conditional diffusion model; during training, using the difference between predicted noise and actual added noise as the loss to update the parameters of the posture control branch, so that the posture control branch learns the correspondence between the posture skeleton diagram and the target human posture.

[0040] Specifically, the training images Input the diffusion model encoder to obtain the latent space representation. Randomly select diffusion time steps and represent to the latent space Add Gaussian noise To obtain the noisy latent variables .in, The calculation formula is: Adding noise to latent variables diffusion time step The text conditional vectors are input into the U-Net network of the diffusion model, while the pose skeleton map is input into the pose conditional control module ControlNet. ControlNet transforms the pose skeleton map into multi-scale pose control features that match the resolution and channel dimensions of each layer of U-Net through convolution and progressive downsampling. Zero-initialized convolutional layers are set at the output of each layer of ControlNet, and the processed pose control features are superimposed onto the connection positions of the corresponding downsampling, intermediate, and upsampling layers of the U-Net network. During training, the mean squared error between the predicted noise and the actual added noise is calculated to update the ControlNet parameters, enabling it to learn the correspondence between the pose skeleton map and the target human pose.

[0041] After the above training process, the joint optimization of identity tokens, modal identifiers, and attitude control networks is completed, resulting in a conditional diffusion model with identity control, modal control, and attitude control capabilities, which provides a foundation for the subsequent generation of attitude-controllable RGB-IR paired samples.

[0042] Step S02. Diffusion Enhancement Sample Generation: Input the target pedestrian identity tag, target modality tag, and target pose skeleton map into the trained conditional diffusion model to generate RGB-IR paired samples to expand the initial training set into an extended training set.

[0043] Traditional pose-controlled generation methods typically input a reference pedestrian image into a reference U-Net network to preserve the identity of the person in the generated image. They use skeleton maps, depth maps, or surface normal maps obtained from a 3D human model as spatial control conditions to generate single-modal pedestrian images with different poses and viewpoints. However, these methods require a reference image for each generation and can only focus on pose and viewpoint expansion within a single modality, making it difficult to generate paired RGB and IR images with consistent identity and pose alignment.

[0044] This embodiment uses identity tokens, modal identifiers, and pose skeleton maps to jointly constrain the diffusion generation process. The pre-learned identity token serves as the identity condition, eliminating the need to input a reference pedestrian image in each generation process. Furthermore, by setting visible light and infrared modal identifiers as constraints, the modality of the generated images can be switched, thereby generating RGB-IR image pairs with consistent identities and pose correspondences. In other words, by switching modal conditions while keeping identity and pose conditions unchanged, RGB-IR paired samples sharing the same identity and pose conditions are generated. For example, keeping the identity token, target pose skeleton map, and other generation parameters unchanged, first, a visible light modal identifier is input to generate an RGB pedestrian image. Then, only the modal identifier is switched to an infrared modal identifier to generate the corresponding IR pedestrian image, thus obtaining RGB-IR paired samples with the same identity, aligned poses, but different modalities. By sequentially inputting multiple representative pose templates for the same identity, RGB-IR paired samples with different poses can be generated, and the generated samples are merged with real training data to form an augmented training dataset.

[0045] As an optional implementation method, such as Figure 2 As shown, the steps for generating pose-controllable RGB-IR image pairs include: selecting a target pose skeleton map from the pose training data, and combining the target identity token with a visible light modal identifier / infrared modal identifier to form text prompts, for example, using a prompt template. Fields without parentheses (such as " "", "", This constitutes a fixed semantic skeleton; Placeholders representing modal identifiers. A placeholder representing an identity token; this is achieved by using variables in this template. and The text prompts are replaced with specific feature phrases. The text prompts are input into the text encoder to obtain text conditional vectors containing the same identity information and different modal information. These text conditional vectors are then output to the U-Net backbone network in the conditional diffusion model. Through the cross-attention layer in U-Net, they participate in noise prediction to control the identity and target modality of the generated image. The target pose skeleton map is also input into the pose conditional control module ControlNet in the conditional diffusion model to guide the generated image to follow the target pose. Finally, an image with the target identity, target modality, and target pose is generated. Specifically, the pose conditional control module ControlNet converts the pose skeleton map into multi-scale pose control features that match each layer of U-Net, and then injects these pose control features layer by layer into the downsampling layers, intermediate layers, and upsampling connection positions of U-Net. During the iterative denoising process, the pose control features continuously constrain the human contour, joint positions, and limb structures in the generated image, thereby guiding the generated image to follow the target pose.

[0046] This embodiment decouples the control of identity conditions, modality conditions, and pose conditions. It uses identity conditions and modality conditions to generate content through text encoding and cross-attention control. The pose conditions are injected into the U-Net network of the conditional diffusion model layer by layer in the form of multi-scale control features by the pose condition control module ControlNet. By keeping the identity conditions and pose conditions unchanged and switching the modality conditions, it is possible to generate RGB images and IR images with clear cross-modal correspondences in pairs, obtain pixel-level training samples, and reduce identity offset and pose deviation between cross-modal training samples.

[0047] Step S03. Diffusion Prior Feature Extraction: Input pedestrian images from the extended training set (i.e., pedestrian images from the initial training set and cross-modal pedestrian images generated by the conditional diffusion model) into the trained conditional diffusion model, extract intermediate features from multiple intermediate layers within the conditional diffusion model as multi-scale diffusion prior information, and aggregate the multi-scale diffusion prior information after feature adaptation processing to generate diffusion prior features.

[0048] This embodiment extracts multi-scale identity semantic information contained in multiple intermediate layers of the diffusion model, and then generates diffusion prior features through feature adaptation and aggregation. This fully explores the large-scale visual semantic prior information and identity structure prior information contained within the diffusion model. Subsequently, the diffusion prior features are transferred to the ReID network, which can improve the identity feature expression ability and cross-modal consistency of the ReID network.

[0049] As an optional implementation, the steps of diffusion prior feature extraction include: Step S301. Input encoding and condition construction: Encode the pedestrian images in the extended training set into the latent space to obtain the initial latent space representation, and fill the identity and modality identifiers of the pedestrian images into the preset text prompt template and pass them through the text encoder to obtain the conditional embedding representation, which is used to provide identity attribute constraints and modality attribute constraints to the conditional diffusion model in the prior extraction process.

[0050] Specifically, the pedestrian image can be input into the variational autoencoder (VAE) of the diffusion model, and the corresponding latent space representation can be obtained through the encoder. Based on the identity label of the input pedestrian image, the corresponding identity token is obtained from the pre-established identity token library; based on the modality to which the image belongs, the corresponding visible light or infrared modality identifier is obtained; the identity identifier (identity token) and modality identifier are filled into the preset text prompt template, and the conditional embedding representation is obtained through the text encoder. This conditional embedding representation is used to provide the diffusion model with the person's identity and image modality constraints during the prior extraction process.

[0051] Step S302. Construction of intermediate diffusion state: Gaussian noise is added to the initial latent space representation according to the preset intermediate diffusion time step to construct the corresponding intermediate diffusion state.

[0052] Specifically, a preset intermediate diffusion time step is selected, and Gaussian noise is added to the initial latent space representation according to the noise scheduling parameters corresponding to that time step to construct the intermediate diffusion state. For example, the calculation method can be expressed as: in, This represents the initial latent space. For Gaussian noise that follows a standard normal distribution, For diffusion time step The corresponding noise scheduling parameters. By selecting an intermediate diffusion time step, it is possible to reduce modality-specific information such as color, brightness, and sensor texture while preserving identity-related information such as human body contours, clothing structure, and local accessories.

[0053] Step S303. Diffusion Prior Information Extraction: Input the intermediate diffusion state and conditional embedding representation into the U-Net network in the conditional diffusion model, perform one forward propagation of the U-Net network under noise conditions, without performing iterative back-diffusion sampling, extract the intermediate feature maps output by multiple upsampling stages of the U-Net network, and use them as multi-scale diffusion prior information.

[0054] Specifically, the intermediate diffusion state Conditional embedding means that the input is fed into the U-Net backbone network in the conditional diffusion model, and multiple intermediate feature maps, i.e. multi-scale feature maps, are extracted from the output of the U-Net backbone network. This allows us to obtain the multi-scale identity semantic information contained in the intermediate layers of the diffusion model, and the multi-scale features are used as the multi-scale diffusion prior information.

[0055] Step S304. Prior Feature Adaptation: Perform feature adaptation processing on the extracted multi-scale diffusion prior information to obtain adapted diffusion prior features. Feature adaptation processing includes multi-scale hierarchical adaptation processing and spectral detail enhancement processing.

[0056] Since the diffusion model is mainly geared towards image generation tasks, its features still include background information, color style, and modality-related textures that are irrelevant to identity recognition. This embodiment adapts the intermediate features of the diffusion model to filter, fuse, and enhance the extracted multi-scale diffusion prior information, making it suitable for the VI-ReID task.

[0057] To achieve effective transfer of diffusion prior features to discriminative identity features, this embodiment first adapts the multi-level features output by the diffusion model U-Net through a feature adaptation module. This feature adaptation module includes a Hierarchical Feature Fusion Network (HFFN) and a Spectral Detail Aggregation (SDA) unit. HFFN performs multi-scale hierarchical adaptation processing to achieve hierarchical fusion of diffusion features at different scales, while SDA performs spectral detail enhancement processing to enhance the high-frequency discriminative details in the fused features.

[0058] Specifically, the multi-scale features output by U-Net at different upsampling stages in the diffusion model are represented as follows: in, Indicates the first The diffusion characteristics of the output during the layer upsampling stage.

[0059] As an optional implementation, performing multi-scale hierarchical adaptation processing includes: First, HFFN performs channel mapping and scale alignment on features at different levels to obtain the pre-processed features at each level. The process can be represented as follows: in, This indicates a pointwise convolution operation, used for channel dimension mapping of features at different levels; This indicates a normalization operation; This represents the activation function.

[0060] Subsequently, the features at each level after preliminary processing were analyzed. Depth convolution is used to further extract local spatial information, and upsampling and fusion are performed step by step in a coarse-to-fine hierarchical order: in, Indicates an upsampling operation. This represents the fused multi-scale features. Through the above process, HFFN achieves joint modeling of high-level semantic information and low-level spatial detail information.

[0061] As an optional implementation, performing spectral detail enhancement processing includes: Obtaining multi-scale features after fusion Subsequently, the high-frequency information related to identity determination is further enhanced using the Spectral Details Aggregation Unit (SDA). This process can be represented as follows: in, This represents the Laplacian high-pass convolution operation, used to extract high-frequency response information from features.

[0062] Subsequently, high frequency response Multi-scale features after fusion with the original Perform adaptive aggregation: in, Represents the feature fusion function. This represents the diffusion prior features after feature adaptation. Finally, semantic fusion of the multi-scale diffusion features is performed using HFFN, and identity discrimination details such as human body contours, clothing textures, and local structures are enhanced using SDA, resulting in a discriminative prior representation suitable for the VI-ReID task.

[0063] Further analysis of the diffusion prior features after feature adaptation By using feature projection processing, the diffusion prior features are mapped from the generation space to the ReID space to obtain the final diffusion prior features.

[0064] Traditional diffusion models typically prioritize low-frequency global semantics and generation consistency in their generated features. However, person re-identification tasks rely more heavily on high-frequency identity details such as clothing texture, body contours, and accessories. This creates a significant "generation-to-discrimination gap" between generated and discriminative features, further limiting the effectiveness of diffusion models in VI-ReID tasks. This embodiment extracts multi-scale feature maps from the diffusion model's backbone network as multi-scale diffusion prior information. Through feature adaptation processing involving feature filtering, fusion, and enhancement, it effectively suppresses modality-related low-frequency information and enhances identity discrimination details such as body edges, clothing textures, and local accessories.

[0065] Optionally, the diffusion prior features can be further processed by refining the direction details to filter out low-frequency information that is irrelevant to the discrimination, such as background style, and highlight high-frequency discriminative details such as clothing texture and outline.

[0066] In the subsequent training process, by inputting pedestrian images into the ReID network to obtain corresponding student features, and then mapping the diffusion prior features to a feature space consistent with the student features through feature adaptation processing, the representation difference between the student features and the diffusion prior features is constrained by the prior alignment loss. This can guide the ReID network to learn identity consistency semantic information and cross-modal shared structural information in the diffusion model, thereby improving the cross-modal feature alignment ability and identity discrimination ability in visible light and infrared pedestrian re-identification tasks.

[0067] Step S04. Joint Training of ReID Network: Construct a ReID network and perform dual-channel joint training on the ReID network. The pixel-level enhancement channel is trained according to the basic optimization objectives, which include identity classification loss and metric learning loss. The feature-level prior transfer channel aligns the diffusion prior features with the features extracted by the ReID network to achieve the transfer of semantic prior knowledge (multi-scale diffusion prior information) inside the diffusion model to the ReID network.

[0068] In this embodiment, a dual-channel joint training approach combining pixel-level enhancement and feature-level prior transfer is employed to jointly train the ReID network. The pixel-level enhancement channel utilizes a diffusion model to generate RGB-IR paired samples with consistent identities and pose correspondences, which, together with real samples, form an expanded training set, thus alleviating the problem of limited data scale in existing visible-near-infrared pedestrian re-identification datasets. The feature-level prior transfer channel utilizes the human structure information and identity semantic priors contained in the intermediate layers of the diffusion model, extracting diffusion prior features and transferring them to the ReID network to enhance cross-modal identity representation capabilities.

[0069] like Figure 3As shown, in the feature-level prior transfer channel (prior-level enhancement channel), text prompts (identity / modal identifiers) and person images are input into the U-Net backbone network of the diffusion model. U-Net extracts output feature maps from multiple intermediate layers of the backbone network as multi-scale diffusion prior information. This diffusion prior information undergoes hierarchical fusion of diffusion features at different scales using HFFN and feature adaptation processing for spectral detail enhancement using SDA to obtain diffusion prior features. These diffusion prior features are then aligned with the features extracted by the ReID network to achieve semantic prior knowledge within the diffusion model. The ReID network is transferred to the feature-level prior transfer channel. In the pixel-level enhancement channel, identity tokens, modal identifiers, and pose skeleton maps are input into the U-Net backbone network of the diffusion model to generate RGB-IR paired samples with consistent identity and pose alignment to form an expanded training set. The feature-level prior transfer channel shares the U-Net backbone network of the diffusion model and identity / modal adaptation parameters (identity token library and modal identifiers, etc.) with the pixel-level enhancement channel. The pose control branch is only enabled in the pixel-level enhancement channel, and the pose control branch is not enabled in the feature-level prior transfer channel. The ReID network is trained using the expanded training set. During the training phase, the ReID network simultaneously receives supervision from the expanded training set and diffusion prior constraints. Through joint optimization, the synergistic improvement of identity discrimination ability and cross-modal generalization ability is achieved.

[0070] As an alternative implementation, the following joint optimization objective can be used: in, To diffuse prior alignment loss, These are the weighting coefficients for the diffusion prior alignment loss, used to balance the contributions of basic supervised learning and prior knowledge transfer. The basic optimization objective for the ReID network on an expanded training set is... It includes at least identity classification loss and loss used to constrain distance relationships in the feature space.

[0071] The above diffusion prior alignment loss This is used to constrain the consistency between diffusion features and ReID features to achieve the transfer of diffusion priors to the ReID network. In this embodiment, a diffusion prior alignment loss is introduced in addition to the basic supervised loss. This loss, by constraining the consistency between diffusion features and ReID features, achieves effective transfer of diffusion priors to the ReID network, fully utilizing the human structural information and identity semantic knowledge learned by the diffusion model.

[0072] Specifically, the pixel-level enhancement channel is optimized through the basic optimization target. Applying to the base ReID loss, the feature-level prior transfer channel aligns with the prior loss through diffusion. It applies to ReID representations.

[0073] As an optional implementation, diffusion prior alignment loss It can be defined as: in, These are diffusion prior features (i.e., diffusion prior features obtained after feature adaptation processing). The identity features extracted by the ReID network are the discriminative feature representations obtained after the input image is forward-propagated through the ReID network encoder. Indicates the number of samples in the batch. Indicates the sample number. denoted as mean squared error. This loss transfers the identity-related structure and semantic priors from the diffusion model to the ReID network by constraining the distance between the two types of features.

[0074] As an optional implementation method, the basic optimization objective It can be defined as: in, Indicates loss in identity classification. The triplet loss is used to constrain distance relationships in the feature space. This represents the auxiliary loss term corresponding to the ReID network used.

[0075] As an optional implementation, the identity classification loss can specifically employ cross-entropy loss, for example, its calculation formula can be expressed as: in, Indicates the number of samples in the batch. This indicates that the i-th sample belongs to the real identity category. The predicted probability. This loss, through supervised identity classification tasks, guides the network to learn feature representations with identity discrimination capabilities, improving the class separability between different identities.

[0076] As an optional implementation, the triplet loss can be defined as: in, , and These represent the features of anchor points, positive samples with the same identity, and negative samples with different identities, respectively. This is the interval threshold. Represents the distance function. The positive part function is represented to preserve positive values. This loss enhances the identity discrimination capability by constraining the distance relationships in the feature space, making the features of cross-modal samples with the same identity more clustered and the features of samples with different identities more separated.

[0077] This embodiment employs the aforementioned dual-channel joint optimization strategy. The pixel-level enhancement channel provides rich, identity-consistent cross-modal training samples, while the feature-level prior transfer channel injects the structural and semantic knowledge learned by the diffusion model into the ReID network. The identity classification loss and triplet loss are responsible for establishing effective identity discrimination constraints. The two channels work together to enable the model to learn more robust identity feature representations with cross-modal generalization capabilities, thereby improving the retrieval performance of the visible-near-infrared pedestrian re-identification task.

[0078] In this embodiment, during the ReID joint training phase, the diffusion teacher and its identity, modality, and pose parameters are fixed, the teacher branch gradient is blocked, and the trainable projection head (composed of a multilayer perceptron MLP) connected to the HFFN network diffuses the prior features. Mapped to the same dimension as ReID features, combined with diffusion prior alignment loss. Achieving diffusion prior features Align with ReID features.

[0079] Step S05. Cross-modal pedestrian re-identification: During the deployment phase, the RGB or IR image to be identified is input into the pedestrian identification model to obtain query features. The pedestrian identification model is the feature extraction network in the trained ReID network. The similarity between the query features and the corresponding image features of the candidate image library is calculated and sorted, and the candidate pedestrian image and / or candidate identity matching result that match the RGB or IR image to be identified is output.

[0080] Specifically, after training, the diffusion model, pose control module, and diffusion prior feature adaptation structure are removed. Pixel-level enhancement channels and feature-level prior transfer channels are not retained; only the feature extraction network in the ReID network is retained and used as the pedestrian recognition model. During model deployment, the visible light or infrared pedestrian image to be identified is input into the pedestrian recognition model to obtain query features. The similarity between the query features and the corresponding features in the candidate image library is calculated and ranked to obtain matched candidate pedestrian images, identity matching results, etc.

[0081] In specific application embodiments, the library features of candidate image libraries can be extracted and stored in advance. The similarity between query features and library features can be calculated using cosine similarity, Euclidean distance, or other feature distance functions.

[0082] This embodiment uses the diffusion model only during offline training, which improves the accuracy and robustness of cross-modal pedestrian recognition without increasing the inference complexity of the online recognition process.

[0083] In summary, this invention presents a visible-infrared pedestrian re-identification method based on diffusion prior transfer. By constructing a dual-channel collaborative framework of pixel-level enhancement and feature-level prior transfer, it effectively combines the generation capabilities of the diffusion model with discriminative ReID feature learning. This not only improves identity recognition capabilities in complex cross-modal scenarios but also enables efficient transfer of diffusion model prior knowledge to the ReID network. Furthermore, by introducing a human pose constraint mechanism, it achieves pose-controllable RGB-IR pairing generation, effectively alleviating the problems of pose repetition and insufficient structural coverage in traditional random diffusion generation.

[0084] To verify the effectiveness of this invention, experiments were conducted on multiple datasets using both the method of this invention and existing diffusion enhancement methods. The experiments show that this invention outperforms existing diffusion enhancement methods on multiple VI-ReID benchmark datasets. The cross-modal person re-identification performance is shown in Table 1. On the SYSU-MM01 and LLCM datasets, experiments were conducted using different person re-identification (ReID) backbone networks, including P2P-ReID, AGW (a person re-identification network based on attention-based generalized mean pooling and weighted triplet loss), CAJ (a visible-infrared person re-identification method based on channel enhancement joint learning), DART (a dual robust training method), and DEEN (a diversified embedding extension network). The results show that the method of this invention outperforms traditional DiVE diffusion enhancement schemes. "+" indicates that the corresponding method is further introduced for training or performance optimization based on the original backbone network. For example, "AGW+DiVE" means using AGW as the base ReID network and incorporating DiVE (Diffusion-based VI-ReID data Expansion, a visible light-infrared pedestrian re-identification data expansion method based on a diffusion model) for training. "AGW+P2P-ReID" means using AGW as the base ReID network and incorporating the P2P-ReID (Pixels-to-Priors) method of this invention for training.

[0085] Table 1: Performance Comparison of Different Methods on SYSU-MM01 and LLCM Datasets As shown in Table 1, under the SYSU-MM01 All-search (full-text search) setting, based on the AGW backbone network, the Rank-1 (first-place hit rate) of this invention improved from 57.47% to 71.97%, and the mAP (mean precision) improved from 56.18% to 68.74%. Under the DEEN backbone network, the indoor-search setting further achieved a Rank-1 of 86.29% and an mAP of 88.28%. In the LLCM dataset IR (infrared image) → VIS (visible light) task, based on the CAJ backbone network, the mAP of this invention reached 69.18%, an improvement of 12.38% compared to DiVE. Experimental results show that the diffusion prior transfer mechanism of this invention can effectively improve the consistency and discriminative ability of cross-modal identity features.

[0086] This embodiment further verifies the effect of pose-controlled diffusion enhancement. As shown in Table 2, under the same IR amplification protocol, the existing DiVE method generates 18 IR images using random poses, while the method of this invention only uses 8 pose-controlled generated images to achieve better performance across multiple backbone networks. Taking the AGW backbone network as an example, Rank-1 increased from 63.72% to 73.37%, and mAP increased from 70.54% to 78.21%; under the DART backbone network, Rank-1 increased from 76.38% to 79.84%. Experimental results show that the method of this invention can achieve higher recognition performance with fewer generated samples through pose-controlled diffusion generation, verifying the high-quality diffusion enhancement advantage of the "fewer-but-better" strategy.

[0087] Table 2: Comparison of Pixel-Level Enhanced Channel Diffusion Enhancement Effects In addition to pixel-level sample enhancement, this invention further utilizes the multi-scale features contained in the intermediate layers of the diffusion model as diffusion priors to achieve feature-level transfer of diffusion knowledge to the ReID network. This embodiment further performs dual-channel ablation experiments, and the results are shown in Table 3. The experimental results show that this invention can effectively improve the cross-modal recognition performance of the model through feature-level prior transfer channels. When used in conjunction with pixel-level enhancement channels, the model performance can be further improved. As shown in Table 3, on the SYSU-MM01 dataset, when only pixel-level enhancement channels are used, the Rank-1 reaches 78.11%; when only feature-level prior transfer channels are used, the Rank-1 reaches 75.40%; and after simultaneous dual-channel joint optimization, the Rank-1 further improves to 80.48%, and the mAP reaches 77.06%.

[0088] Table 3: Experimental Results of Dual-Channel Ablation Furthermore, this invention effectively alleviates the generation-discrimination discrepancy problem between the diffusion generation model and the discriminative ReID task by using a Hierarchical Feature Fusion Network (HFFN) and a Spectral Details Aggregation Unit (SDA), thereby enhancing the adaptability between diffusion prior features and the ReID feature space and improving the discriminativity and robustness of cross-modal identity representation. This embodiment further performs ablation experiments on the HFFN+SDA prior adapter. As shown in Table 4, compared to directly using an MLP for feature alignment, this invention, by employing the HFFN+SDA structure, can further improve Rank-1 and mAP performance.

[0089] Table 4: Ablation experimental results of HFFN+SDA prior adapter on SYSU-MM01 dataset (CAJ backbone network) This embodiment further provides an electronic device, including a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to perform the method as described above.

[0090] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0091] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0092] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0093] Although the invention has been described with reference to preferred embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, the technical features mentioned in the various embodiments can be combined in any manner as long as there is no structural conflict. The invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A visible light-infrared pedestrian re-identification method based on diffusion model prior transfer, characterized in that the steps are as follows: include: Step S01. Conditional diffusion model training: Construct a conditional diffusion model, which is used to generate RGB-IR image pairs under specified identity conditions, modal conditions and attitude conditions. Use visible light images and infrared images with identity labels as the initial training set to train the conditional diffusion model to obtain the trained conditional diffusion model. Step S02. Diffusion Enhancement Sample Generation: Input the target pedestrian identity tag, target modality tag, and target pose skeleton map into the trained conditional diffusion model to generate RGB-IR paired samples to expand the initial training set and form an extended training set; Step S03. Diffusion Prior Feature Extraction: Input the pedestrian images of the extended training set into the trained conditional diffusion model, extract the intermediate features output by multiple intermediate layers inside the conditional diffusion model as multi-scale diffusion prior information, and aggregate the multi-scale diffusion prior information after feature adaptation processing to generate diffusion prior features. Step S04. Dual-channel joint training: Construct a ReID network and perform dual-channel joint training on the ReID network. The pixel-level enhancement channel is trained using the extended training set according to the basic optimization objective, which includes identity classification loss and metric learning loss. The feature-level prior transfer channel aligns the diffusion prior features with the features extracted by the ReID network to achieve the transfer of semantic prior knowledge from the diffusion model to the ReID network. Step S05. Cross-modal pedestrian re-identification: During the deployment phase, the RGB or IR image to be identified is input into the pedestrian identification model to obtain query features. The pedestrian identification model is the feature extraction network in the trained ReID network. The query features are compared with the library features corresponding to the candidate image library images to calculate and sort similarity, and the candidate pedestrian images and / or candidate identity matching results that match the RGB or IR image to be identified are output.

2. The visible light-infrared pedestrian re-identification method based on diffusion model prior transfer as described in claim 1, characterized in that, The conditional diffusion model includes a U-Net backbone network, a pose condition control module, and a conditional embedding module. The input of the pose condition control module is connected to a human pose skeleton map, and the output is connected to the connection positions of the downsampling layer, intermediate layer, and upsampling layer of the U-Net backbone network, respectively, to control the human pose of the generated image. The conditional embedding module encodes pedestrian identity tags and modal tags into identity tokens and modal identifiers, respectively, and inputs them into the U-Net backbone network through a cross-attention mechanism to constrain the identity attributes and modal attributes of the generated image.

3. The visible light-infrared pedestrian re-identification method based on diffusion model prior transfer as described in claim 2, characterized in that, Step S01 includes: An identity identifier is configured for each pedestrian in the initial training set, and the identity identifier is encoded into an identity token as an identity condition. A modal identifier is configured for each image in the initial training set as a modal condition. The identity token is used to encode the identity features of the corresponding pedestrian to control the pedestrian identity. The modal identifier includes a visible light modal identifier and a near-infrared modal identifier to control the modal attributes of the RGB / IR image. The identity token and the modal identifier are used as conditions to input into the constructed conditional diffusion model to constrain the identity attributes and modal attributes of the generated image. The identity token, the modal identifier, and the diffusion model are jointly trained until the conditional diffusion model has the ability to control identity and modality. After completing the identity token training, the backbone network parameters of the diffusion model are frozen, and a posture control branch is constructed. The posture control branch is trained using a posture skeleton graph to obtain a conditional diffusion model with identity control, modal control, and posture control capabilities. The posture skeleton graph is used to control pedestrian posture.

4. The visible light-infrared pedestrian re-identification method based on diffusion model prior transfer according to claim 3, characterized in that, The step of training the posture control branch using the posture skeleton map includes: inputting the posture skeleton map into the posture control branch; processing the posture skeleton map to generate posture control features that match the resolution and channel dimensions of each intermediate layer of the conditional diffusion model; superimposing the posture control features onto the connection positions of the downsampling layer, intermediate layer, and upsampling layer corresponding to the backbone network in the conditional diffusion model; using the difference between predicted noise and actual added noise as the loss during training to update the parameters of the posture control branch, so that the posture control branch learns the correspondence between the posture skeleton map and the target human posture.

5. The visible light-infrared pedestrian re-identification method based on diffusion model prior transfer according to claim 1, characterized in that, Step S02 includes: selecting a target pose skeleton map from the pose training data, combining the identity token with the visible light modality identifier and the infrared modality identifier to form a text prompt word, inputting the text prompt word into a text encoder to obtain a text condition vector containing the same identity information and different modality information, outputting the text condition vector to the U-Net backbone network in the conditional diffusion model to control the person's identity and target modality in the generated image, and inputting the target pose skeleton map into the pose condition control module in the conditional diffusion model to guide the generated image to follow the target pose.

6. The visible light-infrared pedestrian re-identification method based on diffusion model prior transfer according to claim 1, characterized in that, Step S03 includes: Step S301. Input encoding and condition construction: Encode the pedestrian images in the extended training set into the latent space to obtain the initial latent space representation, and fill the identity and modality identifiers of the pedestrian images into the preset text prompt template and pass them through the text encoder to obtain the conditional embedding representation, which is used to provide identity attribute constraints and modality attribute constraints to the conditional diffusion model in the prior extraction process; Step S302. Construction of intermediate diffusion states: Gaussian noise is added to the initial latent space representation according to the preset intermediate diffusion time step to construct the corresponding intermediate diffusion states; Step S303. Diffusion prior information extraction: Input the intermediate diffusion state and the conditional embedding representation into the U-Net network in the conditional diffusion model, perform one forward propagation of the U-Net network under noise conditions, without performing iterative back diffusion sampling, extract the intermediate feature maps output by multiple upsampling stages of the U-Net network, and use them as multi-scale diffusion prior information; Step S304. Prior Feature Adaptation: Perform feature adaptation processing on the extracted multi-scale diffusion prior information to generate the final diffusion prior features. The feature adaptation processing includes multi-scale hierarchical adaptation processing and spectral detail enhancement processing.

7. The visible light-infrared pedestrian re-identification method based on diffusion model prior transfer according to claim 6, characterized in that, In step S304, the multi-scale hierarchical adaptation processing includes: Channel mapping and scale alignment are performed on the features at different levels in the diffusion prior information to obtain the features at each level after preliminary processing. After the initial processing, local spatial information is extracted by deep convolution of each level of features, and upsampling and fusion are performed step by step according to the hierarchical order to obtain fused multi-scale features; The spectral detail enhancement processing includes: A Laplacian high-pass convolution operation is performed on the fused multi-scale features to extract high-frequency information from the features; The extracted high-frequency information is aggregated with the fused multi-scale features to obtain the adapted diffusion prior features.

8. The visible-infrared pedestrian re-identification method based on diffusion model prior transfer according to any one of claims 1 to 7, characterized in that, In step S04, the following joint optimization objective is used: in, To diffuse prior alignment loss, The weighting coefficients are for the diffusion prior alignment loss. The basic optimization objective for the ReID network on an expanded training set is... It includes at least identity classification loss and loss used to constrain distance relationships in the feature space; Diffusion Prior Alignment Loss To constrain the consistency between diffusion features and ReID features in order to achieve the transfer of diffusion priors to the ReID network, it is defined as: in, For diffusion prior characteristics, Features that have undergone projection mapping Indicates the number of samples in the batch. Indicates the sample number. This represents the mean square error.

9. The visible light-infrared pedestrian re-identification method based on diffusion model prior transfer according to claim 8, characterized in that, Basic optimization objectives Defined as: in, Indicates loss in identity classification. The triplet loss is used to constrain distance relationships in the feature space. This represents the auxiliary loss term corresponding to the ReID network used; the identity classification loss uses cross-entropy loss, and the triplet loss is defined as: in, , and These represent the features of anchor points, positive samples with the same identity, and negative samples with different identities, respectively. This is the interval threshold. Represents the distance function. This represents the positive part of the function.

10. An electronic device comprising a processor and a memory, the memory being used to store a computer program, characterized in that, The processor is used to execute the computer program to perform the method as described in any one of claims 1 to 9.

Images