A visible-infrared pedestrian re-identification method and system

Abstract

The application provides a visible light-infrared pedestrian re-identification method and system, and belongs to the technical field of computer vision. Details and contours are extracted by strengthening details and contours according to the characteristics of visible light images or infrared images, and then denoising by mutual calibration of modes, bidirectional optimization of features, to obtain optimized features with cross-modal consistency and discriminability, to generate double-mode enhanced features, to solve the problem of insufficient feature extraction granularity and modal adaptability, and to avoid visible light texture loss or infrared contour fragmentation. Meanwhile, the application also avoids modal specific information degradation through mutual calibration of modes, strengthens cross-modal semantic alignment, and improves the representation capacity of the model and the robustness to complex scenes.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and in particular relates to a visible light-infrared pedestrian re-identification method and system. Background Technology

[0002] Visible-infrared pedestrian re-identification is a core technology in intelligent surveillance, public safety, and traffic management. Its task is to accurately identify and associate the same pedestrian in images or videos using both visible and infrared modalities, providing technical support for all-weather pedestrian tracking. Because visible light images rely on light reflection and contain rich color and texture information, while infrared images rely on target thermal radiation and mainly reflect contours and temperature distribution, the difference in their imaging mechanisms leads to modal differences. This makes traditional single-modal re-identification methods difficult to apply, and the effective alignment and discrimination of cross-modal features becomes the core bottleneck restricting performance. Summary of the Invention

[0003] To address the problems existing in the prior art, this invention provides a visible light-infrared pedestrian re-identification method, comprising:

[0004] Acquire visible light and infrared images from the monitored scene, and extract pedestrian target areas from the images;

[0005] The preprocessed image is transformed into initial features. Details and contours are enhanced based on the characteristics of visible light or infrared images. Then, optimized features with cross-modal consistency and discriminativeness are obtained through modal cross-calibration denoising and bidirectional feature optimization.

[0006] Calculate the feature similarity between the query image and the candidate images, and output the pedestrian identities in order of similarity.

[0007] In some implementations, the process of converting the preprocessed image into initial features, enhancing detail and contour extraction based on the characteristics of visible light or infrared images, and then obtaining optimized features with cross-modal consistency and discriminability through modal cross-calibration denoising and bidirectional feature optimization includes:

[0008] The initial features are obtained through mapping;

[0009] Calculate the channel-level global mean and standard deviation of the initial visible light characteristics, as well as the corresponding indices of the initial infrared characteristics, to clarify the differences in modal characteristics;

[0010] Based on the quantization differences, a weight distribution adapted to the two modalities is generated;

[0011] By using feature extraction paths with different dilation rates, small-scale texture features of visible light, large-scale contour features of infrared light, and mesoscale features that balance local and global characteristics are obtained respectively.

[0012] Based on the generated proprietary weights, the multi-scale features are fused to obtain dual-modal enhanced features that strengthen the core discriminative information;

[0013] Optimized features with cross-modal consistency and discriminativeness are obtained through modal mutual calibration denoising and bidirectional feature optimization.

[0014] In some implementations, generating a weight distribution adapted to the two modalities based on quantization differences includes:

[0015] The channel-level global mean and global standard deviation of the initial visible light features are concatenated in channel index order to form the visible light statistical feature vector; similarly, the channel-level global mean and global standard deviation of the initial infrared features are concatenated to form the infrared statistical feature vector.

[0016] The statistical feature vector is subjected to high-dimensional reduction, nonlinear activation, probability normalization and modality index discrimination in sequence, and discrete mode flags are output.

[0017] Global average pooling is performed on the initial features to obtain a global feature vector. The modality flag is mapped to a continuous embedding vector. The global feature vector and the embedding vector are concatenated and extended to the spatial size of the initial features. After activation normalization, a three-channel weight distribution map is generated.

[0018] In some implementations, the step of fusing multi-scale features according to generated proprietary weights to obtain dual-modal enhanced features that strengthen core discriminative information includes:

[0019] Establish the adaptation relationship between weights and multi-scale features. The exclusive weights are three-channel modal perception maps, corresponding to small, medium and large receptive field features. Among them, the first channel has a dominant weight in the visible light mode and the third channel has a dominant weight in the infrared mode.

[0020] The three receptive field features are multiplied element-wise with the corresponding channel weights to enhance the bimodal discrimination information;

[0021] The weighted three features are summed element-wise to output enhanced features that adapt to the dual-modal characteristics.

[0022] In some implementations, optimized features with cross-modal consistency and discriminativeness are obtained through modal cross-calibration denoising and bidirectional feature optimization, including:

[0023] Visible light features and infrared features are used as mutual references for mutual calibration and noise reduction;

[0024] The calibrated bimodal features are split into modality-specific detail features and cross-modal general semantic features.

[0025] Based on general semantic features, identify and remove invalid noise in modality-specific features while retaining effective discrimination details;

[0026] The refined dual-modal unique detailed features are fused and added to the general semantic features to enrich their fine-grained discriminative information;

[0027] Generate fusion coefficients based on modal requirements, fuse the optimized general features and unique features according to the coefficients, and output the optimized features.

[0028] In some embodiments, the mutual calibration and noise reduction using visible light features and infrared features as mutual references includes:

[0029] Fine-grained texture information is extracted from visible light features in a targeted manner to generate a targeted attention weight. This weight is used as a reference to suppress thermal noise in infrared features while preserving the core contour characteristics of the infrared mode.

[0030] The contour information of the infrared features is extracted in a targeted manner to generate a unique attention weight. This weight is used as a reference to filter out the illumination interference in the visible light features, while preserving the texture details of the visible light modality.

[0031] The calibrated bimodal features are connected via residuals to retain the core discriminative information inherent in the original mode, thus avoiding the loss of key features during the calibration process.

[0032] In some implementations, based on general semantic features, invalid noise in modality-specific features is identified and removed, while retaining valid discriminative details, including:

[0033] Modality-specific features and cross-modality general semantic features are mapped to the same feature space through a learnable projection matrix to establish semantic association;

[0034] Using the global common information of general semantic features as a reference, identify invalid noise that is irrelevant to identity discrimination in modality-specific features;

[0035] By using a channel-wise multiplication semantic calibration method, the feature response corresponding to noise is suppressed, while the effective discrimination of details is enhanced.

[0036] In some implementations, fusing the purified dual-modal unique detail features and supplementing them into the general semantic features to enrich their fine-grained discriminative information includes:

[0037] The purified visible light-specific detail features and infrared-specific detail features are fused together to integrate the differentiated fine-grained information of the two modes;

[0038] The fused detailed features are complemented and combined with cross-modal general semantic features to supplement the lack of detailed information in the general semantics with fine-grained information.

[0039] In some implementations, before acquiring visible light and infrared images of the monitored scene and extracting pedestrian target areas from the images, the method further includes:

[0040] The model is trained under constraints using an overall loss function, which consists of modality-aware weight-guided loss, bidirectional interactive collaboration loss, hard sample triplet loss, and identity classification loss.

[0041] This application also includes a visible light-infrared pedestrian re-identification system, comprising:

[0042] The acquisition module is used to acquire visible light and infrared images of the monitored scene and extract pedestrian target areas from the images;

[0043] The processing module is used to convert the preprocessed image into initial features, enhance details and contour extraction based on the characteristics of visible light or infrared images, and then obtain optimized features with cross-modal consistency and discriminability through modal cross-calibration denoising and bidirectional feature optimization.

[0044] The output module is used to calculate the feature similarity between the query image and the candidate images in the library, and output the pedestrian identities in order of similarity.

[0045] This invention provides a visible light-infrared pedestrian re-identification method and system. By enhancing detail and contour extraction based on the characteristics of visible light or infrared images, and then obtaining optimized features with cross-modal consistency and discriminativeness through modal mutual calibration denoising and bidirectional feature optimization, dual-modal enhanced features are generated. This solves the problems of insufficient feature extraction granularity and modal adaptability, and avoids the loss of visible light texture or fragmentation of infrared contours. At the same time, this application also avoids the degradation of modal-specific information through modal mutual calibration, strengthens cross-modal semantic alignment, and improves the model's representation capacity and robustness to complex scenes. Attached Figure Description

[0046] Figure 1 This is a basic flowchart of a visible light-infrared pedestrian re-identification method provided in the embodiments of this application;

[0047] Figure 2 This is a detailed flowchart illustrating the optimized feature acquisition process of a visible light-infrared pedestrian re-identification method provided in this application embodiment;

[0048] Figure 3 This is a detailed flowchart illustrating the modal weight distribution generation process of a visible light-infrared pedestrian re-identification method provided in this application embodiment;

[0049] Figure 4 This is a detailed flowchart illustrating the multi-scale feature fusion process of a visible light-infrared pedestrian re-identification method provided in the embodiments of this application;

[0050] Figure 5 This is a detailed flowchart illustrating the modal mutual calibration and bidirectional feature optimization of a visible light-infrared pedestrian re-identification method provided in the embodiments of this application;

[0051] Figure 6 This is a schematic diagram of the overall model architecture in a visible light-infrared pedestrian re-identification method provided in the embodiments of this application;

[0052] Figure 7 This is a schematic diagram of the modal perception dynamic receptive field processing flow architecture in a visible light-infrared pedestrian re-identification method provided in this application embodiment;

[0053] Figure 8 This is a schematic diagram of the dual-modal closed-loop interactive processing flow architecture in a visible light-infrared pedestrian re-identification method provided in this application embodiment;

[0054] Figure 9 This is a visual schematic diagram of the bimodal feature distribution under different implementation methods provided in the embodiments of this application;

[0055] Figure 10 This is a schematic diagram comparing the results of different methods provided in the embodiments of this application in the cross-modal pedestrian retrieval task;

[0056] Figure 11 This is a schematic diagram of the structure of a visible light-infrared pedestrian re-identification system provided in the embodiments of this application.

[0057] Explanation of reference numerals in the attached diagram: 1. Acquisition module; 2. Processing module; 3. Output module. Detailed Implementation

[0058] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0060] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0061] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0062] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0063] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0064] In the description of the embodiments of this application, the technical terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0065] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] Reference Figure 1 A visible light-infrared pedestrian re-identification method includes:

[0068] S10. Acquire visible light and infrared images from the monitored scene and extract pedestrian target regions from the images. Specifically, the purpose of this step is to screen the core pedestrian identification objects and construct standardized input data. The original images of the monitored scene contain background noise, such as buildings, vegetation, and vehicles, and the positions and sizes of pedestrians may vary. Extracting target regions can unify the images into an input format suitable for the model, ensuring that subsequent feature extraction focuses only on the pedestrian's appearance, texture, contour, and other identity-related information, avoiding irrelevant information from interfering with model training and inference.

[0069] S20. The preprocessed image is transformed into initial features. Based on the characteristics of visible light or infrared images, details and contour extraction are enhanced. Then, through modal mutual calibration denoising and bidirectional feature optimization, optimized features with cross-modal consistency and discriminativeness are obtained. Specifically, the preprocessed image is first mapped to initial features through a dual-branch independent parameter network. The receptive field is dynamically allocated based on the differences in dual-modal characteristics. The small receptive field in visible light enhances texture, and the large receptive field in the outer infrared region enhances contour. Finally, the modal mutual calibration denoising and bidirectional feature optimization are realized through the dual-modal closed-loop interaction module, and the optimized features are output. This solves the problems of insufficient adaptability of traditional fixed receptive fields and information degradation caused by unidirectional feature interaction, narrows the modal gap, and generates high-quality features with both cross-modal consistency and modal discriminativeness.

[0070] For example, the initial features are mapped using a Transformer dual-branch network, the dual-modal details and contours are dynamically captured through a multi-head self-attention mechanism, the modal mutual calibration is achieved using cross-attention, and the bidirectional optimization of features is constrained by contrastive learning, outputting optimized features;

[0071] Initial features are extracted using a lightweight MobileNet network, and multi-scale conventional convolutions (3×3, 5×5, 7×7) are used to enhance key information of the dual modality. Mutual calibration and denoising are achieved with the help of a modality adaptive gating mechanism, and bidirectional optimization is achieved through feature concatenation and 1×1 convolution.

[0072] Initial features are obtained by fine-tuning a pre-trained ResNet50 model. Modal differences are quantified based on mutual information and feature extraction scales are assigned. Modal mutual calibration is achieved through orthogonal constraints. Bidirectional optimization is completed by combining feature weighted summation to generate optimized features adapted to both modalities.

[0073] In some implementations, refer to Figure 2 S20. The preprocessed image is transformed into initial features. Based on the characteristics of visible light or infrared images, details and contour extraction are enhanced. Then, through modal cross-calibration denoising and bidirectional feature optimization, optimized features with cross-modal consistency and discriminativeness are obtained, including:

[0074] S21. Initial features are obtained through mapping. Specifically, a dual-branch convolutional neural network structure is adopted, with the two branches having the same topology but independent parameters. They process visible light and infrared preprocessed images respectively. The image pixels are mapped to high-dimensional structured initial features through concatenated operations of convolutional blocks, avoiding information interference between the two modes and providing pure and consistent underlying feature support for subsequent modal characteristic quantification and multi-scale feature extraction.

[0075] For example, a basic feature extraction network is constructed, which adopts a dual-branch convolutional neural network structure. The network topology of the two branches is exactly the same, but the parameters are independent of each other. One branch is dedicated to processing visible light modal images, and the other branch is dedicated to processing infrared modal images. In this embodiment, the input visible light and infrared pedestrian images are uniformly 128×64 in size. Each branch specifically contains four concatenated convolutional blocks. Each convolutional block is composed of a 3×3 convolutional layer, a batch normalization layer, a ReLU activation function, and a 2×2 max pooling layer in sequence (it should be noted that the "convolutional block" in this embodiment is a conventional combination, and those skilled in the art can replace it with existing feature extraction units such as residual blocks and bottleneck blocks according to actual needs). The number of output channels of each convolutional block is configured sequentially as 64, 128, 256, and 512 according to the training process. After processing by the last convolutional block, the basic features of the visible light modality are finally output. Basic characteristics of infrared modes Where B is the batch size, C=512 is the number of channels, H=8 and W=4 are the height and width of the feature map, respectively.

[0076] S22. Calculate the channel-level global mean and standard deviation of the initial visible light features, and the corresponding indices of the initial infrared features, to clarify the differences in modal characteristics. Specifically, for the initial visible light and infrared features, calculate the channel-level global mean and standard deviation for all spatial locations. Quantify the activation level and distribution dispersion of the dual-modal features through two types of statistical indices, clarify the differences in modal characteristics, and transform the abstract characteristics of dual-modality into quantifiable statistical indices, providing data support for the subsequent generation of modality-specific weight distributions.

[0077] For example, the channel-level global mean is calculated for the initial features of visible light and infrared light respectively, using the following formula:

[0078]

[0079]

[0080] in, These are the characteristic values ​​of visible light at channel c, height h, and width w. The value corresponds to the position of the infrared feature; , , , , Given the feature map size, the summation operation iterates through all spatial locations of the feature map. (Average visible light intensity) is generally higher than (Infrared mean), used to characterize the overall activation level of features.

[0081] The channel-level global standard deviation is calculated for each of the dual-modal features, using the following formula:

[0082]

[0083]

[0084] The calculation results are used to reflect the degree of dispersion of the feature distribution. Since infrared is activated only by the contour, Generally higher than .

[0085] S23. Based on quantized differences, generate weight distributions adapted to the two modalities. Specifically, first, the mean and standard deviation of the two modalities are concatenated into a statistical feature vector. After dimensionality reduction, activation, and probability discrimination, discrete modal labels are generated. Then, through global average pooling, modal embedding, concatenation, broadcast expansion, and activation normalization, a three-channel modal perception weight distribution is generated, providing modality-specific guidance for multi-scale feature fusion. The quantified modal differences are transformed into spatially adaptive weight distributions, ensuring that visible light emphasizes small receptive fields and the outer infrared light emphasizes large receptive fields, providing a clear basis for subsequent multi-scale feature extraction.

[0086] For example, principal component analysis (PCA) is used to reduce the dimensionality of the statistical feature vectors. The modal probability distribution is generated by Softmax and mapped to a continuous vector through an embedding layer. After concatenation with the global pooling features, a three-channel weight distribution is generated. The channel correlation of the bimodal features is calculated based on mutual information to generate a modal difference matrix. This matrix is ​​mapped to a modal label through a fully connected layer and then expanded into a spatial weight map through upsampling to achieve modality-specific guidance. An attention mechanism is used to directly perform modality discrimination on the initial features to generate a single-channel modal attention map. Then, through channel splitting and weight adjustment, a three-channel weight distribution adapted to multi-scale features is obtained.

[0087] In some implementations, refer to Figure 3 S23. Based on quantization differences, the weight distribution adapted to the two modalities is generated, including:

[0088] S231. Concatenate the channel-level global mean and global standard deviation of the initial visible light features in channel index order to form a visible light statistical feature vector; similarly, concatenate the channel-level global mean and global standard deviation of the initial infrared features to form an infrared statistical feature vector; specifically, in this step, concatenate the channel-level global mean and global standard deviation of each modality in channel index order to form a high-dimensional feature vector containing the statistical differences between the two modalities, thereby integrating the two types of statistical information, activation level and distribution dispersion of the two modalities, into a unified vector, providing complete statistical data support for subsequent modality discrimination and weight generation.

[0089] For example, the mean and standard deviation of each channel are concatenated to form a statistical feature vector.

[0090] For the visible light mode, the calculated global mean of each channel will be used. The calculated global standard deviation of each channel The channels are concatenated sequentially according to their indexes to form the statistical feature vector of the visible light modes:

[0091]

[0092] Similarly, for the infrared mode, the global average value of each channel is... Compared with global standard deviation The data is concatenated in order of channel index to form the statistical feature vector of the infrared modes. .

[0093] Each modality has a statistical feature vector with a dimension of 1024, which is used to generate subsequent modality labels.

[0094] S232. Perform high-dimensional reduction, nonlinear activation, probability normalization, and modality index discrimination on the statistical feature vector in sequence, and output discrete modality labels. Specifically, this step performs high-dimensional reduction, nonlinear activation, probability normalization, and modality index discrimination on the high-dimensional statistical feature vector in sequence. Through multi-step mapping, the modality category is accurately determined, and discrete modality labels are output. This transforms the high-dimensional statistical features into clear discrete modality identifiers, providing a direct basis for the modality-specific guidance of the subsequent weight distribution map.

[0095] For example, modality labeling is achieved through feature dimensionality reduction, nonlinear activation, probability normalization, and modality index selection, as shown in the following formula:

[0096]

[0097] in The statistical feature vector (visible light) obtained in the previous step Infrared ); This indicates a fully connected layer 1. This indicates a fully connected layer 2; ReLU is the activation function, Softmax is used for probability normalization, and argmax is used to select the modality index.

[0098] The specific process can be represented as: converting statistical feature vectors... The input is a fully connected layer 1, with an input dimension of 1024 and an output dimension of 256. This layer achieves dimensionality reduction and nonlinear mapping of high-dimensional statistical features, outputting a 256-dimensional feature vector. The output of is activated by ReLU, as shown in the formula: This enhances the nonlinear expressive power of features and suppresses gradient vanishing. The activated 256-dimensional features are input into a fully connected layer 2, which has an input dimension of 256 and an output dimension of 2. The output vector corresponds to the original discriminant scores of the two modalities. Applying the Softmax function to the output yields the modal probability distribution. The modal flag M is obtained by taking the index corresponding to the maximum probability using the argmax function. If the value is 0, then the value of M is 0 (corresponding to the visible light mode); otherwise, the value of M is 1 (corresponding to the infrared mode).

[0099] S233. Perform global average pooling on the initial features to obtain a global feature vector. Map the modality markers to continuous embedding vectors. Concatenate the global feature vector and the embedding vectors and expand to the spatial size of the initial features. After activation normalization, generate a three-channel weight distribution map. Specifically, this step performs global average pooling on the initial features to obtain a global feature vector. Map the discrete modality markers to continuous embedding vectors. Concatenate the two types of vectors and expand to the spatial size of the initial features. After activation normalization, generate a three-channel weight distribution map, thereby fusing the modality prior information with the global feature information and transforming it into a spatially adaptive weight distribution, ensuring the matching of weights with modal characteristics and feature spatial location.

[0100] For example, discrete modal labels M are mapped to a spatial adaptive weight distribution to generate a three-channel modal sensing map. The specific formula can be expressed as:

[0101]

[0102] in, The basic features output in step S210; This indicates a global average pooling operation; For embedding layer; For splicing operations; Indicates a broadcast operation; This is the Sigmoid activation function.

[0103] The specific process can be represented as: For each sample's basic features... Global average pooling is performed, iterating through the channel and spatial dimensions of each sample, and calculating the global average value for each channel to obtain a 512-dimensional global feature vector. An embedding layer maps the discrete label M to a 64-dimensional continuous embedding vector E, realizing a numerical representation of the modal prior. The global feature vector and the modal embedding vector E are concatenated along the channel dimension, fusing global feature statistics and modal attribute information to obtain... The fusion features of the dimensions; broadcast the spliced ​​fusion features to The spatial dimensions are ensured to be consistent with the spatial resolution of the base feature map, providing support for pixel-by-pixel weight allocation; [the following is a separate, unrelated sentence:] ...using... The convolution operation compresses the channel dimension of the fused features from 576 to 3; the Sigmoid activation function normalizes the feature values ​​to... Interval, generate three-channel modal sensing map

[0104] S24. Through feature extraction paths with different dilation rates, small-scale texture features for visible light, large-scale contour features for infrared, and mesoscale features that balance local and global information are obtained respectively. Specifically, three parallel dilated convolution branches with different dilation rates are constructed, none of which contain pooling layers, to extract small, medium, and large receptive field features respectively. This adapts to the different needs of dual modalities for details and contours, thereby breaking through the limitations of traditional fixed-size convolution kernels. It provides a small receptive field for visible light to capture textures, a large receptive field for infrared to integrate contours, and at the same time obtains mesoscale features to balance local and global information.

[0105] A three-way parallel dilated convolution branch is constructed for extracting differential receptive field features, with the input of each branch being the basic features. Ensure the spatial resolution of output features Consistent with the 512 channel count and input, this avoids the loss of fine-grained discrimination information. Specific parameter configurations are as follows:

[0106] Branch 1: Expansion Rate The equivalent receptive field is It focuses on extracting local detailed features and adapts to visible light modes;

[0107] Branch 2: Expansion Rate The equivalent receptive field is It emphasizes balancing local and global information;

[0108] Branch 3: Expansion Rate The equivalent receptive field is It focuses on extracting global contour features and adapts to infrared modes.

[0109] Basic features The three parallel dilated convolution branches mentioned above are input separately, and after 3×3 dilated convolution, batch normalization, and ReLU activation, the corresponding small receptive field features are output. Mid-sensory field characteristics Large receptive field characteristics The dimensions of all three types of features are .

[0110] S25. Based on the generated exclusive weights, the multi-scale features are fused to obtain dual-modal enhanced features that strengthen the core discrimination information. Specifically, based on the three-channel modal perception weight distribution, the three multi-scale features are multiplied element-wise with the corresponding channel weights, and then element-wise summation is performed to achieve modality-specific multi-scale feature fusion, strengthen the dual-modal core discrimination information, strengthen the core discrimination information such as visible light texture and infrared contour through modality-specific weights, suppress irrelevant information, and generate enhanced features that adapt to dual-modal characteristics.

[0111] For example, an attention pooling mechanism is used to fuse multi-scale features with modality-aware weights. The fusion ratio is then dynamically adjusted through global average pooling and fully connected layers to output enhanced features. Channel attention weighting is applied to the weighted multi-scale features, and then they are fused with the original features through residual connections to strengthen core information while preserving the details of the original features. A fusion coefficient matrix is ​​generated using modality-aware weights, and matrix multiplication is used to fuse the multi-scale features. The channel dimensions are then calibrated through convolutional layers to output dual-modality enhanced features.

[0112] In some implementations, refer to Figure 4 S25. Based on the generated proprietary weights, the multi-scale features are fused to obtain dual-modal enhanced features that strengthen the core discriminative information, including:

[0113] S251. Establish the adaptation relationship between weights and multi-scale features. The specific weights are three-channel modal perception maps, corresponding to small, medium, and large receptive field features. In the visible light mode, the first channel has a dominant weight, and in the infrared mode, the third channel has a dominant weight. Specifically, this step is to clarify the one-to-one correspondence between the three-channel modal perception map and the small, medium, and large receptive field features. Based on the dual-modal characteristics, a weight distribution preference is set to ensure that the first channel has a dominant weight in the visible light mode and the third channel has a dominant weight in the infrared mode. This provides a clear adaptation rule for subsequent feature weighted fusion, allowing the weight distribution to accurately guide multi-scale features to strengthen the core discriminative information of the dual-modal system.

[0114] For example, a three-channel modal sensing map It features modality-specific weight distribution. For visible light modal input (M=0), the weight value of the first channel (corresponding to the small receptive field feature) is significantly higher than that of the second and third channels, enhancing the capture of texture details. For infrared modal input (M=1), the weight values ​​of the second and third channels (corresponding to the large receptive field feature) are dominant, enhancing the integration of contour structure.

[0115] S252. Multiply the three receptive field features element-wise with the corresponding channel weights to enhance the bimodal discrimination information. Specifically, multiply the small, medium, and large receptive field features element-wise with the corresponding channel weights of the three-channel modal perception map. By adjusting the weights, the core bimodal discrimination information is enhanced and irrelevant information is suppressed. Thus, the modality-specific weights are used to accurately screen and enhance multi-scale features, allowing visible light features to highlight texture details and infrared features to highlight contour structures.

[0116] For example, the small receptive field features output by branch 1 are compared with... The first channel is multiplied element-wise, and the receptive field characteristics output by branch 2 are... The second channel performs element-wise multiplication, and the large receptive field characteristic of the branch 3 output is... The third channel is multiplied element-wise to obtain the corresponding weighted receptive field features. After weighting, the feature response of the texture detail area in the small receptive field feature of the visible light mode is enhanced, the feature response of the contour structure area in the large receptive field feature of the infrared mode is enhanced, and the feature response of the irrelevant noise area is suppressed.

[0117] S253. The weighted three-way features are summed element-wise to output enhanced features adapted to bimodal characteristics. Specifically, the small, medium, and large receptive field features, after element-wise weighting, are summed element-wise to integrate modality-specific multi-scale enhancement features and output enhanced features adapted to bimodal characteristics. This integrates enhancement features of different scales to form high-quality enhanced features that combine core modality discrimination information with multi-scale complementary information, laying the foundation for subsequent modality mutual calibration and bidirectional optimization.

[0118] For example, the three-way weighted feature maps obtained in step S252 are subjected to element-wise summation to form a single-way feature map. The fusion formula is as follows:

[0119]

[0120] in The basic features output in step S210; The weight of the i-th channel of the three-channel modal perception map generated in step S222 corresponds to the weight of the i-th receptive field. Indicates the expansion rate The 3×3 dilated convolution operation is used to extract the i-th receptive field feature; The final output is the fused feature. Time indicates visible light enhancement characteristics, Time indicates infrared enhancement features; This indicates element-wise multiplication. This indicates element-wise summation.

[0121] S26. Optimized features with cross-modal consistency and discriminativeness are obtained through modal mutual calibration denoising and bidirectional feature optimization. Specifically, mutual calibration denoising is first completed through a bimodal bidirectional mutual guidance mechanism. Then, the calibrated features are decoupled into modal-specific features and cross-modal shared features. A bidirectional interactive closed loop is constructed, in which shared features purify specific features and specific features feed back into shared features. Finally, the two types of features are adaptively fused according to modal requirements to output optimized features. This solves the problem of modal-specific information degradation caused by unidirectional feature interaction, strengthens cross-modal semantic alignment, retains unique bimodal discriminative details, and improves the comprehensive feature representation capability.

[0122] For example, contrastive learning is used to constrain bimodal feature mutual calibration. Features are decoupled through orthogonal decomposition, and bidirectional interaction is achieved using cross-attention. Fusion weights are generated based on a multilayer perceptron to output optimized features. Modality calibration is completed through lightweight convolution, features are decoupled through channel splitting, bidirectional enhancement is achieved by feature addition, and the two types of features are fused at a fixed ratio to output optimized features adapted to bimodality. Generative adversarial networks are used to achieve modality mutual calibration and denoising. Features are decoupled through an autoencoder, a recurrent feedback mechanism is constructed to complete bidirectional optimization, and a gated fusion strategy is used to output the final optimized features.

[0123] In some implementations, refer to Figure 5 S26. Optimized features with cross-modal consistency and discriminativeness obtained through modal cross-calibration denoising and bidirectional feature optimization include:

[0124] S261. Mutual calibration and denoising are performed using visible light features and infrared features as mutual references. Specifically, a target-oriented bimodal bidirectional mutual guidance calibration mechanism is adopted, with the inherent defects of the bimodality as the repair target. Through customized feature extraction and directional attention generation, precise noise suppression and complementary feature enhancement between modes are achieved. After calibration, the features retain the core discrimination information of the original mode through residual connection. Thus, the rich texture of the visible light features provides a noise suppression reference for the infrared features. At the same time, the illumination invariance of the infrared features assists the visible light features in overcoming the influence of illumination fluctuations, achieving preliminary feature alignment and complementarity.

[0125] For example, a multi-head self-attention mechanism is used to extract bimodal global features and generate mutually guided attention weights. Visible light global features are used to suppress infrared noise, and infrared global features are used to filter visible light interference. The calibration features are then output via residual connections. Modality-adaptive gated convolution is used to dynamically adjust the convolution kernel parameters to adapt to the bimodal characteristics. The feature flow is controlled by the gate unit to achieve noise suppression and preservation of core features, thus completing bimodal mutual calibration. Based on mutual information, the correlation between bimodal features is calculated to generate a modality-adaptive weight matrix. Mutual guidance calibration of features is achieved through matrix multiplication, preserving core discriminative information while removing invalid noise.

[0126] In some implementations, S261, performing mutual calibration and noise reduction using visible light features and infrared features as mutual references, includes:

[0127] Fine-grained texture information is extracted from visible light features in a targeted manner to generate specific attention weights. These weights are then used to filter out accurate texture information that matches infrared features, assisting in the calibration of infrared features. This process suppresses thermal noise in infrared features while preserving the core contour characteristics of the infrared mode. Specifically, fine-grained texture information is extracted from visible light enhancement features using a custom texture extraction operator. After mapping with a learnable projection matrix, specific attention weights are generated. These weights are then used to filter the fine-grained texture information extracted by TE (Technical Engineering) to obtain accurate texture references that strongly match infrared features. After adapting this accurate texture reference to the dimensions of the infrared features, a residual connection is made with the original infrared enhancement features. This process uses the accurate texture information from visible light to help filter thermal noise in infrared features, while the residual connection preserves the unique core contour structure information of the infrared mode.

[0128] For example, using a custom texture extraction operator Fine-grained texture information is extracted from visible light features to generate targeted attention weights, achieving precise suppression of thermal noise in infrared features while preserving the core contour characteristics of the infrared mode. The calculation formula is as follows:

[0129]

[0130] Among them, the input visible light enhancement features With infrared enhancement features All dimensions are ; As a learnable projection matrix, the former learns an effective representation of visible light texture features, while the latter learns an effective representation of infrared features, enabling it to more accurately find visible light texture information that matches itself. As a texture extraction operator, it is designed to address the characteristics of rich visible light textures that are easily affected by lighting. It consists of a 1×1 convolutional layer and a 3×3 edge detection kernel connected in series. The 1×1 convolutional layer is used to calibrate channel redundancy, retaining only the effective channels related to texture. The parameters of the 3×3 edge detection kernel are [[-1,-1,-1], [-1,8,-1], [-1,-1,-1]]. The core basis for setting these parameters is that the weight of the kernel center is set to 8, and the weights of the 8 surrounding positions are set to -1, forming a center enhancement plus a surrounding suppression mode. This results in higher detection sensitivity for fine-grained texture edges (such as clothing textures and human body contour details), which is suitable for the texture extraction requirements of visible light modes. The visible light focuses on the infrared direction, and its value increases significantly in the contour edge region of the infrared feature and decreases significantly in the thermal noise region, thus achieving accurate noise reduction.

[0131] It should be noted that the parameters of the above-mentioned 3×3 edge detection kernel are only example settings. This invention does not impose any special restrictions on specific parameters. As long as the kernel meets the characteristics of being sensitive to fine-grained textures and suppressing low-frequency light interference (such as adjusting the center weight to 7 and the surrounding weight to -0.8, etc.), it is within the protection scope of this invention.

[0132] The method involves extracting illumination-invariant contour information from infrared features in a targeted manner, generating a dedicated attention weight, and using this weight as a reference to filter illumination interference in visible light features while preserving visible light modal texture details. Specifically, a custom edge-preserving operator is used to extract illumination-invariant contour information from infrared enhancement features in a targeted manner. After mapping with a learnable projection matrix, a dedicated attention weight is generated and interacts with the visible light enhancement features element-wise to filter illumination interference while preserving visible light texture details. By using the illumination-invariant contour characteristics of infrared features as a reference, the method accurately overcomes illumination fluctuation interference in visible light features without destroying the unique texture information of the visible light modality.

[0133] For example, by using a custom edge-preserving operator The illumination-invariant contour edge information is extracted from infrared features to generate targeted attention weights, filtering out illumination interference from visible light features while preserving visible light texture details. The calculation formula is as follows:

[0134]

[0135] in, The former is a learnable projection matrix, which is used to learn an effective representation of infrared contour features or an effective representation of visible light features, to help it accurately match the infrared contour and thus filter out its own illumination interference area. For edge preservation operators, which are designed for infrared features with clear contours but thermal noise, a 3×3 Gaussian smoothing kernel and a gradient enhancement layer are connected in series. The 3×3 Gaussian smoothing kernel (general specification) is used to specifically suppress infrared thermal noise, while avoiding blurring of the core contour due to its moderate smoothing range. The gradient enhancement layer is built based on conventional edge detection operators and is used to enhance contour details that may be weakened after Gaussian smoothing. The infrared-to-visible light directional attention weight is significantly increased in the texture discrimination region of visible light features and significantly decreased in the light interference region, thus achieving accurate anti-interference;

[0136] It should be noted that the above-mentioned 3×3 Gaussian smoothing kernel and gradient enhancement layer based on conventional edge detection operator are only exemplary settings. This invention does not impose any special restrictions on the specific component specifications and types. As long as the combination meets the core characteristics of sufficient thermal noise suppression and complete contour preservation (such as adjusting the size of the Gaussian smoothing kernel to 5×5 or replacing the gradient enhancement layer with the Prewitt operator, etc.), it is within the protection scope of this invention.

[0137] The calibrated bimodal features are residually joined to retain the core discriminative information inherent in the original modality, thus avoiding the loss of key features during the calibration process. Specifically, during bimodal mutual calibration, the original enhanced features and the weighted reference features are residually joined. By superimposing the features, the core discriminative information inherent in the original modality is retained, thereby avoiding the loss of core features caused by excessive noise suppression during the calibration process. This ensures that the calibrated features are both pure and possess the key discriminative attributes of the original modality.

[0138] For example, in the feature calibration formula, residual connections are constructed by directly superimposing the original enhanced features and the weighted reference features:

[0139]

[0140]

[0141] This residual connection mechanism ensures that the calibration process only suppresses noisy regions without altering the original modal core discrimination information, such as visible light texture details and infrared contour structure, thus avoiding the loss of key features during the denoising process.

[0142] S262. The calibrated bimodal features are decomposed into modality-specific detail features and cross-modal general semantic features. Specifically, through learnable projection matrices and convolution operations, the calibrated bimodal features are decomposed into modality-specific detail features and cross-modal general semantic features. Modality-specific discrimination information is retained and global common semantics are mined to provide accurate input for subsequent bidirectional optimization. This achieves the separation of modality-specific details and cross-modal general semantics, which avoids feature homogenization and provides a clear interaction object for the bidirectional interaction loop.

[0143] For example, enhancing features , The features are decoupled into modality-specific features and cross-modality shared features, as shown in the following formula:

[0144]

[0145] Among these, It is an enhancement feature The result after modal mutual calibration is that infrared feature denoising is guided by visible light texture and visible light anti-interference is guided by infrared contour, which repairs the inherent defects of dual modes and retains the core discrimination information of the original enhanced features through residual connection. As a learnable projection matrix, it automatically focuses on the modality-specific dimensions (texture channel in visible light, contour channel in infrared) of the calibrated features by learning, filtering cross-modal redundant information and avoiding homogenization from the source. It is a 3×3 lightweight convolution (the number of output channels is the same as the number of input channels, C=512, with no pooling layers), adapted to a fine-grained spatial size of 8×4—it both calibrates channel redundancy and preserves spatial details, such as clothing textures and limb contours, to ensure the discriminativeness of specific features; when m is vis, the output is a visible light modality-specific feature, and when m is ir, the output is an infrared modality-specific feature.

[0146] Based on the noise mutual exclusion of calibrated features, modality-specific noise is suppressed, and bimodal information is fused to mine general semantics as an interaction benchmark. The formula is as follows:

[0147]

[0148] in It uses a 3×3 convolution (the number of output channels is half the number of input channels, i.e., 256), which filters fine-grained modal details and focuses on global general semantics such as pose by reducing the number of channels. The global average pooling operation further aggregates global information, ultimately yielding a general semantic feature with a dimension of 256.

[0149] S263. Based on general semantic features, identify and remove invalid noise in modality-specific features while retaining effective discriminative details. Specifically, this step maps modality-specific features and cross-modality general semantic features to the same feature space. Invalid noise is identified by using the global common information of general semantic features as a reference. Noise is suppressed and effective details are enhanced through a channel-by-channel multiplication semantic calibration method. This ensures that the purification process does not destroy the core discriminative information. The unique features are purified using the global perspective of general semantic features, thereby improving the purity and discriminativeness of modality-specific features.

[0150] For example, contrastive learning is used to constrain the similarity between modality-specific features and general semantic features. By mining hard samples, effective detail responses are enhanced, and feature activation corresponding to noise is suppressed, thereby achieving purification of unique features. A gating attention mechanism is used to generate gating weights from general semantic features. The gating unit filters effective information in the unique features and removes invalid noise, outputting the purified unique features. Based on Bayesian inference, the general semantic features are used as prior information to calculate the posterior probability of each element in the unique features, retaining high-probability effective details and removing low-probability noise.

[0151] In some implementations, S263, based on general semantic features, identifies and removes invalid noise from modality-specific features while retaining valid discrimination details, including:

[0152] Modality-specific features and cross-modality general semantic features are mapped to the same feature space using a learnable projection matrix to establish semantic association. Specifically, this step introduces a learnable projection matrix to linearly map the modality-specific features and cross-modality general semantic features respectively, transforming the two types of features into the same feature space with consistent dimensions, establishing semantic association between the two, thereby eliminating the spatial dimensional differences between the two types of features, providing a prerequisite for subsequent noise identification based on general semantics, and ensuring the effectiveness of semantic comparison.

[0153] Using the global common information of general semantic features as a reference, invalid noise unrelated to identity discrimination in modality-specific features is identified; using the global common information of general semantic features in the same feature space as a reference benchmark, by calculating the semantic similarity between modality-specific features and general semantic features, invalid noise in modality-specific features that deviates significantly from the global common information and is unrelated to identity discrimination is identified, thereby accurately locating the noise region in modality-specific features, providing a clear target for subsequent targeted denoising, and avoiding the accidental deletion of effective discrimination details.

[0154] By employing a channel-wise multiplication semantic calibration method, the feature response corresponding to noise is suppressed, while the effective discrimination details are enhanced. This step generates semantic calibration weights based on general semantic features, and performs a channel-wise multiplication operation on these weights with modality-specific features to suppress the feature response in noisy regions, while enhancing the feature activation of effective discrimination details. This achieves targeted optimization of modality-specific features, thereby removing invalid noise and improving the discrimination power of effective details, thus achieving the effect of noise reduction and truth retention.

[0155] For example, cross-modal shared features are used as a global semantic benchmark to clean and enhance modality-specific features. The principle is to utilize the common information encoded in the shared features to identify and suppress modal noise (such as visible light interference and infrared thermal noise) in specific features that is irrelevant to identity discrimination, while preserving and enhancing its effective discriminative details. This process is achieved through the following formula:

[0156]

[0157] in, The general semantic guidance matrix maps the global common semantics encoded by shared features to the channel dimension space of modality-specific features; This indicates channel-by-channel multiplication. This is not a simple feature weighting, but rather a semantic calibration that removes false positives from specific features using a global perspective of shared features.

[0158] S264. The purified dual-modal unique detail features are fused and supplemented into the general semantic features to enrich their fine-grained discrimination information. Specifically, this step involves fusing the purified visible light unique detail features and infrared unique detail features, and then combining the fused detail features with the cross-modal general semantic features to complement each other. This uses fine-grained information to supplement the lack of detail in the general semantic features, thereby solving the problem of insufficient fine-grained detail in the general semantic features. This allows the general semantic features to absorb dual-modal unique details while maintaining global consistency, thus improving the fine-grained discrimination ability of the features.

[0159] For example, an attention fusion mechanism is adopted to calculate the attention weights of the unique features of the two modalities. After weighted fusion, the features are mapped to the general semantic feature space through a 1×1 convolution to achieve detail supplementation. The unique features after fusion are added to the general semantic features using residual connections. While preserving the global information of the general semantics, fine-grained details are supplemented, and the optimized general features are output. Based on the modality adaptive fusion strategy, the fusion ratio of the unique features of the two modalities is dynamically adjusted, and then supplemented to the general semantic features after calibration through a convolutional layer to adapt to the detail requirements of different modalities.

[0160] In some implementations, S264, fusing the purified dual-modal unique detail features and supplementing them into the general semantic features to enrich their fine-grained discriminative information includes:

[0161] The purified visible light-specific detail features and infrared-specific detail features are fused to integrate the differentiated fine-grained information of the two modalities. Specifically, the purified visible light-specific detail features and infrared-specific detail features are element-wise added to integrate the differentiated fine-grained information of the two modalities, forming a unified set of detail features. This aggregates the exclusive fine-grained information of each modality, providing comprehensive detail support for the subsequent supplementation of general semantic features.

[0162] For example, inputting purified visible light unique detail features Unique Infrared Detail Features Both types of features retain the core discriminative details of their respective modalities;

[0163] Perform element-wise addition on the two types of features to integrate the differential fine-grained information of the two modalities and obtain the fused detail features. .

[0164] The fused detailed features are complementary to the cross-modal general semantic features, supplementing the lack of details in the general semantics with fine-grained information. Specifically, the fused detailed features are mapped to the same channel dimension as the general semantic features through 3×3 convolution, and then added to the cross-modal general semantic features execution elements to supplement the lack of details in the general semantics with fine-grained information. This allows the general semantic features to absorb bimodal specific fine-grained information, making up for their deficiencies in detail expression and improving the fine-grained discrimination ability of the features.

[0165] For example, to address the inherent lack of fine-grained details in shared features, purified bimodal specific features are used. and The fusion process is performed, and the resulting details are used to supplement and enhance the shared features, as shown in the following formula:

[0166]

[0167] Through this feedback process, the shared features, while maintaining their global consistency, absorb high-frequency detail information from the bimodal source, thereby achieving an organic unity between semantic information and texture details.

[0168] S265. Generate fusion coefficients based on modal requirements, fuse the optimized general features and unique features according to the coefficients, and output the optimized features. Specifically, generate fusion coefficients based on the purified modal-specific features and accurately match the modal characteristics. Use these coefficients to perform weighted fusion of the optimized general semantic features and the purified unique detail features, and output optimized features that have both cross-modal consistency and modal discriminability. By formulating differentiated fusion strategies for the differences in bimodal characteristics, the consistency of general semantics and the discriminability of unique details are balanced to ensure that the fusion results are adapted to modal requirements.

[0169] For example, based on the purified specific features, fusion coefficients that precisely match the modal characteristics are generated. The formula is as follows:

[0170]

[0171]

[0172] in The goal is to create a learnable projection matrix that adapts to the differences in bimodal characteristics. For the Sigmoid activation function, the modal fit of the fusion coefficients has a clear technical meaning:

[0173] Preferring smaller values: Since the purified visible light-specific features already contain sufficient discriminative texture details, the fusion strategy introduces shared features with low weights, prioritizes trusting and retaining its own fine-grained texture, and avoids global semantic dilution of details;

[0174] Preferring larger values: Although infrared specific features have complete outlines, they lack internal details. The fusion strategy introduces shared features with high weights to supplement cross-modal consistent global semantics and enhance the discriminative power of infrared features.

[0175] Using the aforementioned modality-specific fusion coefficients with clear physical meaning, the two types of features are synthesized:

[0176]

[0177]

[0178] in, It is a 1×1 convolution (the number of output channels equals the number of input channels). For broadcast operations, the shared features obtained from global pooling are added to the spatial size of specific features. ; as well as The final output is an optimized feature that combines cross-modal consistency and modal discriminativeness.

[0179] S30. Calculate the feature similarity between the query image and the candidate images, and output the pedestrian identities in descending order of similarity. Specifically, perform standardized preprocessing on the query image and the candidate images, input the optimized features into the trained and converged model, quantify the feature matching degree using cosine similarity, sort them in descending order of similarity, and output the pedestrian identity recognition results.

[0180] This application extracts pedestrian target regions from dual-modal images of surveillance scenes and dynamically adjusts the feature extraction scale based on modal characteristics. It adapts a small-scale receptive field for visible light and a large-scale receptive field for infrared, generating dual-modal enhanced features. By filtering core pedestrian objects, eliminating background noise, and standardizing the input format, it solves the problem of irrelevant background interference feature extraction in original surveillance images, laying a clean data foundation for cross-modal matching. Simultaneously, through a dynamically adapted receptive field design, it overcomes the limitation of traditional fixed convolutional kernels in accommodating dual-modal characteristics, solving the problem of insufficient feature extraction granularity and modal adaptability, and avoiding visible light texture loss or infrared contour fragmentation. Furthermore, the enhanced features generated by this design simultaneously cover dual-modal information, narrowing the modal gap caused by differences in imaging mechanisms, solving the problems of difficult cross-modal feature alignment and weak discriminative power, ultimately achieving the technical effect of improving the accuracy of cross-modal pedestrian identity matching.

[0181] In some implementations, before S10, acquiring visible light and infrared images of the monitored scene and extracting pedestrian target areas from the images, the method further includes:

[0182] S01. The model is trained under constraints using the overall loss function, which consists of modality-aware weight-guided loss, bidirectional interaction collaboration loss, hard sample triplet loss, and identity classification loss. Specifically, to ensure the modality-aware map... The weight distribution is consistent with the physical characteristics of different modalities, and the learning process is constrained. This invention constructs a modality-aware weight-guided loss function. This loss function provides explicit supervision for the learning of modality-aware maps by introducing a pre-defined modality-specific target weight template.

[0183] Specifically, the loss function is configured to calculate the mean absolute error between the generated modality-aware map and the target template. The calculation formula is as follows:

[0184]

[0185] Where B represents the number of samples in the training batch; For the channel index of the modality-aware map; y represents the predicted weight value of the k-th channel for the i-th sample at spatial location (h, w); i Represents the true modal label (y) of the i-th sample. i =0 represents visible light, y i =1 indicates infrared); With modal label y i The corresponding target weight template.

[0186] The target weight template is set based on the prior physical property that visible light images are rich in detailed textures while infrared images emphasize overall contours. Its core design rule is to assign differentiated channel weight preferences to different modalities in order to guide the model to extract the most discriminative modal features.

[0187] Specifically:

[0188] For the visible light mode, its target template It should satisfy the following condition: the weight of the first channel is greater than the weight of the subsequent channels (i.e., and This guides the network to strengthen its small receptive field and focus on local textures.

[0189] For the infrared mode, its target template It should satisfy the following condition: the weight of the last channel is greater than the weight of the preceding channels (i.e., and This guides the network to enhance its large sensory field and integrate the overall profile.

[0190] As a specific embodiment of the present invention, the values ​​of the target weight template are as follows:

[0191] Visible light target weights The value is set to [0.6, 0.3, 0.1]. This value strictly follows the above rules and significantly strengthens the weight of the first channel.

[0192] Infrared target weight The value is set to [0.1, 0.3, 0.6]. This value strictly follows the above rules and significantly strengthens the weight of the third channel.

[0193] It should be noted that the specific values ​​above are merely an example of how to implement the rules. Those skilled in the art will understand that any combination of values ​​that conforms to the aforementioned channel weight preference relationship (e.g.) =[0.5,0.25,0.25] or =[0.15,0.25,0.6]) are all based on the same inventive principle and should be considered as equivalent substitutions, falling within the protection scope of this invention.

[0194] To ensure that the closed-loop interaction process produces substantial semantic calibration and detail complementarity effects, this invention introduces a novel bidirectional interactive collaborative loss in the example. By quantifying the changes in feature relationships before and after the interaction, it directly optimizes the two collaborative stages of sharing guidance for specific purification and specific feedback sharing enhancement.

[0195] Shared feature calibration gain constraint on specific features: This constraint ensures that the purified specific features... It needs to simultaneously satisfy: A. Compared to the original specific features Closer to shared features A. Global semantics (calibrating modal bias). B. Preserves richer modal details than the original specific features (avoiding detail loss due to calibration). Calibration loss The definition is as follows:

[0196]

[0197] in Represents cosine similarity; the first term is passed through a boundary... The hinge loss forces that the similarity between the purified features and the shared features must be significantly higher than that before purification; the latter term ensures that the purification process is "denoising" rather than "destruction" by constraining the high similarity of specific features before and after purification, effectively preserving the unique discriminative details of the modality. To balance the weighting coefficients of the two constraints.

[0198] Specific feature feedback gain constraint on shared features: This constraint ensures that the enhanced shared features... It can effectively absorb fine-grained information from specific features of the dual-modality model. Feedback loss. Defined as:

[0199]

[0200] This loss is achieved through marginal The hinge loss forces that the similarity between the shared features after the feedback and the specific feature details must be significantly higher than before the feedback, thereby quantifying and driving the detail supplementation effect.

[0201] The overall collaborative loss is the sum of the above two items, which together constitute the following constraint on the complete interactive closed loop:

[0202]

[0203] Step S01 uses minimizing the overall loss function as the optimization objective and trains the cross-modal pedestrian re-identification model constructed in the above steps using a public dataset; iteratively updates the training model parameters through backpropagation until the model converges, enabling the model to learn the semantic alignment ability of cross-modal features and the ability to distinguish pedestrian identities.

[0204] In a specific instance, the overall loss function can be expressed as:

[0205]

[0206] in , These are the innovative constraint losses proposed in this invention, corresponding to the modal consistency constraint of feature calibration and the bidirectional interactive feature complementarity constraint, which are the core losses of this invention for cross-modal feature alignment and complementarity optimization design; The balancing coefficient is used to adjust the contribution of the two proposed constraint losses to the overall optimization objective; By combining the hard sample triplet loss and the identity classification loss, the two work together to enhance the identity discrimination and inter-class differentiation capabilities of the features, thus complementing and optimizing the innovative constraint loss of this invention.

[0207] It should be noted that the hard sample triplet loss and identity classification loss mentioned above are merely exemplary choices. Their core function is to assist in improving the identity discriminative power of features. Those skilled in the art can replace them with other known losses with equivalent functions (such as contrast loss, center loss, etc.) according to the actual scenario, as long as they are consistent with the methods proposed in this invention. , Synergistic effects are all within the scope of protection of this invention.

[0208] In some embodiments, in practical application scenarios, for the acquired visible light modal query image and infrared modal candidate image library, cross-modal pedestrian re-identification model trained and converged by S01 is used to perform cross-modal pedestrian identification and matching.

[0209] The specific reasoning process includes:

[0210] Perform standardized preprocessing on the query image and candidate images in the same way as during the training phase, such as size normalization and pixel normalization;

[0211] The preprocessed image is input into the trained cross-modal person re-identification model, referring to... Figure 6 The model includes a basic feature extraction network, a modality-aware dynamic receptive field module, and a dual-modality closed-loop interaction module. Pedestrian images in visible light and infrared modalities are input into two independent branches of the basic feature extraction network. The basic feature extraction network performs convolution, batch normalization, activation, and pooling operations on the dual-modality images, outputting initial basic features. These initial basic features are input to the modality-aware dynamic receptive field module, where they are adjusted by the dynamic receptive field to generate enhanced features. This process is guided by a modality-aware weight-guided loss. Constraints; Enhanced feature input to the dual-modal closed-loop interactive module, which outputs the final optimized features after multi-step optimization. This process is subject to bidirectional interactive collaborative loss. Constraints; the final optimized features are used to calculate the similarity of pedestrian identity matching, completing the cross-modal pedestrian re-identification task.

[0212] Figure 6 The gray square on the right side of the middle section refers to the final optimized features, BN refers to the Batch Normalization layer, and the slanted cylinder refers to the fully connected layer, whose function is to map high-dimensional feature vectors into low-dimensional feature vectors used for classification / measurement. Identity loss (ID Loss) is a commonly used loss in classification tasks. Its purpose is to enable the model to learn to "classify the features of the same person into the correct identity category". The Triplet Loss, also known as the hard sample triplet loss, is a commonly used loss in metric learning. Its purpose is to make the features of different modalities of the same person closer together and the features of different people farther apart, thereby enhancing the discriminative power of cross-modal matching.

[0213] Therefore, the process on the right is actually part of the joint loss function calculation. It refers to the final optimized features output by the dual-modal closed-loop interaction module. Through standardization, feature mapping and loss calculation, it provides the model with basic identity classification and feature measurement constraints.

[0214] Reference Figure 7 , Figure 7This is a schematic diagram of the core architecture of the modality-aware dynamic receptive field module. It mainly presents the process of multi-receptive field feature extraction and modality-aware weight generation. Dconv2 refers to a 3×3 dilated convolution with an dilation rate of r=2, which is used to extract small receptive field features. Correspondingly, Dconv4 refers to a 3×3 dilated convolution with an dilation rate of r=4 and Dconv6 refers to a 3×3 dilated convolution with an dilation rate of r=6, which are used to extract medium and large receptive field features, respectively. Refers to fully connected layer 1. The fully connected layer 2, the three-channel modal sensing map contains weighted coefficients for large, medium and small receptive fields.

[0215] The overall process can be summarized as follows:

[0216] Input the basic feature extraction network and output the bimodal basic features;

[0217] To address the differentiated requirements of receptive field granularity in dual-modality systems, features at different scales are extracted using three-way parallel dilated convolution. The basic features are input into Dconv2, Dconv4, and Dconv6 respectively, outputting three types of features: small receptive field texture features (Dconv2), medium receptive field balance features (Dconv4), and large receptive field contour features (Dconv6). The channel-level global mean and standard deviation of the basic features are calculated and concatenated into a 1024-dimensional statistical feature vector. This statistical feature vector is then input into... The nonlinearity is enhanced by ReLU activation, and then the input is... Mapping to 2D modality scores, modality probability distributions are obtained through Softmax, and modality markers M are finally determined through argmax. The modality markers M are converted into continuous vectors through an embedding layer, concatenated with the global pooling vectors of the basic features, and broadcast expanded to 8×4 resolution. A 1×1 convolution operation is used to compress the channel dimension of the fused features to 3. Three-channel weights are generated through Sigmoid activation normalization.

[0218] Then, the output of Dconv2 is multiplied element-wise with the first channel of weight, Dconv4 is multiplied with the second channel of weight, and Dconv6 is multiplied with the third channel of weight; the three weighted features are then summed element-wise to obtain the final enhanced features.

[0219] Reference Figure 8 , Figure 8 This is a schematic diagram of the architecture of a dual-modal (visible light-infrared) closed-loop interaction module, the purpose of which is to achieve bidirectional optimization of cross-modal shared features and modality-specific features. Figure 8 middle , The input visible light and infrared features are enhanced features of the initial features; the small rectangular array refers to the convolutional layer; ⊕ indicates the element-wise addition operation. Element-wise multiplication; Refers to the calibrated single-mode characteristics; Modal-specific characteristics; This refers to cross-modal shared features; the shared feature guidance matrix is ​​used to learn optimized weights for shared features, highlighting semantic information that is more important for identity matching; Refers to the specific features of the optimized mode; This refers to a gated layer that fuses modal-specific features and shared features using modality-specific feature coefficients as weights to obtain the final visible light features. and infrared features Finally, low-dimensional matching features that can be directly used for cross-modal pedestrian identity similarity calculation are obtained through pooling and normalization layers.

[0220] This process can be represented as:

[0221] The input visible light and infrared features are each split into their own channels, transformed by a convolutional layer, and then cross-multiplied. The features are initially adjusted using information from the other modality, and then the residuals are added to the original modality features to obtain the calibrated single-modality features. These calibrated single-modality features are decoupled into modality-specific features and cross-modality shared features. The modality-specific features are then processed by a general semantic guidance matrix. By learning and optimizing weights, we obtain optimized modality-specific features that both conform to the global general semantics and retain the fine-grained details specific to the modality. .

[0222] Subsequently, the optimized modality-specific features from the dual-modality model are fused, and the spatial local details contained therein are extracted through a 3×3 convolution operation. Then, pooling is performed to compress the features to the same channel and spatial dimensions as the original shared features. Finally, these features are fused with the original shared features to obtain the optimized shared features. .

[0223] Optimized specific features through The gated layer uses modality-specific fusion coefficients as weights to perform weighted fusion of global semantic information of shared features and modality-specific fine-grained detail information of specific features, ultimately obtaining the final optimized features for visible light and infrared light, respectively. Finally, a pooling layer compresses the high-dimensional spatial features into low-dimensional feature vectors, and then a normalization layer performs numerical standardization, resulting in low-dimensional matching features that can be directly used for cross-modal pedestrian identity similarity calculation. as well as This is used for subsequent cross-modal pedestrian matching.

[0224] Calculate the cosine similarity between the feature vector of the query image and all feature vectors in the candidate image library, and sort them in descending order according to the similarity score;

[0225] Output the identities of the matched pedestrians based on the sorting results.

[0226] It should be understood that the model trained by this invention can be deployed on various electronic devices to achieve real-time recognition. These electronic devices include, but are not limited to: front-end cameras in surveillance systems (such as dual-lens devices with visible light and infrared capabilities), edge computing nodes, back-end servers, mobile terminals (such as mobile phones and tablets), or in-vehicle intelligent systems. The model training process can be completed offline in the cloud or on a high-performance server, and then the model parameters can be deployed to the aforementioned devices to achieve efficient forward inference.

[0227] To verify the effectiveness of the present invention, in a specific embodiment, we implemented the model based on the PyTorch deep learning framework and conducted comprehensive testing on three public benchmark datasets in the field of visible light-infrared pedestrian re-identification: SYSU-MM01, RegDB, and LLCM.

[0228] In the implementation case, the initial learning rate was 2.5 × 10⁻⁶. -4 The AdamW optimizer was used, with the learning rate decreasing to 0.1 times its original value at 60th and 120th epochs, for a total of 160 training epochs; in the hinge loss with margins, Set to 0.1; hyperparameter settings are as follows: =0.3, =0.6, It is 1.0.

[0229] It should be noted that the specific values ​​of the above parameters are a preferred embodiment of the present invention, and those skilled in the art can make appropriate adjustments based on them, and such adjustments should be considered to be within the protection scope of the present invention.

[0230] To objectively evaluate the performance of this invention, the embodiments were compared with several current mainstream methods. The table below shows some of the comparison results on the SYSU-MM01 and RegDB datasets, using Rank-1 recognition rate and mean accuracy as evaluation metrics.

[0231] Table 1. Performance comparison (%) on the SYSU-MM01 and RegDB datasets

[0232]

[0233] SYSU-MM01 is a cross-modal pedestrian retrieval dataset released by Sun Yat-sen University. It contains a large number of visible light and infrared pedestrian image pairs for indoor and outdoor scenes and is one of the benchmark datasets in this field. RegDB is a cross-modal pedestrian retrieval dataset built by Yonsei University in South Korea. It contains pedestrian images in both visible light and infrared modes and is used to evaluate the performance of cross-modal retrieval algorithms.

[0234] Rank-X refers to the proportion of correct matches among the top X results in the search results. The higher the value, the stronger the model's ability to find the correct target within the corresponding ranking.

[0235] mAP stands for Mean Average Precision, a classic metric in information retrieval that comprehensively considers the accuracy and recall of search results. A higher value indicates that the model's search results are both accurate and reasonably ranked.

[0236] AGW / CAJ / CMT refers to the mainstream algorithms already existing in this field, representing different cross-modal feature alignment or interaction strategies.

[0237] Ours refers to the method proposed in this application.

[0238] As shown in Table 1, the experimental results demonstrate that the method of this invention significantly outperforms the existing methods listed in both datasets in terms of Rank-1 and mAP. Compared to the domain benchmark CMT method, the Rank-1 value is improved by 10 percentage points in the full search mode, and other indicators are also significantly improved. This fully demonstrates the effectiveness and advancement of this invention in cross-modal person re-identification tasks.

[0239] To further verify the robustness and generalization ability of this invention in complex scenarios closely resembling real-world applications, comparative experiments were conducted on the more complex and challenging LLCM dataset. This dataset contains more diverse lighting variations, occlusions, and background interference, placing higher demands on the model's discriminative ability. Some comparative results are shown in Table 2:

[0240] Table 2. Partial experimental results (%) on the LLCM dataset

[0241]

[0242] Among them, LLCM stands for Large-scale Cross-Modality, which contains large-scale visible light and infrared modal pedestrian images and is specifically designed to evaluate the cross-modal retrieval performance of algorithms in large-scale scenes.

[0243] DEEN refers to an existing algorithm in this field, short for Dual Enhanced Embedding Network. These algorithms typically improve the alignment and retrieval performance of cross-modal features through a dual-branch structure or feature enhancement strategies.

[0244] As shown in Table 2, the method provided in this invention also achieves optimal performance on the complex LLCM dataset. Compared with the best method, it improves the Rank-1 value by 7-14 percentage points and also improves the mAP value. This result strongly demonstrates that the modality-aware dynamic receptive field module and the bimodal closed-loop interaction module proposed in this invention can effectively cope with modal differences and background interference in complex scenes, and the generated image features have stronger discriminative power and robustness.

[0245] To further verify the technical effects of the present invention from a qualitative perspective, two additional types of verification experiments are added in conjunction with specific embodiments, forming a complete verification system of "quantitative + qualitative" with the aforementioned quantitative experiments, as detailed below:

[0246] (1) Visual verification of feature distribution

[0247] To verify the advantages of the proposed method in feature discrimination capability and cross-modal alignment, the t-SNE algorithm was used to project the high-dimensional features output by the model into a two-dimensional space for visualization analysis. The results are as follows: Figure 9 As shown in the figure, the circular markers correspond to infrared (IR) modal features, the triangular markers correspond to visible light (VIS) modal features, and different colors represent different pedestrian identities, so as to intuitively present the intra-class aggregation degree and inter-class separation degree of features.

[0248] Reference Figure 9 , Figure 9 In the figure, 'a' represents the initial feature distribution: infrared and visible light modal features are completely mixed, with no obvious clustering trend and dispersion within each class; Figure 9 In the figure, b represents the feature distribution of the baseline model: the features show a preliminary clustering trend, but there is a significant modal bias, that is, the bimodal features cluster separately, the cross-modal features of the same pedestrian are not effectively aligned, and the intra-class dispersion is still high. Figure 9 In this paper, c represents the feature distribution of the proposed method (without constraints): the feature distribution is significantly optimized compared to the baseline model, the inter-class separation trend is clear, and the bimodal features of pedestrians of the same class begin to aggregate initially; Figure 9 In this paper, d represents the characteristic distribution of the proposed method (with added constraints): compared to Figure 9 The infrared and visible light features of similar pedestrians in the c-type are highly concentrated, and the modal gap is further compressed.

[0249] (2) Qualitative evaluation of cross-modal retrieval results

[0250] To verify the effectiveness of the present invention in practical cross-modal retrieval tasks, Figure 10 The baseline models were shown respectively. Figure 10 a) Advanced methods IDKL ( Figure 10 (b) and the method proposed in this invention ( Figure 10 c) Typical qualitative retrieval results on the LLCM dataset. In each retrieval, green boxes mark correct matches (i.e., images belonging to the same identity as the query sample), and red boxes mark incorrect matches. The query sample is an infrared modal image, and the candidate library consists of visible light modal images. Compared with the baseline method and the IDKL method, the proposed method significantly improves the ranking accuracy of cross-modal retrieval, placing more correct matches at the top of the retrieval list, fully demonstrating its superiority in enhancing cross-modal feature alignment and discriminative capabilities.

[0251] Empirical verification shows that, on mainstream benchmark datasets such as SYSU-MM01, RegDB, and the more complex LLCM dataset, the present invention significantly outperforms existing mainstream methods such as AGW, CAJ, and CMT in terms of Rank-1 accuracy and mAP. Feature distribution visualization and cross-modal retrieval qualitative verification further confirm that the present invention can effectively improve the intra-class aggregation degree and inter-class separation degree of features, compress modal gaps, and has excellent technical advancement and robustness in complex scenarios.

[0252] Furthermore, this invention does not rely on a specific backbone network or pixel-level alignment data. The technical solution is complete and easy to integrate. The trained model can be flexibly deployed on various electronic devices such as dual-camera monitoring, smart access control, and vehicle recognition, adapting to actual industrial scenarios such as monitoring and security.

[0253] The visible light-infrared pedestrian re-identification method based on modal perception closed-loop interaction proposed in this invention can be implemented in various forms and is protected by relevant laws.

[0254] This invention can be implemented as a computer program product. The computer program product includes one or more computer-readable storage media storing computer-readable program instructions that, when executed by one or more processors, cause the processors to perform all or part of the process of the cross-modal person re-identification method as defined in the previous derivation steps, i.e., the training steps.

[0255] Computer-readable storage media can be tangible devices capable of retaining and storing instructions, such as, but not limited to, electronic storage devices (e.g., solid-state drives, random access memory (RAM), read-only memory (ROM), optical storage devices (e.g., CD-ROM), magnetic storage devices, or any suitable combination thereof.

[0256] Reference Figure 11The present invention can also be implemented as a visible light-infrared pedestrian re-identification system based on modal perception closed-loop optimization, characterized in that it includes:

[0257] Acquisition module 1 is used to acquire visible light and infrared images of the monitored scene and extract pedestrian target areas from the images;

[0258] Processing module 2 is used to convert the preprocessed image into initial features, enhance details and contour extraction based on the characteristics of visible light or infrared images, and then obtain optimized features with cross-modal consistency and discriminability through modal cross-calibration denoising and bidirectional feature optimization.

[0259] Output module 3 is used to calculate the feature similarity between the query image and the candidate images in the library, and output the pedestrian identities in order of similarity.

[0260] Those skilled in the art should understand that the core innovative logic of this invention, including but not limited to the modal perception dynamic receptive field mechanism, the bidirectional interaction enhancement mechanism, and the multi-loss joint optimization strategy, is implemented without relying on any specific programming language, computing framework, or hardware architecture.

[0261] At the software level, any suitable programming language such as Python or C++ can be used, and it can be implemented based on mainstream or dedicated deep learning frameworks such as PyTorch and TensorFlow.

[0262] At the hardware level, the training and inference processes of the method can be deployed on general-purpose computing devices (such as CPUs and GPUs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or integrated into edge computing devices, smart terminals, and cloud servers.

[0263] All software, hardware, and combined software and hardware implementation schemes based on the technical concept of this invention shall fall within the protection scope of this invention.

[0264] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

[0265] It should be noted that this application is not limited to the above-described embodiments. The above embodiments are merely examples, and any embodiments with the same structure and effect as the technical concept within the scope of this application are included in the technical scope of this application. Furthermore, various modifications that can be conceived by those skilled in the art to the embodiments, and other ways of constructing by combining some of the constituent elements of the embodiments, without departing from the spirit of this application, are also included in the scope of this application.

Claims

1. A visible light-infrared pedestrian re-identification method, characterized in that, include: Acquire visible light and infrared images from the monitored scene, and extract pedestrian target areas from the images; The preprocessed image is transformed into initial features. Details and contours are enhanced based on the characteristics of visible light or infrared images. Then, optimized features with cross-modal consistency and discriminativeness are obtained through modal cross-calibration denoising and bidirectional feature optimization. Calculate the feature similarity between the query image and the candidate images, and output the pedestrian identities in order of similarity. The process of converting the preprocessed image into initial features, enhancing details and contour extraction based on the characteristics of visible light or infrared images, and then obtaining optimized features with cross-modal consistency and discriminativeness through modal cross-calibration denoising and bidirectional feature optimization includes: The initial features are obtained through mapping; Calculate the channel-level global mean and standard deviation of the initial visible light characteristics, as well as the corresponding indices of the initial infrared characteristics, to clarify the differences in modal characteristics; Based on the quantization differences, a weight distribution adapted to the two modalities is generated; By using feature extraction paths with different dilation rates, small-scale texture features of visible light, large-scale contour features of infrared light, and mesoscale features that balance local and global characteristics are obtained respectively. Based on the weight distribution of the generated two-mode adaptation, the multi-scale features are fused to obtain dual-mode enhanced features that strengthen the core discriminative information; Optimized features with cross-modal consistency and discriminativeness are obtained through modal cross-calibration denoising and bidirectional feature optimization. The step of generating a weight distribution adapted to the two modalities based on quantization differences includes: The channel-level global mean and global standard deviation of the initial visible light features are concatenated in channel index order to form the visible light statistical feature vector; similarly, the channel-level global mean and global standard deviation of the initial infrared features are concatenated to form the infrared statistical feature vector. The statistical feature vector is subjected to high-dimensional reduction, nonlinear activation, probability normalization and modality index discrimination in sequence, and the discrete mode label is output. Global average pooling is performed on the initial features to obtain a global feature vector. The modality flag is mapped to a continuous embedding vector. The global feature vector and the embedding vector are concatenated and extended to the spatial size of the initial features. After activation normalization, a three-channel weight distribution map is generated.

2. The visible light-infrared pedestrian re-identification method according to claim 1, characterized in that, The process of fusing multi-scale features according to the weight distribution of the generated two-mode adaptation to obtain dual-mode enhanced features that strengthen core discriminative information includes: Establish the adaptation relationship between weights and multi-scale features. The weight distribution for adapting the two types of modes is a three-channel modal perception map, corresponding to small, medium and large receptive field features. Among them, the first channel has a dominant weight in the visible light mode and the third channel has a dominant weight in the infrared mode. The three receptive field features are multiplied element-wise with the corresponding channel weights to enhance the bimodal discrimination information; The weighted three features are summed element-wise to output enhanced features that adapt to the dual-modal characteristics.

3. The visible light-infrared pedestrian re-identification method according to claim 1, characterized in that, Optimized features with cross-modal consistency and discriminative power, obtained through modal cross-calibration denoising and bidirectional feature optimization, include: Visible light features and infrared features are used as mutual references for mutual calibration and noise reduction; The calibrated bimodal features are split into modality-specific detail features and cross-modal general semantic features. Based on general semantic features, identify and remove invalid noise in modality-specific features while retaining effective discrimination details; The refined dual-modal unique detailed features are fused and added to the general semantic features to enrich their fine-grained discriminative information; Generate fusion coefficients based on modal requirements, fuse the optimized general features and unique features according to the coefficients, and output the optimized features.

4. The visible light-infrared pedestrian re-identification method according to claim 3, characterized in that, The method of using visible light features and infrared features as mutual references for mutual calibration and noise reduction includes: Fine-grained texture information is extracted from visible light features in a targeted manner to generate targeted attention weights. These targeted attention weights are used as a reference to suppress thermal noise in infrared features while preserving the core contour characteristics of the infrared mode. The contour information of the infrared features is extracted in a targeted manner to maintain the illumination, and a dedicated attention weight is generated. The dedicated attention weight is used as a reference to filter the illumination interference in the visible light features, while preserving the texture details of the visible light modality. The calibrated bimodal features are connected via residuals to retain the core discriminative information inherent in the original mode, thus avoiding the loss of key features during the calibration process.

5. The visible light-infrared pedestrian re-identification method according to claim 3, characterized in that, Based on general semantic features, invalid noise in modality-specific features is identified and removed, while retaining effective discriminative details, including: Modality-specific features and cross-modality general semantic features are mapped to the same feature space through a learnable projection matrix to establish semantic association; Using the global common information of general semantic features as a reference, identify invalid noise that is irrelevant to identity discrimination in modality-specific features; By using a channel-wise multiplication semantic calibration method, the feature response corresponding to noise is suppressed, while the effective discrimination of details is enhanced.

6. The visible light-infrared pedestrian re-identification method according to claim 3, characterized in that, The process of fusing the purified dual-modal unique detail features and supplementing them into the general semantic features to enrich their fine-grained discriminative information includes: The purified visible light-specific detail features and infrared-specific detail features are fused together to integrate the differentiated fine-grained information of the two modes; The fused detailed features are complemented by cross-modal general semantic features to supplement the lack of detailed information in the general semantics with fine-grained information.

7. The visible light-infrared pedestrian re-identification method according to claim 1, characterized in that, Before acquiring visible light and infrared images of the monitored scene and extracting pedestrian target areas from the images, the process further includes: The model is trained under constraints using an overall loss function, which consists of modality-aware weight-guided loss, bidirectional interactive collaboration loss, hard sample triplet loss, and identity classification loss.

8. A visible light-infrared pedestrian re-identification system, characterized in that, include: The acquisition module is used to acquire visible light and infrared images of the monitored scene and extract pedestrian target areas from the images; The processing module is used to convert the preprocessed image into initial features, enhance details and contour extraction based on the characteristics of visible light or infrared images, and then obtain optimized features with cross-modal consistency and discriminability through modal cross-calibration denoising and bidirectional feature optimization. The process of converting the preprocessed image into initial features, enhancing details and contour extraction based on the characteristics of visible light or infrared images, and then obtaining optimized features with cross-modal consistency and discriminability through modal cross-calibration denoising and bidirectional feature optimization includes: The initial features are obtained through mapping; Calculate the channel-level global mean and standard deviation of the initial visible light characteristics, as well as the corresponding indices of the initial infrared characteristics, to clarify the differences in modal characteristics; Based on the quantization differences, a weight distribution adapted to the two modalities is generated; By using feature extraction paths with different dilation rates, small-scale texture features of visible light, large-scale contour features of infrared light, and mesoscale features that balance local and global characteristics are obtained respectively. Based on the weight distribution of the generated two-mode adaptation, the multi-scale features are fused to obtain dual-mode enhanced features that strengthen the core discriminative information; Optimized features with cross-modal consistency and discriminativeness are obtained through modal cross-calibration denoising and bidirectional feature optimization. The step of generating a weight distribution adapted to the two modalities based on quantization differences includes: The channel-level global mean and global standard deviation of the initial visible light features are concatenated in channel index order to form the visible light statistical feature vector; similarly, the channel-level global mean and global standard deviation of the initial infrared features are concatenated to form the infrared statistical feature vector. The statistical feature vector is subjected to high-dimensional reduction, nonlinear activation, probability normalization and modality index discrimination in sequence, and the discrete mode label is output. Global average pooling is performed on the initial features to obtain a global feature vector. The modality flag is mapped to a continuous embedding vector. The global feature vector and the embedding vector are concatenated and extended to the spatial size of the initial features. After activation normalization, a three-channel weight distribution map is generated. The output module is used to calculate the feature similarity between the query image and the candidate images in the library, and output the pedestrian identities in order of similarity.

Images