Multispectral vehicle re-identification method based on cross-modal compression, medium and equipment

Abstract

The present application relates to the technical field of vehicle re-identification, and discloses a multi-spectral vehicle re-identification method based on cross-modal compression, a medium and equipment. The multi-spectral vehicle re-identification method comprises: acquiring a photographed image of a vehicle to be re-identified; and inputting the photographed image into a multi-spectral vehicle re-identification model to output cross-modal deep fusion features. A dynamic cross-modal feature compression module of the multi-spectral vehicle re-identification model retains only channels that are structurally discriminative through channel shuffling and a double-path gating mechanism, and prunes redundant features; a double-anchor point cyclic fusion module performs cross-modal fusion of RGB and NIR features with TIR features as anchor points, achieving bidirectional alignment of RGB and NIR and bidirectional transfer of thermal infrared fine-grained features to the visible light domain; an auxiliary identity loss is introduced to maintain identity consistency between modalities as intermediate supervision; and cross-modal reconstruction loss is used to achieve cyclic consistency constraint of cross-modalities and reduce distribution differences between spectrums.

Description

Technical Field

[0001] This invention relates to the field of vehicle re-identification technology, and in particular discloses a multispectral vehicle re-identification method, medium and device based on cross-modal compression. Background Technology

[0002] Vehicle re-identification (Re-ID), a core technology in intelligent transportation systems, public safety, and autonomous driving, aims to retrieve target vehicles from multiple non-overlapping cameras and plays a crucial role in intelligent traffic monitoring systems. Traditional multimodal vehicle re-identification methods are often affected by adverse environmental factors such as complex lighting variations and cluttered background interference. In recent years, with the development of deep learning, vehicle re-identification technology based on visible light images has made significant progress. However, in complex real-world scenes, especially under conditions of strong lighting, backlighting, or flares caused by strong reflections from vehicle surfaces, the imaging quality of visible light sensors deteriorates drastically. Strong flares can obscure key discriminative features of vehicles (such as license plates, logos, and textures), causing color distortion and blurred contours, rendering models relying on appearance features completely ineffective. Most existing models are trained on well-lit datasets and lack generalization ability for extreme, localized high-intensity noise such as flares, leading to a sharp drop in recognition accuracy. Therefore, improving the performance and robustness of vehicle re-identification under strong flare conditions is of significant research importance.

[0003] Multi-spectral vehicle re-identification (MRV) aims to retrieve target vehicles from multiple non-overlapping cameras and plays a crucial role in intelligent traffic monitoring systems. Traditional MRV re-identification methods are often affected by adverse environmental factors such as complex lighting variations and cluttered background interference. Although recent research has made some progress, the problem of strong light interference prevalent in real-world traffic scenarios is often overlooked. Because RGB and near-infrared (NIR) spectra are highly sensitive to strong light interference, key discriminative features of these two spectra are often masked, leading to performance degradation. A direct solution is to use image restoration techniques to reconstruct the interfered areas. However, mapping damaged images to clear images requires a large number of paired samples, which are often difficult to obtain. Strong light interference mainly involves two key issues: First, RGB and NIR spectra exhibit different degradation modes under strong light interference, easily incorporating redundant or invalid features, thus weakening the cross-modal fusion effect. Second, thermal infrared (TIR) ​​spectra have natural anti-flare capabilities and can provide stable local feature references for RGB and NIR through cross-modal alignment. However, guiding information from TIR spectra can also introduce mode-specific biases. Therefore, mitigating the feature distribution shift caused by spectral differences and achieving consistent semantic alignment are key to overcoming strong light interference.

[0004] Therefore, it is necessary to provide a new multispectral vehicle re-identification method, medium, and device based on cross-modal compression to solve the above-mentioned technical problems. Summary of the Invention

[0005] The main objective of this invention is to provide a multispectral vehicle re-identification method, medium, and device based on cross-modal compression, aiming to solve the problem that existing multimodal vehicle re-identification methods are easily affected by adverse environmental factors such as complex lighting changes and cluttered background interference.

[0006] To achieve the above objectives, this invention proposes a multispectral vehicle re-identification method based on cross-modal compression, comprising the following steps: S1: Acquire images of the vehicle to be re-identified, including RGB spectral images, TIR spectral images, and NIR spectral images; S2: Input the RGB spectral image, TIR spectral image and NIR spectral image into the multispectral vehicle re-identification model to obtain cross-modal deep fusion features; The specific method for obtaining the multispectral vehicle re-identification model is as follows: A dynamic cross-modal feature compression module capable of outputting cross-modal enhanced fusion features is constructed based on channel shuffling and dual-path gating mechanisms. Based on multi-source feature input and three-stage global distillation, a dual-anchor loop fusion module capable of outputting cross-modal deep fusion features is constructed. The input end of the dual-anchor loop fusion module is connected to the output end of the dynamic cross-modal feature compression module to obtain the re-identification pre-trained model. Based on the existing vehicle re-identification dataset and the total loss function, joint supervised training is carried out on the re-identification pre-trained model until the convergence condition is met, resulting in a multispectral vehicle re-identification model. During the training process, RGB spectral images, TIR spectral images and NIR spectral images are trained independently and their parameters are not shared.

[0007] Optionally, the process by which the dynamic cross-modal feature compression module outputs cross-modal enhanced fusion features is as follows: ① Use the ViT-B / 16 backbone network to extract the token features and corresponding input patch tags of RGB spectral images, TIR spectral images and NIR spectral images respectively; ② Concatenate the token features of the RGB and NIR spectral images along the embedding dimension to generate a hybrid feature map. ; ③ Based on the hybrid feature map, a channel shuffling operation is introduced to enhance the interaction between corresponding channels of the RGB spectral image and the NIR spectral image, resulting in an enhanced fused feature map. ; ④ Based on enhanced post-fusion feature maps Introducing scaling factors to calculate dynamic thresholds ; ⑤ Extract the positive feature map from the enhanced fused feature map based on the dynamic threshold. and negative feature map Specifically: ; ; ⑥ Based on the negative feature map, a series of operations including global average pooling, dimension reshaping, linear layers, and Sigmoid activation are performed to generate a channel-level scaling factor between 0 and 1. ⑦. By assigning different attention weights to the negative feature map using a channel-level scaling factor, the adjusted negative feature map is obtained; ⑧. The adjusted negative and positive feature maps are fused into a cross-modal enhanced fusion feature by pixel-by-pixel addition, where the cross-modal enhanced fusion feature includes a global token feature. and several local token features .

[0008] Optionally, a scaling factor is introduced based on the enhanced fused feature map to calculate the dynamic threshold, specifically including: Global average pooling and reshape operations are performed on the enhanced fused feature map to obtain a channel mean representation; Linear layers are used to model the dependencies between corresponding channels of RGB and NIR spectral images, and scaling factors are generated using the Sigmoid activation function. The dynamic threshold is obtained by multiplying the scaling factor by the channel mean.

[0009] Optionally, the dual-anchor cyclic fusion module uses the token features of the TIR spectral image as anchors and performs cross-modal fusion with the token features of the RGB spectral image and the NIR spectral image, respectively. The process of outputting cross-modal deep fusion features is as follows: token features of NIR spectral images Token features of TIR spectral images and hybrid feature maps As input, three-stage fusion features of TIR and NIR spectral images are generated through three consecutive global distillation modules. Specifically: ① Token features of NIR spectral images Token features of TIR spectral images and hybrid feature maps Input the first global distillation module to generate the first-stage category label corresponding to the NIR spectral image; ② A three-stage fusion feature of TIR and NIR spectral images is generated through three consecutive global distillation modules. Specifically, it includes: The first-stage category labels are paired with the input patch labels to obtain the second-stage input features; Using the second-stage input features as input, the second-stage category label is generated through the second global distillation module; The second-stage category labels are paired with the input patch labels to obtain the third-stage input features; Using the third-stage input features as input, the third-stage category label is generated through the third global distillation module; The third-stage category labels are paired with the input labels to obtain the three-stage fusion features of the TIR and NIR spectral images. ; token features of RGB spectral images Token features of TIR spectral images and hybrid feature maps As input, three-stage fusion features of TIR and RGB spectral images are generated through three consecutive global distillation modules. ; Will and Cross-modal deep fusion features are obtained by concatenating features along the feature dimension. .

[0010] Optionally, the token feature of the RGB spectral image. Token features of TIR spectral images and hybrid feature maps As input, three-stage fusion features of TIR and RGB spectral images are generated through three consecutive global distillation modules. Specifically, it includes: ① Token features of RGB spectrum images Token features of TIR spectral images and hybrid feature maps Input the first global distillation module to generate the first-stage category label corresponding to the RGB spectral image; ② Employing a three-stage fusion feature generation method Using the same method, three-stage fusion features of TIR and RGB spectral images are generated through three consecutive global distillation modules. .

[0011] Optionally, the token features of the NIR spectral image Token features of TIR spectral images and hybrid feature maps The first-stage category label corresponding to the NIR spectral image generated by the first global distillation module is as follows: will come from The global token features are transformed linearly to generate query embeddings. ,in: Global token features for NIR spectral images; A vector space consisting of D real numbers

[0012] will come from The local token features are transformed by two independent linear transformations to generate key embeddings. Sum embedding Among them: In and Layer normalization is introduced in the embedding; For mixed feature maps Local token features, It is a feature matrix composed of M D-dimensional eigenvectors; Based on query embedding Key embedding Sum embedding Generated through a multi-head cross-attention mechanism and Interaction between The specific formula is as follows: ; in, It is the key embedded dimension; Interaction relationships Through feedforward network First-stage category labeling for generating NIR spectral images To achieve NIR spectral images and enhanced image features The initial spatial alignment is calculated using the following formula: ; ; in, Representing modes Update class tag, For NIR spectral images, This represents a mixed mode of NIR and RGB spectral images. TIR spectral image; This is the first stage of the dual-anchor cyclic fusion process; Generate second-stage category tags The specific formula is as follows: ; in, This is the second phase of the dual-anchor cyclic fusion process. To make the mode Feature information is introduced into the modality In the middle, for modes Enhance and update the features; Generate third-stage category tags The specific formula is as follows: ; in, This is the third stage of the dual-anchor cyclic fusion process.

[0013] Optionally, the token features of the RGB spectral image are used. Token features of TIR spectral images and hybrid feature maps The first-stage category label corresponding to the RGB spectral image generated by the first global distillation module is as follows: will come from The global token features are transformed linearly to generate query embeddings. ,in: The global token feature for RGB spectral images; will come from The local token features are transformed by two independent linear transformations to generate key embeddings. Sum embedding Among them: In and Layer normalization is introduced in the embedding; Based on query embedding Key embedding Sum embedding Generated through a multi-head cross-attention mechanism and Interaction between The specific formula is as follows: ; in, It is the key embedded dimension; Interaction relationships The first-stage category labeling corresponding to the NIR spectral image is generated through a feedforward network. To achieve NIR spectral images and enhanced image features The initial spatial alignment is calculated using the following formula: ; ; in Representing modes Update class tag, It is an RGB spectrum image; Generate second-stage category tags The specific formula is as follows: ; in: For modality Feature information is introduced into the modality In the middle, for modes Enhance and update the features; Generate third-stage category tags The specific formula is as follows: .

[0014] Optionally, the total loss function The specific formula is as follows: ; in: This represents the main identity classification loss for the final fused features. It is the standard triple sorting loss. It is a loss due to auxiliary identity supervision. It is the cross-modal reconstruction loss; The weights for the primary identity classification loss; The weights for the standard triplet sorting loss; Weights for assisting identity oversight loss; Weights for cross-modal reconstruction loss; Loss of auxiliary identity supervision The specific formula is as follows: ; in: It is the first in the existing vehicle re-identification dataset. The identity label of each sample; This represents the probability that the auxiliary classifier predicts the correct category. Cross-modal reconstruction loss The specific formula is as follows: ; ; ; in: This refers to the cross-modal reconstruction loss when the NIR mode is used as the input mode; This refers to the cross-modal reconstruction loss when using the NIR-RGB fused mode as the input mode; This represents the cross-modal reconstruction loss when the TIR mode is used as the input mode; For Using modal features as input, reconstruct Modal features and real Modal feature calculation and reconstruction error; The number of samples used in the calculation of cross-modal reconstruction loss; For Cross-modal reconstruction loss when a mode is used as an input mode; For the first A sample from Modal mapping to Modal characteristics; Indicates directly from Reference features for modality extraction.

[0015] In addition, the present invention provides a readable storage medium storing computer program instructions that, when executed by a processor, implement the multispectral vehicle re-identification method based on cross-modal compression as described above.

[0016] In addition, the present invention also provides an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, wherein the computer program instructions are executed by the processor to perform the multispectral vehicle re-identification method based on cross-modal compression as described above.

[0017] The dynamic cross-modal feature compression module of this invention retains only structurally discriminative channels and removes redundant features through channel shuffling and dual-path gating mechanisms. Secondly, the dual-anchor cyclic fusion module uses TIR features as anchors to perform cross-modal fusion with RGB and NIR features respectively, achieving bidirectional alignment between RGB and NIR, thus enabling bidirectional transfer of fine-grained thermal infrared features to the visible light domain. Furthermore, to maintain intermodal identity consistency, this invention introduces an auxiliary identity loss as intermediate supervision. Finally, cross-modal cyclic consistency constraints are achieved through cross-modal reconstruction loss, explicitly reducing spectral distribution differences. This invention solves the problem that multimodal vehicle re-identification methods are easily affected by adverse environmental factors such as complex lighting changes and cluttered background interference, improving vehicle re-identification detection performance under extreme lighting conditions. It provides new ideas for feature extraction theory and research, and plays a certain reference and guiding role in the development of vehicle re-identification target detection. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the method for obtaining the multispectral vehicle re-identification model in an embodiment of the present invention; Figure 2(a) is a flowchart of the dynamic cross-modal feature compression module in an embodiment of the present invention; Figure 2(b) is a schematic diagram of the process of cross-modal fusion of the dual-anchor point cyclic fusion module with the token feature of the TIR spectral image as the anchor point and the token feature of the NIR spectral image in an embodiment of the present invention. Figure 2(c) is a schematic diagram of the process of cross-modal fusion of the dual-anchor point cyclic fusion module with the token feature of the TIR spectral image as the anchor point and the token feature of the RGB spectral image in an embodiment of the present invention. Figure 3 This is a schematic diagram of the model structure for calculating the dynamic threshold in an embodiment of the present invention; Figure 4 This is a schematic diagram of the model structure for generating positive and negative feature maps in an embodiment of the present invention; Figure 5 This is a schematic diagram of the model structure for global distillation in an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the accuracy analysis of vehicle re-identification under different weight configurations of the CSR loss function in an embodiment of the present invention; Figure 7(a) is a schematic diagram of the distance distribution characteristics of the existing ViT-B / 16 backbone network in the feature space; Figure 7(b) is a schematic diagram of the distance distribution characteristics of the multispectral vehicle re-identification method based on cross-modal compression in the feature space in an embodiment of the present invention; Figure 8(a) is a schematic diagram of the effectiveness visualization analysis of the ViT-B / 16 backbone network; Figure 8(b) is a schematic diagram of the effectiveness analysis of combining the ViT-B / 16 backbone network and the dynamic cross-modal feature compression module; Figure 8(c) is a schematic diagram of the effectiveness analysis of combining the ViT-B / 16 backbone network, dynamic cross-modal feature compression module and dual-anchor point cyclic fusion module; Figure 8(d) is a schematic diagram of the effectiveness analysis of combining the ViT-B / 16 backbone network, dynamic cross-modal feature compression module, dual-anchor cyclic fusion module and cross-modal loss function.

[0020] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0022] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0023] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0024] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0025] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0026] This invention proposes a multispectral vehicle re-identification method, medium, and device based on cross-modal compression, aiming to solve the problem that existing multimodal vehicle re-identification methods are easily affected by adverse environmental factors such as complex lighting changes and cluttered background interference.

[0027] See Figures 1 to 5A multispectral vehicle re-identification method based on cross-modal compression includes the following steps: S1: Acquire images of the vehicle to be re-identified, including RGB spectral images, TIR spectral images, and NIR spectral images; S2: Input the RGB spectral image, TIR spectral image and NIR spectral image into the multispectral vehicle re-identification model to obtain cross-modal deep fusion features; Referring to Figures 2(a) to 2(c), the specific method for obtaining the multispectral vehicle re-identification model is as follows: A dynamic cross-modal feature compression module capable of outputting cross-modal enhanced fusion features is constructed based on channel shuffling and dual-path gating mechanisms. The process by which the dynamic cross-modal feature compression module outputs cross-modal enhanced fusion features is as follows: ① Use the ViT-B / 16 backbone network to extract the token features and corresponding input patch tags of RGB spectral images, TIR spectral images and NIR spectral images respectively; ② Concatenate the token features of the RGB and NIR spectral images along the embedding dimension to generate a hybrid feature map. ; ③ Based on the hybrid feature map, a channel shuffling operation is introduced to enhance the interaction between corresponding channels of the RGB spectral image and the NIR spectral image, resulting in an enhanced fused feature map. The specific formula is as follows: ; in: Indicates a channel mixing operation; Global features of NIR spectral images extracted by the ViT-B / 16 backbone network; Global features of the RGB spectral image extracted by the ViT-B / 16 backbone network; ④ Based on enhanced post-fusion feature maps Introducing scaling factors to calculate dynamic thresholds Specifically, it includes: Global average pooling and reshape operations are performed on the enhanced fused feature map to obtain the channel mean representation. The specific formula is as follows: ; Where: GAP represents global equal pooling; Linear layers are used to model the dependencies between corresponding channels of RGB and NIR spectral images, and a scaling factor is generated using a sigmoid activation function. The specific formula is as follows: ; in: Modeling operations for linear layers; The dynamic threshold is obtained by multiplying the scaling factor by the channel mean. The specific formula is as follows: ; ⑤ Extract the positive feature map from the enhanced fused feature map based on the dynamic threshold. and negative feature map Specifically: ; ; ⑥ Based on the negative feature map, a series of operations including global average pooling, dimension reshaping, linear layers, and sigmoid activation are performed to generate a channel-level scaling factor between 0 and 1. The specific formula is as follows: ; ⑦. By assigning different attention weights to the negative feature map using a channel-level scaling factor, the adjusted negative feature map is obtained. The specific formula is as follows: ; ⑧. The adjusted negative and positive feature maps are fused into a cross-modal enhanced fusion feature by pixel-by-pixel addition, where the cross-modal enhanced fusion feature includes a global token feature. and several local token features .

[0028] In this embodiment, current multispectral vehicle re-identification methods typically rely on simple feature stitching or modality-specific networks to achieve cross-spectral matching. These methods often depend on global max pooling to generate compact image-level representations, making them susceptible to redundant activations and flare noise. Our designed network is based on dynamic cross-spectral feature compression. Through channel shuffling and dual-path gating mechanisms, it retains structurally discriminative channels and removes redundant features, enhancing the interaction between feature channels and laying the foundation for subsequent dynamic threshold calculation.

[0029] Based on multi-source feature input and three-stage global distillation, a dual-anchor loop fusion module capable of outputting cross-modal deep fusion features is constructed. The input end of the dual-anchor loop fusion module is connected to the output end of the dynamic cross-modal feature compression module to obtain the re-identification pre-trained model. In this embodiment, the dual-anchor-point cyclic fusion module effectively establishes a deep transmission and recursive fusion relationship for cross-modal enhancement features. The dual-anchor-point cyclic fusion module uses the token features of the TIR spectral image as anchors and performs cross-modal fusion with the token features of the RGB spectral image and the NIR spectral image, respectively. The process of outputting the deep cross-modal fused features is as follows: token features of NIR spectral images Token features of TIR spectral images and hybrid feature maps As input, three-stage fusion features of TIR and NIR spectral images are generated through three consecutive global distillation modules. Specifically: ① Token features of NIR spectral images Token features of TIR spectral images and hybrid feature maps The first global distillation module generates the first-stage category label corresponding to the NIR spectral image; specifically: will come from The global token features are transformed linearly to generate query embeddings. ,in: Global token features for NIR spectral images; Let D be a vector space consisting of D real numbers; will come from The local token features are transformed by two independent linear transformations to generate key embeddings. Sum embedding Among them: In and Layer normalization is introduced in the embedding; For mixed feature maps Local token features, It is a feature matrix composed of M D-dimensional eigenvectors; Based on query embedding Key embedding Sum embedding Generated through a multi-head cross-attention mechanism and Interaction between The specific formula is as follows: ; in, It is the key embedded dimension; Interaction relationships Through feedforward network First-stage category labeling for generating NIR spectral images To achieve NIR spectral images and enhanced image features The initial spatial alignment is calculated using the following formula: ; ; in, Representing modes Update class tag, For NIR spectral images, This represents a mixed mode of NIR and RGB spectral images. TIR spectral image; This is the first stage of the dual-anchor cyclic fusion process; Generate second-stage category tags The specific formula is as follows: ; in, This is the second phase of the dual-anchor cyclic fusion process. To make the mode Feature information is introduced into the modality In the middle, for modes Enhance and update the features; Generate third-stage category tags The specific formula is as follows: ; in, This is the third stage of the dual-anchor cyclic fusion process.

[0030] ② A three-stage fusion feature of TIR and NIR spectral images is generated through three consecutive global distillation modules. Specifically, it includes: The first-stage category labels are paired with the input patch labels to obtain the second-stage input features; Using the second-stage input features as input, the second-stage category label is generated through the second global distillation module; The second-stage category labels are paired with the input patch labels to obtain the third-stage input features; Using the third-stage input features as input, the third-stage category label is generated through the third global distillation module; The third-stage category labels are paired with the input labels to obtain the three-stage fusion features of the TIR and NIR spectral images. ; token features of RGB spectral images Token features of TIR spectral images and hybrid feature maps As input, three-stage fusion features of TIR and RGB spectral images are generated through three consecutive global distillation modules. Specifically, it includes: ① Token features of RGB spectrum images Token features of TIR spectral images and hybrid feature maps Input the first global distillation module to generate the first-stage category label corresponding to the RGB spectral image; Token features of RGB spectrum images Token features of TIR spectral images and hybrid feature maps The first-stage category label corresponding to the RGB spectral image generated by the first global distillation module is as follows: will come from The global token features are transformed linearly to generate query embeddings. ,in: The global token feature for RGB spectral images; will come from The local token features are transformed by two independent linear transformations to generate key embeddings. Sum embedding Among them: In and Layer normalization is introduced in the embedding; Based on query embedding Key embedding Sum embedding Generated through a multi-head cross-attention mechanism and Interaction between The specific formula is as follows: ; in, It is the key embedded dimension; Interaction relationships The first-stage category labeling corresponding to the NIR spectral image is generated through a feedforward network. To achieve NIR spectral images and enhanced image features The initial spatial alignment is calculated using the following formula: ; ; in Representing modes Update class tag, It is an RGB spectrum image; Generate second-stage category tags The specific formula is as follows: ; in: For modality Feature information is introduced into the modality In the middle, for modes Enhance and update the features; Generate third-stage category tags The specific formula is as follows: .

[0031] ② Employing a three-stage fusion feature generation method Using the same method, three-stage fusion features of TIR and RGB spectral images are generated through three consecutive global distillation modules. ; Will and Cross-modal deep fusion features are obtained by concatenating features along the feature dimension. The specific formula is as follows: ; in: This refers to the set of modal features involved in the cyclic fusion branch with NIR modes as the fusion main line. ; The set of modal features involved in the cyclic fusion branch with RGB modality as the main fusion line. .

[0032] Based on the existing vehicle re-identification dataset and the total loss function, joint supervised training is carried out on the re-identification pre-trained model until the convergence condition is met, resulting in a multispectral vehicle re-identification model. During the training process, RGB spectral images, TIR spectral images and NIR spectral images are trained independently and their parameters are not shared.

[0033] The total loss function The specific formula is as follows: ; in: This represents the main identity classification loss for the final fused features. It is the standard triple sorting loss. It is a loss due to auxiliary identity supervision. It is the cross-modal reconstruction loss; The weights for the primary identity classification loss; The weights for the standard triplet sorting loss; Weights for assisting identity oversight loss; Weights for cross-modal reconstruction loss; In this embodiment, and Set to 1.0. The weight for cross-modal reconstruction loss is set to 0.5. The value was adjusted within the range of 0.2 to 2, and the optimal value was determined through multiple sets of comparative experiments. When the value is 1.0, the model achieves optimal overall performance on the validation set.

[0034] To facilitate identity feature learning and preserve identity information, we introduce an auxiliary identity supervision loss after the first-stage global distillation. Auxiliary Identity Supervision Loss The specific formula is as follows: ; in: It is the first in the existing vehicle re-identification dataset. The identity label of each sample; This represents the probability that the auxiliary classifier predicts the correct category. The convergence condition in this embodiment is: when the total loss function When the model stabilizes on the validation set and no longer decreases significantly, and the re-identification performance index reaches a preset threshold or remains stable over several consecutive training cycles, the model is considered to have reached the convergence condition, and the training process ends.

[0035] In this embodiment, Instead of being an independent training objective, this involves applying cross-modal consistency regularization constraints to each modal feature branch within a jointly supervised training framework. During training, for each training batch, corresponding modal features are obtained from RGB, NIR, and TIR modal input images via a shared-structure but parameter-independent feature extraction network. These features are then used to generate the final fused features through a dynamic cross-modal compression and recurrent fusion module. Based on this, identity classification loss is simultaneously calculated. Triple sorting loss Losses from auxiliary identity monitoring and cross-modal reconstruction loss Each loss term participates in backpropagation according to a preset weight coefficient, and is used to jointly update network parameters. By constraining the reconstruction consistency between the latent space representations of different modal features after cyclic fusion, the structural alignment at the semantic level is achieved while maintaining the discriminativeness of each modal branch, thereby improving the robustness of cross-spectral re-identification.

[0036] Cross-modal reconstruction loss The specific formula is as follows: ; ; ; in: This refers to the cross-modal reconstruction loss when the NIR mode is used as the input mode; This refers to the cross-modal reconstruction loss when using the NIR-RGB fused mode as the input mode; This represents the cross-modal reconstruction loss when the TIR mode is used as the input mode; For Using modal features as input, reconstruct Modal features and real Modal feature calculation and reconstruction error; The number of samples used in the calculation of cross-modal reconstruction loss; For Cross-modal reconstruction loss when a mode is used as an input mode; For the first A sample from Modal mapping to Modal characteristics; Indicates directly from Reference features for modality extraction.

[0037] The above cross-modal reconstruction loss The specific formula can be expanded as follows: ; ; ; ; ; ; ; ; ; ; in: To reconstruct NIR–RGB fused modal features using NIR modal features as input and compare them with the real modal features... Feature calculation and reconstruction error; To reconstruct TIR modal features using NIR modal features as input and compare them with the true modal features... Feature calculation and reconstruction error; Using NIR–RGB fused features as input, reconstruct NIR modal features and compare them with the true NIR modal features. Feature calculation and reconstruction error; Using NIR–RGB fused features as input, reconstruct TIR modal features and compare them with the true NIR modal features. Feature calculation and reconstruction error; To reconstruct NIR modal features using TIR modal features as input and compare them with the true NIR modal features... Feature calculation and reconstruction error; Using TIR modal features as input, reconstruct NIR–RGB modal features and compare them with the true NIR–RGB modal features. Feature calculation and reconstruction error.

[0038] See Figure 6 This embodiment uses an Ubuntu 22.04 operating system to build a training and testing platform, employing the PyTorch 2.0.0 deep learning framework and CUDA 11.8 acceleration library. The hardware configuration includes an Intel i9-13900KF processor, 32GB of RAM, and runs on an NVIDIA RTX 4090 GPU. The model uses ViT-B / 16 as the backbone network, with pre-trained weights derived from the ImageNet-21K dataset. The SGD optimizer is used for network training, with an initial learning rate of 3×10⁻⁶ on the WMVeID863 dataset. -3 On the RGBNT100 and MSVR310 datasets, the initial learning rate was set to 1×10. -3 The weight decay coefficient is set to 1×10. -4 The total number of training rounds is 120 epochs. The image size of the input network is uniformly adjusted. During the training phase, the feature branches of different spectral modes such as RGB, NIR, and TIR are trained independently with no parameter sharing. Joint supervised training is carried out by combining cross-entropy loss and triplet loss.

[0039] The WMVeID863 dataset is a low-quality multispectral vehicle re-identification dataset for real-world complex scenes. It contains 863 vehicle identities, corresponding to 4709 multispectral image triplets (RGB, NIR, and TIR modes). The data was collected from eight different camera perspectives, with the number of image triplets for a single vehicle ranging from 1 to 39. The dataset is split according to an approximately 3:1 training-to-test ratio, with 3482 image triplets for 603 vehicle identities used for model training and 1226 image triplets for 260 vehicle identities used for model testing. The test set's image library contains all 1226 image triplets for the 260 test identities, while the query samples are randomly selected from the image library, covering 210 vehicle identities and corresponding to 959 image triplets.

[0040] The RGBN300 dataset is a multispectral vehicle re-identification benchmark dataset containing 300 vehicle identities and a total of 100,250 images. The data was collected from 2,004 different scenes and covers both RGB and near-infrared (NIR) spectral modes.

[0041] The RGBNT100 dataset is a small-scale multispectral vehicle re-identification dataset containing 100 vehicle identities and a total of 17,250 images, covering three spectral modes: RGB, NIR, and TIR. The data was collected from 690 different scenes.

[0042] To verify the effectiveness of our proposed method, we conducted comparative experiments on the WMVeID863 dataset. As shown in Table 1, among the single-spectral methods, the Transformer pedestrian re-identification network achieved the best results, with an mAP of 67%. The Transformer-based multispectral enhancement methods, including a Transformer-optimized pedestrian re-identification method and a cross-spectral feature editing and reconstruction network, further improved performance. Notably, the glare-aware cross-modal enhancement network, by utilizing thermal infrared as the dominant spectrum to enhance other glare-affected modalities, achieved an mAP of 69.8%. In contrast, our method effectively mitigates the loss of spectral information under different conditions, achieving an mAP 1.8% higher and a Rank-1 score 2.4% higher than the glare-aware cross-modal enhancement network. Our proposed method places greater emphasis on leveraging the natural immunity of thermal infrared to glare to guide the enhancement module in learning features, thereby achieving superior overall feature representation.

[0043] Table 1. Performance comparison results of our method with existing methods in the field of multispectral person re-identification on the WMVeID863 dataset.

[0044] We also conducted comparative experiments on the RGBNT100 and MSVR310 datasets. As shown in Table 2, although the single-spectral method achieved competitive results, its performance was still slightly inferior to the multispectral method. Among existing multispectral methods, the Transformer-based cross-spectral feature editing and reconstruction network achieved an mAP of 82.1% on RGBNT100 and 39.0% on MSVR310. The decoupled modality optimization network further improved the results by decoupling the spectrum into non-overlapping representations, achieving the best baseline performance. In comparison, our method outperforms the decoupled modality optimization network, improving the mAP by 2.3% and 4.9% on RGBNT100 and MSVR310, respectively. These results validate the superiority of the method of this invention in handling complex scenes and large-scale multispectral vehicle datasets, demonstrating its significant advantages over existing methods.

[0045] Table 2 shows the performance comparison results of our method and existing methods in the field of multispectral person re-identification on the RGBNT100 and MSVR310 datasets.

[0046] To further verify the effectiveness of this method, ablation experiments were conducted on the WMVeID863 dataset in this embodiment. The hyperparameter settings, such as the number of training epochs and batch size, were kept consistent for each experiment, and various modules and loss functions were introduced progressively. As shown in Table 3, compared with the baseline model, the introduction of the DACF module improved mAP by 0.8% and R-1 (first-position hit rate) by 0.6%, verifying the effectiveness of thermal infrared-guided cyclic fusion for cross-modal feature enhancement. When both DCFC and DACF were introduced simultaneously, mAP increased to 69.4%. Based on this, the combined performance reached its highest level, demonstrating that DCFC and DACF have synergistic gains in suppressing redundant features and improving cross-spectral consistency. In Table 3, R-5 represents the top five hit rates, and R-10 represents the top ten hit rates.

[0047] Table 3 Ablation experimental results on the WMVeID863 dataset

[0048] To verify the effectiveness of the CSR loss function proposed in this embodiment in multispectral re-identification tasks, it was compared with several existing state-of-the-art loss functions, as shown in Table 4. The existing 3M loss function imposes independent constraints on the thermal infrared (TI) mode. This constraint weakens the joint optimization capability between different modes to some extent, making it difficult for the feature compression and cyclic fusion modules to fully function, thus limiting overall performance improvement. The CdC loss function, by forcibly aligning RGB and near-infrared (NI) features to thermal infrared (TI) features, enhances cross-modal consistency to some extent, but simultaneously weakens the discriminative feature representation capability of the RGB and NI modes themselves, leading to a decrease in feature discriminability. The HC loss function is mainly designed for cross-modal re-identification tasks. By constraining the consistency between RGB and NI modes, it can mine shared effective features between different visible spectra, thus outperforming the 3M and CdC losses in experiments. However, this loss function relies excessively on the feature center similarity constraint enhanced by thermal infrared, easily introducing information redundancy and limiting further performance improvement. Furthermore, while the MC loss function improves semantic consistency to some extent by constraining the consistency of predicted distributions across different modalities through KL divergence, its enhancement of cross-modal feature reconstruction capabilities is limited, making it difficult to effectively reduce the essential information differences between different spectra. In contrast, the CSR loss function proposed in this embodiment, based on the introduction of semantic consistency constraints, further combines a cross-spectral feature reconstruction mechanism. This not only fully explores the deep semantic similarities between different modalities but also effectively reduces the core information differences between different spectra, thereby significantly improving the cross-modal feature reconstruction capability and overall discriminative performance. Comparative experimental results fully demonstrate that the loss function proposed in this embodiment has superior technical performance in multispectral re-identification tasks.

[0049] Table 4 Comparison results of this method with existing advanced loss functions

[0050] To verify the impact of the CSR loss function proposed in this embodiment on vehicle re-identification performance under different weight configurations, this embodiment conducted a systematic experimental analysis of its loss weights. The experimental results are as follows: Figure 6As shown, inspired by existing multi-branch network architecture design, triplet loss and cross-entropy loss are uniformly introduced into each branch, with their weights fixed at 1.0 to ensure the basic discriminative learning ability of each branch. Based on this, the proposed CSR loss is combined with the baseline model. By changing the weight parameters of the CSR loss within the range of 0.2 to 2.0, comparative experiments are conducted, using mean average accuracy (mAP) and Rank-1 accuracy as evaluation metrics. Experimental results show that when the CSR loss weight is set to 1.0, the system performance reaches its optimal state, with the mean accuracy increasing to 69.9% and the Rank-1 accuracy increasing to 77.2%. This result demonstrates that the proposed CSR loss has good stability and robustness within a reasonable weight range, and can significantly improve the overall performance of cross-spectral vehicle re-identification under optimal configuration, thus verifying the effectiveness and practicality of the loss design.

[0051] Finally, the effectiveness of the proposed method was verified from the perspectives of feature discriminativeness and region of interest. This embodiment compared and analyzed the distance distribution characteristics of the baseline method and the proposed method in the feature space, as shown in Figures 7(a) and 7(b). Through systematic analysis of inter-vehicle and intra-vehicle feature distances, it was found that the proposed method significantly increases the feature distance between different vehicles while effectively compressing the feature distribution range of the same vehicle sample, thereby forming a clearer discrimination boundary in the feature space and significantly enhancing the feature discriminative ability.

[0052] Furthermore, this embodiment introduces the Grad-CAM visualization analysis method into each proposed functional module to visualize and verify the region of interest in the final output of the model, as shown in Figures 8(a) to 8(d). The visualization results demonstrate that this embodiment can fully utilize the inherent immunity of thermal infrared spectroscopy to flare interference, guiding the model to focus on the remaining effective discrimination region in the RGB and near-infrared spectra where glare interference exists, thereby effectively suppressing the negative impact of flares on the feature extraction process. The above visualization analysis results further verify the technical effectiveness of this embodiment in achieving stable, multispectral fusion feature representation under complex lighting conditions.

[0053] The dynamic cross-spectral feature compression and enhancement mechanism in this embodiment adaptively compresses and enhances RGB and NIR cross-spectral features through global statistical modeling, channel shuffling, and soft attention weighting, effectively suppressing cross-spectral redundancy and noise features. The dual-anchor cyclic fusion mechanism in this embodiment, anchored by thermal infrared features, achieves deep semantic interaction between RGB, NIR, and TIR features through multi-head cross-attention and multi-stage cyclic distillation, addressing the semantic shift and information inconsistency issues between multiple spectra. Furthermore, a cross-spectral consistency reconstruction loss function is proposed, which collaboratively optimizes multi-spectral features by jointly introducing auxiliary identity supervision and cross-spectral feature reconstruction constraints, enhancing the consistency and reconstructability of cross-spectral features.

[0054] This embodiment provides a readable storage medium storing computer program instructions, which, when executed by a processor, implement the multispectral vehicle re-identification method based on cross-modal compression as described above.

[0055] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0056] This embodiment also includes an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, wherein the computer program instructions are executed by the processor to perform the multispectral vehicle re-identification method based on cross-modal compression as described above.

[0057] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0058] The electronic device can be a mobile phone, desktop computer, laptop, handheld computer, cloud server, or other computing device. The electronic device may include, but is not limited to, processors and memory. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0059] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting all parts of the electronic device via various interfaces and lines.

[0060] The memory can be used to store the computer program and / or modules. The processor implements the computer program by running or executing the computer program and / or modules stored in the memory, and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0061] If the modules / units integrated in the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0062] The above description is only a preferred embodiment of the present invention and does not limit the scope of the present invention. All equivalent structural transformations made under the inventive concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the protection scope of the present invention.

Claims

1. A multispectral vehicle re-identification method based on cross-modal compression, characterized in that, Includes the following steps: S1: Acquire images of the vehicle to be re-identified, including RGB spectral images, TIR spectral images, and NIR spectral images; S2: Input the RGB spectral image, TIR spectral image and NIR spectral image into the multispectral vehicle re-identification model to obtain cross-modal deep fusion features; The specific method for obtaining the multispectral vehicle re-identification model is as follows: A dynamic cross-modal feature compression module capable of outputting cross-modal enhanced fusion features is constructed based on channel shuffling and dual-path gating mechanisms. Based on multi-source feature input and three-stage global distillation, a dual-anchor loop fusion module capable of outputting cross-modal deep fusion features is constructed. The input end of the dual-anchor loop fusion module is connected to the output end of the dynamic cross-modal feature compression module to obtain the re-identification pre-trained model. Based on the existing vehicle re-identification dataset and the total loss function, joint supervised training is carried out on the re-identification pre-trained model until the convergence condition is met, resulting in a multispectral vehicle re-identification model. During the training process, RGB spectral images, TIR spectral images and NIR spectral images are trained independently and their parameters are not shared.

2. The multispectral vehicle re-identification method based on cross-modal compression according to claim 1, characterized in that, The process by which the dynamic cross-modal feature compression module outputs cross-modal enhanced fusion features is as follows: ① Use the ViT-B / 16 backbone network to extract the token features and corresponding input patch tags of RGB spectral images, TIR spectral images and NIR spectral images respectively; ② Concatenate the token features of the RGB and NIR spectral images along the embedding dimension to generate a hybrid feature map. ; ③ Based on the hybrid feature map, a channel shuffling operation is introduced to enhance the interaction between corresponding channels of the RGB spectral image and the NIR spectral image, resulting in an enhanced fused feature map. ; ④ Based on enhanced post-fusion feature maps Introducing scaling factors to calculate dynamic thresholds ; ⑤ Extract the positive feature map from the enhanced fused feature map based on the dynamic threshold. and negative feature map Specifically: ； ； ⑥ Based on the negative feature map, a series of operations including global average pooling, dimension reshaping, linear layers, and Sigmoid activation are performed to generate a channel-level scaling factor between 0 and 1. ⑦. By assigning different attention weights to the negative feature map using a channel-level scaling factor, the adjusted negative feature map is obtained; ⑧. The adjusted negative and positive feature maps are fused into a cross-modal enhanced fusion feature by pixel-by-pixel addition, where the cross-modal enhanced fusion feature includes a global token feature. and several local token features .

3. The multispectral vehicle re-identification method based on cross-modal compression according to claim 2, characterized in that, The dynamic threshold is calculated based on a scaling factor introduced from the enhanced fused feature map, specifically including: Global average pooling and reshape operations are performed on the enhanced fused feature map to obtain a channel mean representation; Linear layers are used to model the dependencies between corresponding channels of RGB and NIR spectral images, and scaling factors are generated using the Sigmoid activation function. The dynamic threshold is obtained by multiplying the scaling factor by the channel mean.

4. The multispectral vehicle re-identification method based on cross-modal compression according to claim 3, characterized in that, The dual-anchor cyclic fusion module uses the token features of the TIR spectral image as anchors and performs cross-modal fusion with the token features of the RGB spectral image and the NIR spectral image, respectively. The process of outputting cross-modal deep fusion features is as follows: token features of NIR spectral images Token features of TIR spectral images and hybrid feature maps As input, three-stage fusion features of TIR and NIR spectral images are generated through three consecutive global distillation modules. Specifically: ① Token features of NIR spectral images Token features of TIR spectral images and hybrid feature maps Input the first global distillation module to generate the first-stage category label corresponding to the NIR spectral image; ② A three-stage fusion feature of TIR and NIR spectral images is generated through three consecutive global distillation modules. Specifically, it includes: The first-stage category labels are paired with the input patch labels to obtain the second-stage input features; Using the second-stage input features as input, the second-stage category label is generated through the second global distillation module; The second-stage category labels are paired with the input patch labels to obtain the third-stage input features; Using the third-stage input features as input, the third-stage category label is generated through the third global distillation module; The third-stage category labels are paired with the input labels to obtain the three-stage fusion features of the TIR and NIR spectral images. ; token features of RGB spectral images Token features of TIR spectral images and hybrid feature maps As input, three-stage fusion features of TIR and RGB spectral images are generated through three consecutive global distillation modules. ; Will and Cross-modal deep fusion features are obtained by concatenating features along the feature dimension. .

5. The multispectral vehicle re-identification method based on cross-modal compression according to claim 4, characterized in that, token features of RGB spectral images Token features of TIR spectral images and hybrid feature maps As input, three-stage fusion features of TIR and RGB spectral images are generated through three consecutive global distillation modules. Specifically, it includes: ① Token features of RGB spectrum images Token features of TIR spectral images and hybrid feature maps Input the first global distillation module to generate the first-stage category label corresponding to the RGB spectral image; ② Employing a three-stage fusion feature generation method Using the same method, three-stage fusion features of TIR and RGB spectral images are generated through three consecutive global distillation modules. .

6. The multispectral vehicle re-identification method based on cross-modal compression according to claim 5, characterized in that, Token features of NIR spectral images Token features of TIR spectral images and hybrid feature maps The first-stage category label corresponding to the NIR spectral image generated by the first global distillation module is as follows: will come from The global token features are transformed linearly to generate query embeddings. ,in: Global token features for NIR spectral images; Let D be a vector space consisting of D real numbers; will come from The local token features are transformed by two independent linear transformations to generate key embeddings. Sum embedding Among them: In and Layer normalization is introduced in the embedding; For mixed feature maps Local token features, It is a feature matrix composed of M D-dimensional eigenvectors; Based on query embedding Key embedding Sum embedding Generated through a multi-head cross-attention mechanism and Interaction between The specific formula is as follows: ； in, It is the key embedded dimension; Interaction relationships Through feedforward network First-stage category labeling for generating NIR spectral images To achieve NIR spectral images and enhanced image features The initial spatial alignment is calculated using the following formula: ； ； in, Representing modes Update class tag, For NIR spectral images, This represents a mixed mode of NIR and RGB spectral images. TIR spectral image; This is the first stage of the dual-anchor cyclic fusion process; Generate second-stage category tags The specific formula is as follows: ； in, This is the second phase of the dual-anchor cyclic fusion process; To make the mode Feature information is introduced into the modality In the middle, for modes Enhance and update the features; Generate third-stage category tags The specific formula is as follows: ； in, This is the third stage of the dual-anchor cycle fusion process.

7. The multispectral vehicle re-identification method based on cross-modal compression according to any one of claims 1-5, characterized in that, Token features of RGB spectrum images Token features of TIR spectral images and hybrid feature maps The first-stage category label corresponding to the RGB spectral image generated by the first global distillation module is as follows: will come from The global token features are transformed linearly to generate query embeddings. ,in: The global token feature for RGB spectral images; will come from The local token features are transformed by two independent linear transformations to generate key embeddings. Sum embedding Among them: In and Layer normalization is introduced in the embedding; Based on query embedding Key embedding Sum embedding Generated through a multi-head cross-attention mechanism and Interaction between The specific formula is as follows: ； in, It is the key embedded dimension; Interaction relationships The first-stage category labeling corresponding to the NIR spectral image is generated through a feedforward network. To achieve NIR spectral images and enhanced image features The initial spatial alignment is calculated using the following formula: ； ； in Representing modes Update class tag, RGB spectrum image; Generate second-stage category tags The specific formula is as follows: ； in: For modality Feature information is introduced into the modality In the middle, for modes Enhance and update the features; Generate third-stage category tags The specific formula is as follows: 。 8. The multispectral vehicle re-identification method based on cross-modal compression according to claim 7, characterized in that, The total loss function The specific formula is as follows: ； in: This represents the main identity classification loss for the final fused features. It is the standard triple sorting loss. It is a loss due to auxiliary identity supervision. It is the cross-modal reconstruction loss; The weights for the primary identity classification loss; The weights for the standard triplet sorting loss; Weights for assisting identity oversight loss; Weights for cross-modal reconstruction loss; Loss of auxiliary identity supervision The specific formula is as follows: ； in: It is the first in the existing vehicle re-identification dataset. The identity label of each sample; This represents the probability that the auxiliary classifier predicts the correct category. Cross-modal reconstruction loss The specific formula is as follows: ； ； ； in: This refers to the cross-modal reconstruction loss when the NIR mode is used as the input mode; This refers to the cross-modal reconstruction loss when using the NIR-RGB fused mode as the input mode; This represents the cross-modal reconstruction loss when the TIR mode is used as the input mode; For Using modal features as input, reconstruct Modal features and real Modal feature calculation and reconstruction error; The number of samples used in the calculation of cross-modal reconstruction loss; For Cross-modal reconstruction loss when a mode is used as an input mode; For the first A sample from Modal mapping to Modal characteristics; Indicates directly from Reference features for modality extraction.

9. A readable storage medium, characterized in that, It stores computer program instructions that, when executed by a processor, implement the multispectral vehicle re-identification method based on cross-modal compression as described in any one of claims 1 to 8.

10. An electronic device, characterized in that, include: At least one processor, at least one memory, and computer program instructions stored in the memory, wherein the computer program instructions are executed by the processor as described in any one of claims 1 to 8, a multispectral vehicle re-identification method based on cross-modal compression.

Images