A multi-spectral vehicle re-identification method under strong light based on cycle consistency

Abstract

A kind of multi-spectral vehicle re-identification method under strong light based on cycle consistency belongs to vehicle re-identification technical field, solve the vehicle re-identification problem under strong light;The present application carries out data preprocessing to spectral image, extracts the high-dimensional feature corresponding to spectrum, and uses strong light label predictor to mark strong light false label to the image polluted by strong light;Strong light pollution hot infrared spectrum and damaged spectrum are fused into de-speckle spectrum by using de-speckle generator to replace damaged spectrum;An image quality predictor is trained using the false label obtained by the strong light label predictor, an interactive fusion module with strong light perception is designed, and the feature level fusion is dynamically adjusted according to the image quality during the feature fusion process;Image level fusion is carried out on multi-spectrum, and an effective image level fusion framework is proposed, and the relationship between multi-branch features is adaptively adjusted using the prediction module with strong light perception to enhance the effective features invariant to strong light and suppress the invalid features susceptible to strong light interference.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of vehicle re-identification, and relates to a multi-spectral vehicle re-identification method under strong light based on cyclic consistency. BACKGROUND

[0002] Vehicle re-identification is an important research direction in computer vision, and its main goal is to identify the same vehicle under different camera perspectives, different times and different places. This has important value for intelligent transportation systems, city monitoring, vehicle tracking and other application scenarios. The main challenges of vehicle re-identification include but are not limited to: perspective change, light change, occlusion, diversity of vehicle color and model, etc. In order to solve these problems, researchers have proposed many methods, such as the deep learning-based method adopted in the document "Cross-directional consistency network with adaptive layer normalization for multi-spectral vehicle re-identification and a high-quality benchmark" (Aihua Z, Xianpeng Z, Zhiqi M, et al. Information Fusion, 2023, 100.).

[0003] Compared with other objects, vehicles are more susceptible to light interference because vehicles have their own light sources, such as car lights, and smooth car surfaces can reflect light. This type of light interference can seriously affect the imaging of existing cameras, further leading to the decline of vehicle re-identification performance. A common solution is to use image generation methods to restore images affected by strong light. However, this method requires paired strong light-non-strong light data, which is very difficult to collect in real-world scenarios. Therefore, existing methods based on visible light images are difficult to maintain superior performance for vehicle re-identification under strong light. Multi-spectral vehicle re-identification methods overcome the weakness of traditional visible light-based methods in extreme conditions (night, strong light, fog, etc.) by utilizing the imaging advantages of different spectra (visible light, near-infrared, thermal infrared) in various environments. Thermal infrared spectrum solves the problem of light interference in imaging principle and has strong light invariance, but it is inferior to visible light and near-infrared spectrum in imaging quality. Existing multi-spectral methods, such as the literature "Multi-Spectral Vehicle Re-Identification: A Challenge" (Li H, Li C, Zhu X, et al. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(07).) and the literature "A Multi-Spectral Vehicle Re-Identification Method Based on Cross-Consistency" (Patent Application No.: 202210049114X), mainly focus on utilizing the complementary and heterogeneous information between multi-spectra and feature fusion. In dealing with strong light problems, visible light and near-infrared spectra are easily affected by strong light, resulting in the loss of discriminative clues in their images, which significantly affects multi-spectral feature fusion. SUMMARY

[0004] The technical solution of the present application is used to solve the problem of vehicle re-identification under strong light.

[0005] The present application solves the above technical problems by the following technical solutions:

[0006] A multi-spectral vehicle re-identification method based on cycle consistency under strong light, comprising the following steps:

[0007] Step 1: First, data preprocessing is performed on the input thermal infrared, visible light, and near-infrared spectral images; the data preprocessing includes converting the images to tensors and adjusting them to tensors with a width of 256 pixels and a height of 128 pixels for subsequent deep neural network feature extraction;

[0008] Step 2: The input thermal infrared, visible light, and near-infrared spectral tensors are extracted through three independent deep neural networks ResNet50 to obtain high-dimensional features corresponding to each spectrum;

[0009] Step 3: Determine whether the input image is contaminated by strong light using a strong light label predictor, and mark the image contaminated by strong light with a strong light pseudo label;

[0010] Step 4: Train a strong light-aware de-glare generator using the paired strong light interference data simulated by the random glare method, so as to fuse the strong light contaminated thermal infrared spectrum and the damaged spectrum into a de-glare spectrum to replace the damaged spectrum;

[0011] Step 5: Train an image quality predictor using the pseudo label obtained by the strong light label predictor, predict the overall quality of the picture according to the strong light interference of different modalities, obtain the conditioning factors a and b of the visible light and near infrared modalities, and dynamically adjust the process of multi-modal feature fusion in the adaptive fusion module;

[0012] Step 6: Design a strong light-aware interactive fusion module to dynamically adjust the feature fusion process according to the image quality for feature-level fusion.

[0013] Further, the method of determining whether the input image is contaminated by strong light using a strong light label predictor in step 3 is as follows: Since the thermal infrared spectrum is immune to strong light, only the strong light pseudo label of the visible light and near infrared spectrum is calculated. For the visible light and near infrared spectrum which is easily affected by strong light, the pixel value of the local area of the glare contaminated image is high, so according to the digital image features of the input image, the histogram of the image can be obtained, and the number of pixels in the image is counted to determine whether the image is affected by strong light; according to the statistical information of the image itself, the distribution of pixel values in the R channel can be obtained, the histogram shows that the pixel quantity is distributed between 0 and 255, then the pixel points higher than a certain threshold value θ are set as high value pixels, the proportion of high value pixels to all pixels in the image is calculated, and then it is determined whether the proportion of a specific spectrum is greater than the threshold value set to 1.5%, so as to finally determine whether the spectrum is contaminated by glare.

[0014] Further, the method of training a strong light-aware de-glare generator using the paired strong light interference data simulated by the random glare method in step 4, so as to fuse the strong light contaminated thermal infrared spectrum and the damaged spectrum into a de-glare spectrum to replace the damaged spectrum, is as follows:

[0015] The samples of the thermal infrared modal and the visible light modal not disturbed by strong light interference are obtained from the data set, then the strong light interference is randomly added to the visible light modal, after convolution layer fusion, it is sent into the multi-modal de-light generator A based on cycle consistency, the corresponding relationship between {strong light interference visible light modal, thermal infrared modal} and {de-light visible light modal} is established, the effective information in {strong light interference visible light modal, thermal infrared modal} two modalities is fused, and the ideal state de-light visible light modal is generated. Then, the generated visible light modal not disturbed by strong light interference is sent into the cycle generator B, the strong light is added to it again after convolution layer fusion, and then it is input into the next cycle to form a multi-modal de-light spot generator based on cycle consistency;

[0016] The clean images selected from the data set are added with random light spots to obtain image pairs I and I f , and then the 3-channel RGB image I f and the 3-channel TI image I T are projected into a 3-channel mixed feature f m through a convolution layer. Then f m is input into Generator A. The encoder in Generator A extracts features from f m , and the decoder converts the features into a de-light spot image I df . Generator B shares the same structure as Generator A, and adds light spots to the de-light spot image I df so that it is the same as the synthesized light spot contaminated image I f . In the lower half circle, the original light spot-free image I df is constrained to be the same as the output I df of Generator A through Generator B, and then is mixed with the TI image and converted into I cc through Generator A.

[0017] The loss function is composed of two types of losses: the generative adversarial network (GAN) loss as shown in formulas (1)-(2), and the cycle consistency loss (Cycle Loss) as shown in formulas (3)-(5).

[0018]

[0019]

[0020] loss A (G B ,G A ) B =||G A (conv(R i ,T)))-Ri || (3)

[0021] loss cc (G A ,G B ) B =||G A (conv(G B (R),T))-R|| (4)

[0022] loss cc (G A ,G B )=loss cc (G A ,G B ) A +loss cc (G A ,G B ) B (5)

[0023] By combining the above two GAN loss functions, the goal is to train a Generator A to generate a de-light picture that integrates multi-modal feature discrimination features, achieving image-level fusion.

[0024] Further, the method of training an image quality predictor using the pseudo label obtained by the strong light label predictor in step 5 is as follows: the features f R , f N extracted from ResNet50 are input into a prediction subnetwork containing multiple convolutional layers and a classifier, and the output is the strong light influence factors a and b of the RGB spectrum and the NI spectrum; the pseudo label label f is used as a supervision signal to constrain feature extraction of the prediction subnetwork, which contains multiple convolutional layers, a pooling layer, a fully connected layer, and a classifier; the binary cross-entropy loss formula is used to train the subnetwork.

[0025] Further, the method of designing a strong light perception interactive fusion module to dynamically adjust the feature fusion process according to image quality for feature-level fusion in step 6 is as follows:

[0026] The strong light factors a and b of the RGB and NI spectrum are obtained from the prediction subnetwork, and the feature interaction relationship between different spectral branches is adaptively adjusted according to the degree of influence of the picture on the strong light in the following way, and the feature output is shown in formulas (7), (8), and (9):

[0027] f’ R =f nt +(1-α)·f R ​(7)

[0028] f' N = f rt + (1-beta) * f N (8)

[0029] f' T = f rn + 1 * f T (9)

[0030] In the residual connection with the interaction features f rn , f rt , f nt , the multi-branch features f R , f N , f T participate in feature fusion to establish the relationship between different modalities; the multi-branch features of the interaction fusion module are input into the classifier respectively, and cross entropy loss and triplet loss are used to constrain feature learning.

[0031] The advantages of the present application are:

[0032] The method of the present application analyzes the spot pollution problem from the feature level of the digital image, avoids manual annotation to determine the strong light pollution sample, further designs the adaptive feature fusion of strong light perception, and the strong light perception de-speckle generator can generate a de-speckle image combining the advantages of multiple modalities, and shows excellent performance in the experimental results; the strong light perception interaction fusion module adjusts the feature fusion process of the specific sample quality according to the strong light problem, and compared with the common feature fusion method, it can better fuse the multi-modal information; the feature interaction method, the overall network structure designed for the strong light problem and the different image interference areas of the different light spectrum of the collected multi-spectral vehicle re-identification dataset are different; for the first time, the multi-spectral image-level fusion is proposed, and an effective image-level fusion framework is proposed, in addition, the strong light perception prediction module is used to adaptively adjust the relationship between the multi-branch features, instead of equally considering the multiple modal branches affected by the strong light and the branches not affected by the strong light, to enhance the effective features invariant to strong light and suppress the invalid features susceptible to strong light. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is the spot perception double-layer fusion network structure diagram of the present application, wherein: FLP represents a strong light label predictor, FDG represents a strong light perception de-speckle generator, QP represents a quality predictor, and FIF represents a strong light perception interaction fusion module;

[0034] Figure 2is a structural diagram of a flare label predictor of the present application, wherein: Flare Label Predictor represents a flare label predictor, Histogram represents a statistical histogram;

[0035] Figure 3 is a structural diagram of a flare-aware cycle-consistency-based de-glaring generator of the present application, wherein: Cycle-ConsistencyLoss represents a cycle-consistency loss, Cony represents a convolution operation, and Generator represents a generator;

[0036] Figure 4 is a structural diagram of a quality predictor of the present application, wherein: Quality Predictor represents a quality predictor, PredictionSubnetwork represents a prediction subnetwork, and Cross-Entropy Loss represents a cross-entropy loss function;

[0037] Figure 5 is a structural diagram of a flare-aware interactive fusion module of the present application, wherein: Flare-Aware Interactive Fusion represents a flare-aware interactive fusion. DETAILED DESCRIPTION

[0038] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0039] The technical solutions of the present application will be further described below in conjunction with the drawings in the specification and specific embodiments:

[0040] Embodiment One

[0041] A multi-spectral vehicle re-identification method under flare based on cycle consistency of the present application includes the following contents:

[0042] Step 1: First, the input thermal infrared, visible light, and near-infrared spectral images are preprocessed; the contents of the data preprocessing include: converting the images into tensors, adjusting them to tensors with a width of 256 pixels and a height of 128 pixels, so as to facilitate the feature extraction of the subsequent deep neural network.

[0043] Step 2: The input thermal infrared, visible light, and near-infrared spectral corresponding tensors are extracted through three independent deep neural networks ResNet50 to obtain high-dimensional features corresponding to each spectrum.

[0044] Step 3: Use a strong light label predictor to determine whether the input image is contaminated by strong light, and mark the contaminated image with a strong light pseudo label.

[0045] Whether an input image is affected by strong light cannot be accurately labeled. To determine whether it is affected by strong light, we need to start from the pixel distribution of the image itself. This invention designs a strong light label predictor that uses the digital image features of the image itself to obtain corresponding strong light pseudo-labels.

[0046] like Figure 2 As shown, since the thermal infrared spectrum is immune to strong light, only strong light pseudo-labels in the visible and near-infrared spectra are calculated. For the visible and near-infrared spectra, which are susceptible to strong light interference, the pixel values ​​in localized areas of the image contaminated by light spots are relatively high. Therefore, based on the digital image characteristics of the input image, the image histogram can be obtained, and the number of pixels in the image can be counted to determine whether the image is affected by strong light.

[0047] Based on the statistical information of the image itself, the distribution of pixel values ​​in the R channel can be obtained. The histogram shows that the number of pixels is distributed between 0 and 255. Then, pixels with values ​​higher than a certain threshold θ (set to 250) are set as high-value pixels. The ratio of high-value pixels to all pixels in the image is calculated. Then, it is determined whether the ratio of a specific spectrum is greater than the threshold set to 1.5%, thereby finally determining whether the spectrum is contaminated by light spots.

[0048] However, in some cases, the visible light spectrum may be heavily polluted by light, while the near-infrared spectrum is less affected, and vice versa. Therefore, different labels need to be used for the two spectra, by using bool(λ) RGB >1.5%) or bool(λ) NI >1.5%) to determine whether the two spectra are contaminated by strong light, to ensure the reliability of strong light tampering tags.

[0049] Step 4: Use the paired strong light interference data simulated by the random spot method to train the strong light sensing spot generator, thereby fusing the thermal infrared spectrum of strong light pollution and the damaged spectrum into a spot-removed spectrum to replace the damaged spectrum.

[0050] To minimize the impact of additional light, it is desirable for the de-spot generator to synthesize an ideal spot-free image and maintain identity-related information. The simplest solution is to train an image pair of light pollution images and spot-free images, but it is time-consuming and impractical to collect so many pixel-level aligned image pairs. Therefore, a proper data synthesis method is needed, since the camera only captures the thermal infrared light reflected by the object, the thermal infrared image is never affected by the ambient light. Therefore, the present application uses the CycleGAN architecture to fuse the thermal infrared spectrum immune to strong light pollution and the damaged spectrum into a de-spot spectrum to replace the damaged spectrum.

[0051] A multi-modal de-spot generator is trained using paired strong light interference data simulated by the random spot method, as follows:

[0052] First, samples of thermal infrared modal and visible light modal not affected by strong light interference are obtained from the data set, and then strong light interference is randomly added to the visible light modal. After fusion through the convolution layer, it is sent to the multi-modal de-light generator A based on cycle consistency, to establish the correspondence between {strong light interference visible light modal, thermal infrared modal} and {de-light visible light modal}, to fuse the effective information in {strong light interference visible light modal, thermal infrared modal} two modal, to generate an ideal state of de-light visible light modal. Then, the generated visible light modal not affected by strong light interference is sent to the cycle generator B, and after adding strong light and then through the convolution layer fusion, it enters the next cycle, forming a multi-modal de-spot generator based on cycle consistency.

[0053] As shown in Figure 3 , first, a clean image selected from the data set is added with random spots to obtain image pairs I and I f , and then a convolution layer is applied to project the 3-channel RGB image I f and the 3-channel TI image I T to a 3-channel mixed feature f m , and then f m is input into Generator A, and the encoder in Generator A extracts features from f m , and the decoder converts the features into a de-spot image I df . Generator B shares the same structure as Generator A, and adds spots to the de-spot image I df so that it is the same as the synthesized spot pollution image I f . In the lower half circle, the original spot-free image I is constrained to be the same as the output I df of Generator A, where I is converted to I df through Generator B, then mixed with the TI image, and converted to I df through Generator A.

[0054] The loss function is composed of two types of losses: a generative adversarial network (GAN) loss as shown in equations (1)-(2), and a cycle consistency loss (Cycle Loss) as shown in equations (3)-(5).

[0055]

[0056]

[0057] loss cc (G A ,G B ) A =||G B (G A (conv(R i ,T)))-R i || (3)

[0058] loss cc (G A ,G B ) B =||G A (conv(G B (R),T))-R|| (4)

[0059] loss cc (G A ,G B )=loss cc (G A ,G B ) A +loss cc (G A ,G B ) B (5)

[0060] By combining the two GAN loss functions described above, the goal is to train a Generator A to generate a de-light image that integrates multi-modal feature discriminative features, achieving image-level fusion. In addition, in order to only process images affected by strong light, the pseudo labels obtained in step 3 are used to filter the training samples.

[0061] Step 5: Use the pseudo labels obtained by the strong light label predictor in step 3 to train an image quality predictor, which predicts the overall quality of the image based on the different modalities affected by strong light, and obtains the corresponding quality adjustment factors a and b for the visible light and near-infrared modalities, and dynamically adjusts the multi-modal feature fusion process in the adaptive fusion module.

[0062] Specifically, as shown in FIG. 6, the image quality predictor is trained by using the pseudo labels obtained in step 3, and the image quality predictor is used to predict the overall quality of the image based on the different modalities affected by strong light, and obtain the corresponding quality adjustment factors a and b for the visible light and near-infrared modalities, and dynamically adjust the multi-modal feature fusion process in the adaptive fusion module. Figure 4As shown, the pseudo-labels obtained through Flare Label Predictor f It can determine whether a sample is contaminated by light spots, but the label... f This does not represent the degree of light spot contamination in a particular image or the quality of the image itself after multimodal fusion. Therefore, this invention proposes using labels. f We will train an image quality predictor to predict the quality of a sample that is contaminated by strong light.

[0063] First, the features f extracted from ResNet50 will be... R f N The input is fed into a prediction subnetwork containing multiple convolutional layers and a classifier to obtain the probabilities α and β of RGB (visible light) and NI (near infrared).

[0064] Use pseudo-labels from Flare Label Predictor f A prediction subnetwork is trained to obtain the modulation parameters α and β for RGB and NI. This subnetwork contains multiple convolutional layers, one pooling layer, a fully connected layer, and a classifier. Binary cross-entropy loss (BCE) is used. loss The subnetwork is trained as shown in formula (6).

[0065]

[0066] Step 6: Design an interactive fusion module for strong light perception, and dynamically adjust the feature fusion process based on image quality to perform feature-level fusion.

[0067] like Figure 5 As shown, through (1-α)·f R (1-β)·f N 1·f T The relationships between samples subjected to strong light pollution are adaptively adjusted, and the feature outputs are shown in formulas (7), (8), and (9):

[0068] f' R =f nt +(1-α)·f R (7)

[0069] f' N =f rt +(1-β)·f N (8)

[0070] f' T =f rn +1·f T (9)

[0071] In interaction with feature frn , f rt , f nt When performing residual connection, the multi-branch feature f R , f N , f T participates in feature fusion to establish the relationship between different modalities; the multi-branch features of the interactive fusion module are respectively input into the classifier, and cross-entropy loss and triplet loss are used to constrain feature learning.

[0072] Taking the feature interactive fusion of the RGB branch as an example: wherein f N and f T interact through channel attention and residual connection, and then f R is modulated by the corresponding factor a obtained through the Quality Predictor. R , and f NT are added to obtain the final feature f'. R For the features of the other two branches (near-infrared and thermal infrared), the interactive fusion module will do the same operation. Then f' R , f' N , f' T are connected to obtain the final feature representation.

[0073] The method of the application designs a flare-aware bi-level fusion network, which includes a flare label predictor (FLP), a flare-aware de-flare generator (FDG), a quality predictor (QP), and a flare-aware interactive fusion module (FIF), and is dedicated to solving the problem of extra strong light in the vehicle re-identification task in real scenes, analyzes the flare pollution problem from the feature level of digital images, and avoids manual labeling to locate the flare pollution samples, so as to further design a flare-aware adaptive feature fusion module. For the first time, multispectral image-level fusion is proposed, and a feasible and effective image-level fusion framework based on CycleGAN is proposed. In addition, the flare-aware prediction module is used to adaptively adjust the relationship between the multi-branch features, instead of equally considering the modal branches disturbed and not disturbed, so as to enhance the effective features invariant to strong light and suppress the invalid features susceptible to strong light interference.

[0074] The above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some technical features can be replaced by equivalent features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for multi-spectral vehicle re-identification under strong light based on cycle consistency, characterized in that, The method comprises the following steps: Step 1: first, data preprocessing is performed on the input thermal infrared, visible light and near-infrared spectral images; The data preprocessing includes: converting the image into a tensor, adjusting it to a tensor with a width of 256 pixels and a height of 128 pixels, so as to extract features from the subsequent deep neural network; Step 2: the input thermal infrared, visible light and near-infrared spectral corresponding tensors are extracted through three independent deep neural networks ResNet50 to obtain high-dimensional features corresponding to each spectrum; Step 3: a strong light label predictor is used to determine whether the input image is polluted by strong light, and the image polluted by strong light is marked with a strong light pseudo label; Step 4: a strong light perception de-glare generator is trained by using the paired strong light interference data simulated by the random spot method, so as to fuse the thermal infrared spectrum polluted by strong light and the damaged spectrum into a de-glare spectrum to replace the damaged spectrum, and the specific method is as follows: The samples of the thermal infrared mode and the visible light mode not interfered by strong light are obtained from the data set, then strong light interference is randomly added to the visible light mode, after convolution layer fusion, the visible light mode not interfered by strong light is generated by inputting the visible light mode into a multi-modal de-glare generator A based on cycle consistency, and the corresponding relationship between {strong light interference visible light mode, thermal infrared mode} and {de-glare visible light mode} is established, so as to fuse the effective information in {strong light interference visible light mode, thermal infrared mode} two modes to generate the ideal state de-glare visible light mode; then, the generated visible light mode not interfered by strong light is input into a cycle generator B, and the strong light is added to the visible light mode, then the visible light mode is input into the next cycle after convolution layer fusion, so as to form a multi-modal de-glare generator based on cycle consistency; Random light spots are added to clean images selected from the dataset to obtain image pairs I and I. f Then, a convolutional layer is applied to convert the 3-channel RGB image I f and 3-channel TI image I T Projected onto a 3-channel blending feature F m Then F m Input Generator A, the encoder in Generator A starts from F m Feature extraction, the decoder converts the features into a despotted image I df Generator B shares the same structure as Generator A, which is used for removing smudges from image I. df Add a light spot to make it similar to the synthesized light spot contamination image I f Same; in the lower semicircle, constrain the original spotless image I to be the same as the output I of Generator A. df The same applies, where I is processed by Generator B, then mixed with the TI image, and converted to I by Generator A. df ; The loss function is composed of two types of losses: the generative adversarial network (GAN) loss as shown in formulas (1)-(2), and the cycle consistency loss (Cycle Loss) as shown in formulas (3)-(5): (1) (2) (3) (4) (5) By combining the above two types of GAN loss functions, the goal is to train a Generator A to generate a de-glare picture that fuses multi-modal feature discrimination features, achieving image-level fusion; Step 5: a picture quality predictor is trained by using the pseudo label obtained by the strong light label predictor, the overall quality of the picture is predicted according to the strong light interference of different modes, the quality adjustment factors α and β of the corresponding visible light and near-infrared modes are obtained, and the process of multi-modal feature fusion is dynamically adjusted in the adaptive fusion module; Step 6: an interactive fusion module with strong light perception is designed, and the feature fusion process is dynamically adjusted according to the image quality for feature-level fusion.

2. The cycle consistency based multi-spectral vehicle re-identification method under strong light according to claim 1, characterized in that, The method for judging whether the input image is polluted by strong light and marking the image polluted by strong light with a strong light false label by using the strong light label predictor in step 3 is specifically as follows: since the thermal infrared spectrum is immune to strong light, only the strong light false label of the visible light and near infrared spectrum is calculated; for the two spectra of visible light and near infrared which are easily interfered by strong light, since the pixel value of the local area of the light spot polluted image is high, according to the digital image features of the input image, the histogram of the image can be obtained, and the number of pixels in the image is counted to determine whether the image is affected by strong light; according to the statistical information of the image itself, the distribution of the pixel value in the R channel can be obtained, the histogram shows that the pixel quantity is distributed between 0 and 255, then the pixel points higher than a certain threshold θ are set as high value pixels, the proportion of the high value pixels and all the pixels in the image is calculated, then whether the proportion of the specific spectrum is greater than the threshold set as 1.5% is determined, so that whether the spectrum is polluted by the light spot is finally determined.

3. The cycle consistency based multi-spectral vehicle re-identification method under strong light according to claim 2, characterized in that, The method of training an image quality predictor using the pseudo-labels obtained by the strong light label predictor in step 5 is as follows: the features f R , f N extracted from ResNet50 are input into a prediction subnetwork containing multiple convolutional layers and a classifier, and the output is the strong light influence factors a, b of the RGB spectrum and the NI spectrum; the pseudo-labels label f are used as a supervision signal to constrain the feature extraction of the prediction subnetwork, which contains multiple convolutional layers, a pooling layer, a fully connected layer, and a classifier; and the subnetwork is trained using the binary cross-entropy loss formula: .

4. The cycle consistency based multi-spectral vehicle re-identification method under strong light according to claim 3, characterized in that, The method for dynamically adjusting the feature fusion process according to the image quality to perform feature level fusion by designing the strong light aware interactive fusion module in step 6 is specifically as follows: The strong light factors α and β of the RGB and NI spectrum are obtained from the predictor network, the feature interaction relationship between different spectral branches is adaptively adjusted according to the degree of influence of the picture by strong light in the following way, and the feature output is shown in formulas (7), (8) and (9). (7) (8) (9) In the interaction feature f rn , f rt , f nt Residual connection is performed, and the multi-branch feature f R , f N , f T Participate in feature fusion to establish the relationship between different modalities; the multi-branch feature of the interaction fusion module is respectively input into the classifier, and cross entropy loss and ternary loss are used to constrain feature learning.

Images