Feature fusion method applied to vehicle re-identification in foggy weather

Abstract

The invention discloses a feature fusion method applied to foggy vehicle re-identification. The method specifically comprises the following steps: inputting a foggy vehicle image to a public feature extraction module to extract public features; related defogged features are extracted from the public features by using a two-stage recovery method; pyramid enhancement is used for further processing, and defogging loss is obtained through calculation; performing further feature extraction on the public features by using a re-recognition network, extracting global and local features by using a convolutional block attention mechanism and a feature pyramid method, fusing the global and local features, and calculating to obtain triple loss and vehicle identity classification loss; and carrying out fusion processing on the three losses to obtain an overall loss function, and finally identifying images which are shot by other cameras and consistent with the input image in identity in the gallery set. According to the vehicle re-identification method, the re-identification effect is better under the foggy weather condition, and compared with an existing method, the vehicle re-identification method achieves more excellent performance in precision.

Description

Technical Field

[0001] The present invention relates to the field of re-identification technology in computer vision target detection, and in particular to a feature fusion method applied to vehicle re-identification in foggy weather. Background Art

[0002] Fog is one of the important weather conditions that lead to traffic accidents. Low visibility can limit the driver's field of vision and increase the risk of traffic accidents. With the rapid development of intelligent transportation systems and autonomous driving technologies, the research on vehicle re-identification technology in foggy weather has become particularly important. Vehicle re-identification in foggy weather is a challenging task. Traditional visual systems are easily disturbed in foggy weather, resulting in a significant reduction in the accuracy of vehicle detection and recognition. Especially in intelligent transportation systems, vehicle re-identification is a key link in achieving traffic flow monitoring, violation identification and accident warning, while the complexity and uncertainty in foggy weather further increase the technical difficulty. With the acceleration of urbanization and the sharp increase in the number of motor vehicles, traffic management and road safety are facing huge challenges. Under severe weather conditions such as foggy weather, the incidence of traffic accidents has increased significantly, posing a serious threat to the safety of people's lives and property. Therefore, how to achieve efficient and accurate vehicle re-identification in foggy weather and improve the reliability and safety of intelligent transportation systems is of great significance to protecting people's lives and property.

[0003] Early foggy vehicle re-identification methods mainly rely on traditional image enhancement techniques and hand-crafted feature extraction methods. Image enhancement methods, such as histogram equalization and contrast-limited adaptive histogram equalization (CLAHE), can improve the visibility of images to a certain extent, but often lead to the loss of image details and the amplification of noise, making it difficult to maintain stable performance in complex foggy environments. Hand-crafted feature extraction methods, such as scale-invariant feature transform (SIFT) and histogram of oriented gradients (HOG), show good recognition results in clear images, but in foggy conditions, the performance of these methods is significantly reduced due to reduced image contrast and blurred details. In addition to using traditional image processing techniques, some scholars have also tried to use physical models to estimate the concentration of fog and perform defogging, but these methods are not effective in dense or non-uniform fog. With the development of deep learning technology, vehicle re-identification methods based on convolutional neural networks have gradually become mainstream. These methods can automatically extract more discriminative features through end-to-end learning, significantly improving the performance of vehicle re-identification. However, in foggy environments, due to the degradation of image quality and blurred features, traditional convolutional neural network methods still face the problem that the re-identification accuracy is not enough to meet practical needs. Summary of the invention

[0004] In order to solve the problems of low accuracy and insufficient generalization ability of existing convolutional neural network vehicle re-identification methods under foggy conditions, the present invention combines the advantages of existing methods and designs a feature fusion method for vehicle re-identification in foggy weather; the overall framework of the method consists of two parts: a defogging network and a re-identification network: first, a foggy vehicle image to be re-identified is input, and the image is sent to a common feature extraction module to extract common features to be sent to the defogging network and the re-identification network, the defogging network adopts semi-supervised learning, and the re-identification network adopts supervised learning; secondly, the extracted common features are passed through the defogging network and the re-identification network to extract the key features of defogging and the key features of re-identification respectively, and a feature fusion module and an attention mechanism are introduced in the re-identification network to extract more detailed re-identification features; finally, stable re-identification features that are not affected by foggy weather are extracted, and the input image is re-identified. Task.

[0005] Specifically, the present invention provides a feature fusion method for vehicle re-identification in foggy weather, which mainly includes the following steps: S1. Input foggy vehicle images into the overall network framework, and extract the common features required for subsequent use through the common extraction module; S2: Use a two-stage recovery method on the common features extracted by S1 to enhance the accuracy and pertinence of feature representation and extract relevant features for defogging; S3: Use pyramid enhancement to enhance the defogging features obtained in S2 to form a richer and more detailed feature representation, and calculate the defogging loss L D ; S4: For the common features extracted by S1, use the re-identification network to extract features, use the features processed by the attention mechanism as global features, and use the multi-scale features extracted by the feature pyramid method as local features, fuse the global features with the local features, generate a feature map that retains rich local details and combines contextual information, and calculate the triplet loss L T and vehicle identity classification loss L ID ; S5. The losses obtained in S3 and S4 are integrated to obtain the overall loss function, and the weights and parameters of the common feature extraction module are changed using the overall loss function. Finally, the re-identification branch is used to identify images taken by other cameras that are consistent with the identity of the input image from the gallery set.

[0006] The common extraction module described in step S1 is composed of the first two stages of Resnet50 and is used to roughly extract the features of the image.

[0007] The two-stage restoration method described in step S2 is to first use the physical model to perform defogging, and then use the convolutional network to perform further detail optimization to obtain the feature vector required for defogging.

[0008] Step S3 L D According to the difference between supervised learning and unsupervised learning, it can be divided into two types: supervised learning L DFS , for unsupervised learning L DFU .

[0009] The attention mechanism described in step S4 is a convolutional block attention mechanism, which processes the features finally generated by Renset50. The multi-scale features are the features obtained after feature pyramid processing, and the input of the feature pyramid is the intermediate features generated by the first four stages of Resnet50.

[0010] The fusion processing described in step S5 refers to the process of using the three loss functions obtained from the two branches to perform weighted summation to obtain the overall loss function; using the overall loss function to change the parameters and weights of the common feature extraction module so that the network common feature extraction module can achieve the purpose of extracting stable features. Since the defogging branch adopts semi-supervised learning, there are two overall loss functions: L S , the formula is as follows: ; in, L T represents the triplet loss of the re-identification branch; L ID represents the classification loss of the re-identification branch; L DFS represents the dehazing branch loss in the supervised learning stage; λ 1 is the weight coefficient used to balance the loss of the defogging branch; The overall loss of the dehazing branch for unsupervised learning is L U , the formula is as follows: ; in, L DFU is the loss of the dehazing branch in the unsupervised learning stage; L T and L ID They are the triplet loss and classification loss of the re-identification branch respectively. When actually performing the re-identification task, the defogging branch does not participate in the re-identification process, and only the re-identification branch is used to complete the vehicle re-identification task.

[0011] Beneficial effects: A more accurate re-identification of target vehicles in foggy environments is achieved. The use of the present invention for foggy vehicle re-identification can help traffic management departments and autonomous driving systems to more efficiently identify and track target vehicles in adverse weather conditions, improve traffic safety management capabilities and environmental perception capabilities, and prevent traffic accidents caused by low visibility. The present invention can capture local details and global structural information of vehicles at different scales by designing a multi-scale feature extraction module and combining it with an attention mechanism. This strategy of fusing multi-scale local features with global features can effectively address the problems of blurred image details and low contrast in foggy environments, ensuring that the model can obtain reliable feature representations under different fog concentration conditions, thereby improving the accuracy of vehicle re-identification under foggy conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flow chart of a feature fusion method for vehicle re-identification in foggy weather according to the present invention; Figure 2 It is a network structure diagram of generating multi-scale local features by the feature pyramid of the present invention; Figure 3 This is a diagram of the re-identification results of the vehicle image in foggy weather according to the present invention, including the query image, the top ten re-identification results, and the correctness of the re-identification results. The green box indicates that the re-identification result is correct. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0014] like Figure 1 , Figure 2 , Figure 3 The present invention provides a feature fusion method for vehicle re-identification in foggy weather, comprising the following steps: S1. Input the foggy vehicle image to be re-identified into the overall network. After the common features are extracted by the common feature extraction module, it will pass through two branches, the defogging network branch and the re-identification network branch: the defogging network branch is responsible for defogging the foggy image; the re-identification branch is responsible for re-identifying the input image and finding other image data that are consistent with the input foggy vehicle image; S2, using a two-stage recovery method for the common features extracted by S1 to extract defogging related features; S3: Perform pyramid enhancement operation on the defogging features extracted by S2 to further enhance the defogging related features and obtain the defogging loss L D ; S4, using the subsequent three stages of Resnet50 to further process the common features extracted by S1, after feature extraction, use the attention mechanism to enhance the re-identification related features to obtain a stronger feature representation, and use it as a global feature; the four feature maps of different sizes extracted by the first four stages of Resnet50 are used as the input of the feature pyramid to obtain multi-scale local features; finally, the global features are spliced ​​with the local features, fused into the final re-identification features, and the triplet loss is calculated L T and the classification loss of vehicle identity L ID ; S5. Finally, the three losses obtained in S3 and S4 are fused to obtain the overall loss function, and the weights and parameters of the common feature extraction module are changed; and the re-identification branch is used to find vehicle images taken by other cameras that are consistent with the input foggy vehicle image.

[0015] All input images in step S1 are reshaped to 384×384 to facilitate subsequent unified feature extraction. The dehazing branch uses semi-supervised learning. This example uses the ResNet-50 model pre-trained for the ImageNet image classification task. The first two stages of the model are used as our common feature extraction module, and the feature maps of these two stages are saved so that they can be used as input for subsequent feature pyramids. The number of channels of the saved feature maps is 64 and 256 respectively.

[0016] In step S4, the feature maps generated by the last three stages of Resnet50 are saved accordingly, and the number of channels is 512, 1024 and 2048 respectively; the attention mechanism is a convolutional block attention mechanism, which combines the spatial attention mechanism and the channel attention mechanism; the feature pyramid uses the four feature maps generated by the first four stages of Resnet50 as input, and the multi-scale local feature generation process is as follows Figure 2 .

[0017] The fusion processing described in step S5 refers to the process of calculating the overall loss function by performing a weighted sum operation on the three losses obtained from the defogging branch and the re-identification branch during the training process; and subsequently using the overall loss function to make corresponding changes to the weights and parameters of the common feature extraction module so that the common feature extraction module can extract stable key features that are conducive to re-identification in vehicle images on foggy days.

[0018] In a specific implementation case of the present invention, the dataset used is the FVRID dataset, which is obtained by processing the two datasets VehicleID and VERI-Wild. The FVRID dataset contains two parts: real data and synthetic data. Each dataset includes a training set (Train Set), a probe set (Probe Set) and a gallery set (Gallery Set). The probe set contains a set of specific query samples, which are used to test whether the system can correctly find the corresponding matching items in the gallery set. In the training stage, for the vehicle re-identification task, the basic Resnet50 network is selected as the basic network architecture, and the initial weights are loaded from the ImageNet pre-trained model. As for the defogging branch, its parameters are initialized by the Kaiming normal initialization method. In order to avoid the overfitting problem that may be caused by the limited number of training samples, the data enhancement technology of horizontal flipping and random cropping is introduced in the training process. The entire model is trained for 120 rounds, the training batch size is set to 32, and a gradually warming learning rate scheduling mechanism is adopted: the initial learning rate is 1.09e-5, and it is increased to 1e-4 after the 10th epoch. In order to optimize the model, in the specific implementation case of the present invention, the optimizer used for training the model is the Adam optimizer, in which a learning rate decay coefficient of 0.6 is set, a learning rate of 0.0001, the deep learning framework used is Pytorch1.8, and the hyperparameters λ 1 is set to 1. In the training phase, both branches are involved, and the dehazing branch helps the re-identification branch to extract stable re-identification features; while in the testing phase, only the re-identification branch performs the re-identification task.

[0019] The corresponding data re-identification results are shown in the figure Figure 3 As shown in the figure, the first column of the image is the query image, and the following ten images are the top ten results of the method for re-identifying the input vehicle image. Compared with the previous methods, the present invention combines multi-scale feature fusion and global-local feature complementary mechanism, and has higher vehicle re-identification accuracy in foggy environment.

[0020] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A feature fusion method for vehicle re-identification in foggy weather, characterized by: The method comprises the following steps:

1. Input foggy vehicle images into the overall network framework, extract common features through the first two stages of Resnet50, and save the feature maps of these two stages; Second, use a two-stage recovery method on the common features extracted in step 1 to obtain the relevant features for defogging; 3. Use pyramid enhancement to enhance the defogging features obtained in step 2 to form a richer and more detailed defogging feature map, and calculate the defogging loss; Fourth, the common features extracted in step 1 are further extracted using the three stages of Resnet50, and the feature maps of these three stages are saved. The features processed by the convolutional block attention mechanism are used as global features, and the multi-scale features extracted using the feature pyramid method are used as local features. These two parts of features are fused to form the final re-identification feature map, and the triplet loss and vehicle identity classification loss are calculated; 5. The three losses obtained in step 3 and step 4 are fused to obtain the overall loss function, and the overall loss function is used to change the weights of the first two stages of Resnet50 in step 1. Finally, the re-identification branch is used to identify images taken by other cameras that have the same identity as the input image from the gallery set.

2. According to claim 1, a feature fusion method for vehicle re-identification in foggy weather, characterized in that: The first two stages of Resnet50 in step 1 serve as the common feature extraction module of the entire framework, and the number of channels of the two feature maps saved is 64 and 256.

3. According to claim 1, a feature fusion method for vehicle re-identification in foggy weather, characterized in that: The number of channels of the feature maps of the three stages saved in step 4 are 512, 1024, and 2048 respectively; the input of the convolutional block attention mechanism is the features extracted in the last stage of Resnet50; The feature pyramid takes the features extracted by the first four stages of Resnet50 as input.

4. According to claim 1, a feature fusion method for vehicle re-identification in foggy weather, characterized in that: The fusion processing described in step 5 refers to the process of calculating the overall loss function by performing a weighted sum operation using the three losses obtained in steps 3 and 4 during the training process.

Images