Method and system for vehicle image re-identification and computer storage medium

Abstract

The embodiment of the invention discloses a method and system for vehicle image re-identification and a computer storage medium, and the method for vehicle image re-identification comprises the steps: inputting a preprocessed vehicle image into a first feature network, and obtaining a plurality of first feature maps of different levels and different scales; utilizing a second feature network to perform feature fusion on a part of the first feature map output by the first feature network to generate a second feature map; using a third feature network to perform cross-orthogonal regularization on a part of the first feature map output by the first feature network to generate a third feature map; and carrying out splicing operation on the first feature map, the second feature map and the third feature map so as to carry out vehicle image re-identification by utilizing the feature maps obtained by splicing.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer vision technology, and more particularly, to a method, system, and computer storage medium for vehicle image re-identification. Background Art

[0002] Vehicle Re-identification (VRI) is a technology that can search for a specific vehicle across non-overlapping areas. It can find and track vehicles monitored by devices across the entire network, enabling the tracking and retrieval of a specific target vehicle across regions.

[0003] Early vehicle image re-identification methods were mainly based on sensor methods, such as infrared sensors, ultrasonic sensors, and three-dimensional magnetic sensors. However, since this method requires the installation of a large amount of hardware, the experimental environment is harsh, difficult to reproduce, and the recognition rate is low.

[0004] Because the license plate number is a unique credential for a vehicle, license plate re-identification became the mainstream solution for a long time. By comparing the captured license plate information with the vehicle information in the database, it is determined whether they belong to the same vehicle label. This method has high recognition accuracy in ideal scenarios with sufficient lighting, no obstructions, and good weather conditions. However, it has significant limitations in complex traffic conditions. For example, when faced with obscured license plates, inability to capture license plates due to inappropriate camera angles, unclear license plate images due to low camera resolution, and unlicensed vehicles, effective recognition is impossible, requiring a large amount of manual assistance, which greatly reduces work efficiency. It is even more incapable of tasks with complex environments and high requirements for timeliness and accuracy.

[0005] Methods based on artificially designed features do not require special hardware and are highly interpretable, but they have high time complexity, and the recognition rate often depends on the researcher's prior knowledge and feature selection. They have weak generalization capabilities and cannot effectively improve vehicle re-identification accuracy. Summary of the Invention

[0006] In view of this, the embodiments of this specification provide a method, system and computer storage medium for vehicle image re-identification, which are used to solve the problems in the existing technology that vehicle image re-identification methods cannot adapt to drastic changes in vehicle viewing angles and cannot solve the low resolution of vehicle images.

[0007] The embodiments of this specification adopt the following technical solutions:

[0008] An embodiment of this specification provides a method for vehicle image re-identification, the method for vehicle image re-identification comprising:

[0009] Inputting the preprocessed vehicle image into the first feature network to obtain multiple first feature maps of different levels and scales;

[0010] Using a second feature network, performing feature fusion on part of the first feature map output by the first feature network to generate a second feature map;

[0011] Using a third feature network, cross-orthogonal regularization is performed on a portion of the first feature map output by the first feature network to generate a third feature map;

[0012] A splicing operation is performed on the first feature map, the second feature map, and the third feature map, so as to use the spliced ​​feature map to perform vehicle image re-identification.

[0013] The embodiments of this specification further provide a system for vehicle image re-identification, the system for vehicle image re-identification comprising:

[0014] A first processing module inputs the preprocessed vehicle image into a first feature network to obtain a plurality of first feature maps of different levels and scales;

[0015] a feature fusion module, using a second feature network to perform feature fusion on part of the first feature map output by the first feature network to generate a second feature map;

[0016] A second processing module, using a third feature network, performs cross-orthogonal regularization on part of the first feature map output by the first feature network to generate a third feature map;

[0017] The splicing module performs a splicing operation on the first feature map, the second feature map, and the third feature map, so as to use the spliced ​​feature map to perform vehicle image re-identification.

[0018] The embodiments of this specification also provide a computer storage medium including a program for use in conjunction with an electronic device, the program being executable by a processor to perform the following steps:

[0019] Inputting the preprocessed vehicle image into the first feature network to obtain multiple first feature maps of different levels and scales;

[0020] Using a second feature network, performing feature fusion on part of the first feature map output by the first feature network to generate a second feature map;

[0021] Using a third feature network, cross-orthogonal regularization is performed on a portion of the first feature map output by the first feature network to generate a third feature map;

[0022] A splicing operation is performed on the first feature map, the second feature map, and the third feature map, so as to use the spliced ​​feature map to perform vehicle image re-identification.

[0023] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects:

[0024] By inputting the preprocessed vehicle image into the first feature network, multiple first feature maps of different levels and scales are obtained, and then the second feature network and the third feature network are used to perform different processing operations on part of the first feature maps to obtain different second feature maps and third feature maps. The first feature map, the second feature map and the third feature map are then spliced ​​together to use the spliced ​​feature maps for vehicle image re-identification.

[0025] In this way, different feature networks can be used to perform different processing on feature maps at different stages. The feature maps at different stages contain different detail features and semantic features, which fully utilizes the strengths of features at different stages. The final spliced ​​feature map contains feature representations with rich information, which can be used for vehicle re-identification tasks and improve the performance of vehicle re-identification. This feature fusion method can effectively improve the performance, recognition accuracy and robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the embodiments of this specification and constitute a part of the embodiments of this specification. The illustrative embodiments and descriptions of this specification are used to explain this application and do not constitute an improper limitation on this application. In the drawings:

[0027] Figure 1 A schematic flow chart of a vehicle image re-identification method in the prior art provided in the embodiments of this specification;

[0028] Figure 2 A flowchart of a method for vehicle image re-identification provided in an embodiment of this specification;

[0029] Figure 3 A schematic diagram of the structure of an aggregation network corresponding to a method for vehicle image re-identification provided in an embodiment of this specification;

[0030] Figure 4 A schematic diagram of the structure of an improved ResNet-50 backbone network in a method for vehicle image re-identification provided in an embodiment of this specification;

[0031] Figure 5 A schematic diagram of the structure of a feature aggregation branch in a method for vehicle image re-identification provided in an embodiment of this specification;

[0032] Figure 6 A schematic diagram of the structure of a feature pyramid extraction branch in a method for vehicle image re-identification provided in an embodiment of this specification;

[0033] Figure 7 A schematic diagram of the structure of a system for vehicle image re-identification provided in an embodiment of this specification;

[0034] Figure 8 A schematic diagram of the structure of a computer storage medium corresponding to a method for vehicle image re-identification provided in an embodiment of this specification. DETAILED DESCRIPTION

[0035] Vehicle re-identification (VRI) is a computer vision technology designed to identify and track the same vehicle across different camera views. It can be applied to urban surveillance systems, traffic management, security monitoring, and intelligent transportation. One of the key challenges in VRI is the variability of camera viewpoints and lighting conditions. Due to the varying positions and angles of different cameras, as well as varying lighting conditions, the appearance of the same vehicle can vary significantly across different cameras. Currently, most VRI methods tend to extract regional and component features to distinguish vehicles. These methods place high demands on the input images and require a large amount of detailed annotation information about vehicle components.

[0036] In the prior art, such as Figure 1 As shown, the specific process steps of vehicle image re-identification may include:

[0037] First, traffic surveillance video is decomposed frame by frame to obtain vehicle images. These images are then divided into a training dataset and a test dataset. The test dataset is further divided into a query set and a gallery set. When testing the performance of the deep learning-based vehicle re-identification model, images from the query set and gallery set are fed into the deep learning network. By extracting features, feature distances such as Euclidean distance and cosine distance are calculated between the query set images and the gallery set images to determine the feature similarity of the images. The retrieved images are then sorted by similarity, with smaller feature distances representing higher image similarity and a higher ranking. This ultimately results in a similarity ranking table for the retrieved images, with correctly matched images typically indicated by green boxes and incorrectly matched images by red boxes.

[0038] Although deep learning technology represented by convolutional neural network (CNN) has greatly promoted the development of natural language processing (NLP) and computer vision (CV) since its introduction, it has solved the problems of poor generalization performance and insufficient learning feature capabilities of traditional machine learning methods, and is very suitable for tasks such as target detection, semantic segmentation, and image classification. However, current methods generally have problems such as insufficient ability to extract discriminative features, inability to adapt to drastic changes in vehicle perspective, and inability to solve low vehicle image resolution.

[0039] Therefore, the embodiments of this specification provide a method, system, and computer storage medium for vehicle image re-identification. The preprocessed vehicle image is input into a first feature network to obtain multiple first feature maps of different levels and scales. The second feature network and the third feature network are then used to perform different processing operations on part of the first feature maps to obtain different second feature maps and third feature maps. The first feature map, the second feature map, and the third feature map are then spliced ​​together to use the spliced ​​feature maps for vehicle image re-identification.

[0040] In this way, different feature networks can be used to perform different processing on feature maps at different stages. The feature maps at different stages contain different detail features and semantic features, which fully utilizes the strengths of features at different stages. The final spliced ​​feature map contains feature representations with rich information, which can be used for vehicle re-identification tasks and improve the performance of vehicle re-identification. This feature fusion method can effectively improve the performance, recognition accuracy and robustness of the model.

[0041] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0042] The technical solutions provided by the embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0043] like Figure 2 , which is a flow chart of a method for vehicle image re-identification provided by an embodiment of this specification.

[0044] In the embodiment of this specification, the method for vehicle image re-identification may specifically include the following steps:

[0045] S201: Inputting the preprocessed vehicle image into a first feature network to obtain a plurality of first feature maps of different levels and scales;

[0046] S203: Using a second feature network, perform feature fusion on part of the first feature map output by the first feature network to generate a second feature map;

[0047] S205: Using a third feature network, cross-orthogonal regularize part of the first feature map output by the first feature network to generate a third feature map;

[0048] S207: performing a splicing operation on the first feature map, the second feature map, and the third feature map, so as to use the spliced ​​feature map to perform vehicle image re-identification.

[0049] In the embodiments of this specification, the method for vehicle image re-identification specifically adopts an aggregate network including a first feature network, a second feature network and a third feature network, and utilizes the aggregate network to perform the vehicle image re-identification task, wherein the aggregate network has been trained and can be directly used for the vehicle image re-identification task.

[0050] Specifically, the first feature network is the backbone network, which is used to extract features from the preprocessed vehicle image to obtain multiple first feature maps of different levels and scales; the second feature network and the third feature network are different branch networks, which are used to perform different feature processing operations on different first feature maps respectively, which can give full play to the advantages of features at different stages, strengthen the mutual fusion between features, and improve the accuracy and efficiency of vehicle image re-identification.

[0051] As an application embodiment of this specification, in step S201, the pre-processed vehicle image is input into the first feature network to obtain multiple first feature maps of different levels and scales, which may specifically include:

[0052] Inputting the preprocessed vehicle image into the initial stage module of the first feature network, so as to perform initial feature extraction on the vehicle image using the initial stage module to obtain an initial feature map;

[0053] Inputting the initial feature map into the first stage module of the first feature network and outputting the first stage feature map;

[0054] Inputting the first-stage feature map into the second-stage module of the first feature network and outputting the second-stage feature map;

[0055] Inputting the second-stage feature map into the third-stage module of the first feature network and outputting the third-stage feature map;

[0056] The third stage feature map is input into the fourth stage module of the first feature network, and a fourth stage feature map is output, where the first feature map includes the initial feature map, the first stage feature map, the second stage feature map, the third stage feature map and the fourth stage feature map.

[0057] In an embodiment of the present specification, the first feature network may specifically include an initial stage module, a first stage module, a second stage module, a third stage module, and a fourth stage module connected in sequence. The preprocessed vehicle image is input into the initial stage module, which outputs an initial feature map. The initial feature map is then input into the first stage module, which outputs a first stage feature map. Similarly, the second stage feature map, the third stage feature map, and the fourth stage feature map are output in sequence to obtain feature maps of different levels and scales.

[0058] In specific application scenarios, before inputting vehicle images into the aggregation network for processing, the vehicle images need to be preprocessed. The purpose is to convert the original images into a format suitable for deep learning model input, while retaining and enhancing information useful for vehicle identification as much as possible.

[0059] The following is the specific process of vehicle image preprocessing:

[0060] 1. Image resizing

[0061] Resizing: Resizing the input images to a fixed size required by the model, such as 224×224 pixels. This step ensures that all input images have the same size so that they can be fed into the aggregation network.

[0062] Maintaining aspect ratio: When resizing an image, it is usually necessary to maintain the aspect ratio of the original image to avoid image deformation. This can be achieved through padding, that is, after adjusting the image to the target size, fill it with black or gray pixels around it.

[0063] 2. Image Cropping

[0064] Cropping the region of interest: If the image contains irrelevant background information, you can retain the region of interest of the vehicle by cropping it. This step can reduce the impact of background noise on the model and improve recognition accuracy.

[0065] Random cropping: During training, random cropping can be used to increase data diversity and enhance the generalization ability of the model.

[0066] 3. Image Enhancement

[0067] Random rotation, translation, and scaling: By randomly rotating, translating, and scaling images, we increase the diversity of the data and enable the model to better adapt to different shooting conditions.

[0068] Lighting Compensation: Adjusts the brightness and contrast of an image to compensate for image differences under different lighting conditions.

[0069] Contrast Enhancement: Enhances the contrast of the image to make the vehicle's features more distinct.

[0070] 4. Color space conversion

[0071] RGB to Grayscale: If the model does not require color information, the RGB image can be converted to a grayscale image to reduce computational complexity.

[0072] Color normalization: Normalize the color channels of an image, for example, normalize the RGB values ​​to the range of [0, 1] or [-1, 1].

[0073] 5. Normalization

[0074] Pixel value normalization: Normalize the pixel values ​​of the image to a fixed range, such as [0, 1] or [-1, 1]. This step helps speed up the model training process and improve the stability of the model.

[0075] Global normalization: Normalize the image using the mean and standard deviation of the entire dataset to ensure that the distribution of input images is consistent.

[0076] Local normalization: Normalization is performed within a local area of ​​the image, such as within the neighborhood of each pixel, to enhance local features.

[0077] 6. Data Augmentation

[0078] Horizontal flip: Flip the image horizontally to increase data diversity.

[0079] Noise addition: Add a small amount of noise to the image to simulate the noise interference in actual shooting and enhance the robustness of the model.

[0080] Through these preprocessing steps, vehicle images can better adapt to the input requirements of deep learning models while retaining and enhancing information useful for vehicle identification. In practical applications, these preprocessing methods can be flexibly adjusted and combined according to specific needs.

[0081] Furthermore, the method may further include:

[0082] A batch normalization layer is embedded between the feature layer and the fully connected layer at the end of the first feature network.

[0083] In the embodiments of this specification, the batch normalization layer is also called the Batch Normalization (BN) layer. The BN layer normalizes each feature map by subtracting its mean and dividing it by its standard deviation in each mini-batch of data, thereby making the distribution of the feature maps more stable. This helps reduce internal covariate shift, that is, the problem of input distribution changes caused by parameter updates during training, thereby making training more stable and accelerating convergence.

[0084] Secondly, the BN layer can also help reduce the risk of model overfitting and improve the generalization ability of the model. The BN layer normalizes the input values ​​of each layer so that they fall within the sensitive area of ​​the activation function, thereby avoiding the problem of gradient disappearance.

[0085] Embedding the BN layer at the end of the first feature network can balance each dimension of the feature vector, making it easier for both triplet loss and cross entropy loss to converge, thereby optimizing the coordination loss function.

[0086] As an application embodiment of this specification, for step S203, using a second feature network to perform feature fusion on part of the first feature map output by the first feature network to generate a second feature map, including:

[0087] The second feature network is used to perform feature fusion on the first-stage feature map, the second-stage feature map, and the third-stage feature map to generate the second feature map that fuses shallow detail features and deep semantic features.

[0088] In the implementation of this specification, the second feature network receives the output feature maps from the first stage module, the second stage module and the third stage module of the backbone network, and fuses them through the feature aggregation module (FAM) to generate a feature map that combines shallow detail features and deep semantic features.

[0089] As an application embodiment of this specification, for step S205, using a third feature network, cross-orthogonal regularization is performed on part of the first feature map output by the first feature network to generate a third feature map, including:

[0090] Utilizing the third feature network, extracting the second-stage feature map and the third-stage feature map;

[0091] performing a unified operation on the scales of the second-stage feature map and the third-stage feature map;

[0092] A cross-orthogonal regularization operation is performed on the second-stage feature map and the third-stage feature map with unified scales to generate the third feature map containing multi-scale features.

[0093] In the implementation of this specification, the third feature network receives the output feature maps from the second-stage module and the third-stage module of the backbone network, and is composed of a position attention module and a channel attention module, which are used to enhance the recognition of features, and then unify the scales of the two different feature maps through the convolution layer to ensure that they have the same dimension.

[0094] Furthermore, the cross-orthogonal regularization module is used to enhance the diversity of features and improve the robustness of the model. The feature map after cross-orthogonal regularization is subjected to dimensionality reduction operation to reduce the dimension of the features.

[0095] As an application embodiment of this specification, in step S207, performing a splicing operation on the first feature map, the second feature map, and the third feature map to perform vehicle image re-identification using the spliced ​​feature map includes:

[0096] Converting the first feature map, the second feature map, and the third feature map into corresponding first feature vectors, second feature vectors, and third feature vectors, respectively;

[0097] A splicing operation is performed on the first feature vector, the second feature vector, and the third feature vector to obtain a spliced ​​target feature vector, so as to use the target feature vector to perform vehicle image re-identification.

[0098] Furthermore, converting the first feature map, the second feature map, and the third feature map into corresponding first feature vectors, second feature vectors, and third feature vectors, respectively, includes:

[0099] Converting the fourth-stage feature map into the first feature vector through global average pooling and batch normalization layers;

[0100] Convert the second feature map into the second feature vector through global average pooling;

[0101] Performing a dimensionality reduction operation on the third feature map to obtain the third feature vector.

[0102] Furthermore, performing a splicing operation on the first eigenvector, the second eigenvector, and the third eigenvector to obtain a spliced ​​target eigenvector includes:

[0103] The first eigenvector, the second eigenvector, and the third eigenvector are concatenated along the channel dimension to obtain the target eigenvector of a higher dimension.

[0104] In the implementation of this specification, the fourth-stage feature map contains global semantic information, the second feature map integrates shallow detail features and deep semantic features, and the third feature map contains multi-scale features. The feature vectors corresponding to these three feature maps are spliced ​​together through a splicing operation to form a higher-dimensional feature vector, so that the aggregation network can make full use of feature information at different levels and scales to generate more effective feature representations, thereby improving the performance of vehicle re-identification.

[0105] An embodiment of this specification provides a method for vehicle image re-identification. The method inputs a preprocessed vehicle image into a first feature network to obtain multiple first feature maps of different levels and scales. The second feature network and the third feature network are then used to perform different processing operations on parts of the first feature maps to obtain different second feature maps and third feature maps. The first feature map, the second feature map, and the third feature map are then spliced ​​together to perform vehicle image re-identification using the spliced ​​feature maps.

[0106] In this way, different feature networks can be used to perform different processing on feature maps at different stages. The feature maps at different stages contain different detail features and semantic features, which fully utilizes the strengths of features at different stages. The final spliced ​​feature map contains feature representations with rich information, which can be used for vehicle re-identification tasks and improve the performance of vehicle re-identification. This feature fusion method can effectively improve the performance, recognition accuracy and robustness of the model.

[0107] It should be noted that the above-mentioned specific method for vehicle image re-identification is merely a specific application embodiment and does not limit the scope of the embodiments of this specification. Other specific embodiments may also be included, which will not be described in detail here.

[0108] Based on the same inventive concept, the embodiments of this specification also provide specific application embodiments of the above-mentioned method for vehicle image re-identification.

[0109] like Figure 3 , which is a schematic diagram of the structure of an aggregation network corresponding to a method for vehicle image re-identification provided by an embodiment of this specification.

[0110] In the embodiments of this specification, the idea of ​​feature pyramid is applied to the backbone network, and combined with the multi-head attention module in Transformer, a new three-branch feature aggregation network (TFAN) is proposed to make up for the shortcomings of existing models such as poor feature extraction ability and low re-identification accuracy.

[0111] like Figure 3As shown in the figure, the three-branch feature aggregation network is generally divided into three branches: the upper, middle, and lower branches, which respectively represent the feature aggregation branch, the improved ResNet-50 backbone network, and the feature pyramid extraction branch. The improved ResNet-50 backbone network is the first feature network, the feature aggregation branch is the second feature network, and the feature pyramid extraction branch is the third feature network.

[0112] Specifically, the improved ResNet-50 backbone network includes five stage modules: Stage0, Stage1, Stage2, Stage3, and Stage4, which correspond to the initial stage module, the first stage module, the second stage module, the third stage module, and the fourth stage module, respectively, and are used to extract feature maps of different levels and scales.

[0113] The resolution of the feature map output by Stage 0 (that is, the initial feature map) is 1 / 4 of the original image, and the feature information contained includes image details such as edges and textures.

[0114] The resolution of the feature map output by Stage 1 (that is, the feature map of the first stage) is 1 / 4 of the original image, and the feature information contained includes local features and partial semantic information of the image;

[0115] The resolution of the feature map output by Stage 2 (that is, the feature map of the second stage) is 1 / 8 of the original image. The feature information contained includes more semantic information while retaining some detailed features.

[0116] The resolution of the feature map output by Stage 3 (that is, the third stage feature map) is 1 / 16 of the original image, and the feature information contained includes the semantic information of the image, such as the shape and structure of the object;

[0117] The resolution of the feature map output by Stage 4 (that is, the fourth stage feature map) is 1 / 32 of the original image, and the feature information contained therein includes global semantic information, which is used for high-level feature extraction and classification.

[0118] In addition, it should be noted that in order to obtain a larger-sized output feature map, the embodiment of this specification changes the Stride of Stage 4 to 1.

[0119] The improved ResNet-50 backbone network also includes the following modules:

[0120] Attention modules AM1 and AM2 are inserted after Stage2 and Stage3 respectively to enhance the recognition of features.

[0121] The GAP (global average pooling) module performs a global average pooling operation after Stage 4 to convert the feature map output by Stage 4 into a feature vector.

[0122] BN (batch normalization layer), embedded between GAP and the fully connected layer, is used to normalize the feature vector, balance the feature dimension, promote the convergence of the loss function, and optimize the coordination of triple loss and cross entropy loss in the backbone network.

[0123] Cross Entropy Loss module, a loss function for classification tasks.

[0124] The Triplet Loss module is used to measure the loss function of learning and optimize the similarity of features.

[0125] like Figure 4 , which is a schematic diagram of the structure of an improved ResNet-50 backbone network in a method for vehicle image re-identification provided in an embodiment of this specification.

[0126] The BN layer is embedded between the feature layer and the fully connected layer at the end of the network. Since the BN layer will aggravate the oscillation of the triple loss and promote the convergence of the cross entropy loss, the feature f t After the BN layer, the normalized feature vector f is obtained i , balances each dimension of the feature vector, making it easier for both loss functions to converge. During the training phase, f t and f i They are used to calculate triplet loss and cross entropy loss respectively, and Kaiming initialization is used to initialize the fully connected layer.

[0127] Furthermore, the feature aggregation branch is composed of a stack of feature aggregation modules, through which the output features of Stage 1, Stage 2, and Stage 3 are cumulatively matched and fused with the feature maps of the next stage, and shallow detail features are used as prior information to guide the generation of deep semantic features.

[0128] The feature aggregation branch specifically includes the following modules:

[0129] FAM (Feature Aggregation Module): stacked in networks of different depths, using shallow detail features as prior information to guide the generation of deep semantic features.

[0130] Concatenation module: concatenates and fuses features from different stages.

[0131] Auxiliary Loss module: Auxiliary loss introduced in the feature aggregation process to optimize the training of the feature aggregation module.

[0132] like Figure 5 , which is a structural diagram of a feature aggregation branch in a method for vehicle image re-identification provided by an embodiment of this specification.

[0133] The Feature Aggregation Module (FAM) proposed in the embodiments of this specification can be inserted into networks of different depths, using high-resolution low-level detail information as a global prior to guide the generation of high-level semantic features.

[0134] Feature Z s First, it is passed to the multi-head self-attention layer. In each head, Z s Generate three projections in different dimensions to form and Here is the number of heads. Self-attention trains the association between Query and Key and uses the softmax function to normalize the obtained attention weights.

[0135] Furthermore, the feature pyramid extraction branch consists of an attention module and a cross-orthogonal regularization module, which extract the output feature maps of Stage 2 and Stage 3 respectively. First, the scales of the two different feature maps are unified through the attention module and the convolution layer, and then the feature diversity is enhanced through cross-orthogonal regularization. Finally, the dimension is reduced and the output features are spliced ​​and fused with the output features of the improved ResNet-50 backbone network to obtain the final output of the three-branch feature aggregation network.

[0136] Among them, the feature pyramid extraction branch specifically includes the following modules:

[0137] AM (Attention Module): Inserted after Stage 2 and Stage 3, it consists of a Position Attention Module (PAM) and a Channel Attention Module (CAM) to enhance the recognition of features.

[0138] Cross-orthogonal regularization module: Applied in the feature pyramid extraction branch to obtain more diverse features.

[0139] Horizontal convolution module: It consists of a 1×1 convolution layer, a batch normalization layer, and a ReLU activation function, and is responsible for converting feature maps of different dimensions into a unified 256-dimensional model.

[0140] Dimensionality reduction convolution module: It consists of a 3×3 convolution layer, a batch normalization layer, and a ReLU activation function to aggregate features of different scales.

[0141] Cross-scale connection: Top-down connections are achieved through neighboring interpolation, and bottom-up connections are achieved through maximum pooling and downsampling.

[0142] Average Pooling module: The obtained feature vector is divided into multiple parts through average pooling.

[0143] Dim Reduction module: compresses the feature vector to 256 dimensions through a dimensionality reduction layer.

[0144] like Figure 6 , which is a structural diagram of a feature pyramid extraction branch in a method for vehicle image re-identification provided by an embodiment of this specification.

[0145] The self-attention mechanism leverages the correlation between features to help the model obtain more discernible features, effectively improving the model's generalization capabilities. The difficulty in vehicle re-identification lies in identifying highly similar features in regions between vehicles with different IDs. Introducing the self-attention mechanism allows for more targeted processing of similar features. Here, self-attention modules are inserted after Stage 2 of the backbone network and after the first feature pyramid branch. These self-attention modules consist of a Position Attention Module (PAM) and a Channel Attention Module (CAM).

[0146] The feature pyramid branch mainly includes a horizontal convolution module and a dimensionality reduction convolution module. The horizontal convolution module mainly consists of a convolution layer with a convolution kernel size of 1×1, a batch normalization layer, and a ReLU activation function. It is responsible for converting feature maps of different dimensions to a uniform size of 256.

[0147] The dimensionality reduction convolution module aggregates features of different scales and primarily consists of a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function. Furthermore, two cross-scale connections exist between feature maps of different sizes. The top-down connection is implemented through neighboring interpolation to increase the size of the feature map, while the bottom-up connection is implemented through 2×2 max pooling and downsampling.

[0148] In addition, the cross-connection downsampling operation in the feature pyramid branch is similar to the residual structure in ResNet, which realizes the cascade between different feature maps. Finally, the N 1024-dimensional feature vectors obtained are divided into And through the dimensionality reduction layer Compressed into N 256-dimensional feature vectors To optimize the classification loss on this branch during the training phase.

[0149] Because feature maps at different stages contain varying levels of detail and semantic features, TFAN uses a feature pyramid branch to extract detailed information from the shallow network and semantic information from the deep network for fusion. It also utilizes a feature aggregation branch to use shallow features as priors to guide the generation of deep features, leveraging the strengths of features at different stages. By combining the feature pyramid with a multi-head self-attention mechanism, the accuracy and robustness of vehicle re-identification are effectively improved. Building on deep learning, this model innovates and optimizes existing methods, achieving particularly strong performance when processing vehicle images in complex traffic scenarios.

[0150] The specific implementation process of the embodiments of this specification can refer to the corresponding implementation steps of the above embodiments, which will not be repeated here.

[0151] Based on the same inventive concept, the embodiment of this specification also provides a system for vehicle image re-identification. Figure 7 FIG. 1 is a schematic diagram of the structure of a system for vehicle image re-identification provided in an embodiment of this specification.

[0152] The system for vehicle image re-identification may specifically include:

[0153] The first processing module 701 inputs the pre-processed vehicle image into the first feature network to obtain a plurality of first feature maps of different levels and scales;

[0154] A feature fusion module 702 uses a second feature network to perform feature fusion on part of the first feature map output by the first feature network to generate a second feature map;

[0155] The second processing module 703 uses a third feature network to perform cross-orthogonal regularization on part of the first feature map output by the first feature network to generate a third feature map;

[0156] The splicing module 704 performs a splicing operation on the first feature map, the second feature map, and the third feature map, so as to use the spliced ​​feature map to perform vehicle image re-identification.

[0157] based on Figure 7 The present specification also provides some specific implementation plans of the system, which are described below.

[0158] Furthermore, the preprocessed vehicle image is input into the first feature network to obtain multiple first feature maps of different levels and scales, including:

[0159] Inputting the preprocessed vehicle image into the initial stage module of the first feature network, so as to perform initial feature extraction on the vehicle image using the initial stage module to obtain an initial feature map;

[0160] Inputting the initial feature map into the first stage module of the first feature network and outputting the first stage feature map;

[0161] Inputting the first-stage feature map into the second-stage module of the first feature network and outputting the second-stage feature map;

[0162] Inputting the second-stage feature map into the third-stage module of the first feature network and outputting the third-stage feature map;

[0163] The third stage feature map is input into the fourth stage module of the first feature network, and a fourth stage feature map is output, where the first feature map includes the initial feature map, the first stage feature map, the second stage feature map, the third stage feature map and the fourth stage feature map.

[0164] Furthermore, the system further comprises:

[0165] A batch normalization layer is embedded between the feature layer and the fully connected layer at the end of the first feature network.

[0166] Furthermore, using a second feature network, performing feature fusion on part of the first feature map output by the first feature network to generate a second feature map, including:

[0167] The second feature network is used to perform feature fusion on the first-stage feature map, the second-stage feature map, and the third-stage feature map to generate the second feature map that fuses shallow detail features and deep semantic features.

[0168] Furthermore, using a third feature network, cross-orthogonal regularization is performed on part of the first feature map output by the first feature network to generate a third feature map, including:

[0169] Utilizing the third feature network, extracting the second-stage feature map and the third-stage feature map;

[0170] performing a unified operation on the scales of the second-stage feature map and the third-stage feature map;

[0171] A cross-orthogonal regularization operation is performed on the second-stage feature map and the third-stage feature map with unified scales to generate the third feature map containing multi-scale features.

[0172] Furthermore, performing a splicing operation on the first feature map, the second feature map, and the third feature map to perform vehicle image re-identification using the spliced ​​feature map includes:

[0173] Converting the first feature map, the second feature map, and the third feature map into corresponding first feature vectors, second feature vectors, and third feature vectors, respectively;

[0174] A splicing operation is performed on the first feature vector, the second feature vector, and the third feature vector to obtain a spliced ​​target feature vector, so as to use the target feature vector to perform vehicle image re-identification.

[0175] Furthermore, converting the first feature map, the second feature map, and the third feature map into corresponding first feature vectors, second feature vectors, and third feature vectors, respectively, includes:

[0176] Converting the fourth-stage feature map into the first feature vector through global average pooling and batch normalization layers;

[0177] Convert the second feature map into the second feature vector through global average pooling;

[0178] Performing a dimensionality reduction operation on the third feature map to obtain the third feature vector.

[0179] Furthermore, performing a splicing operation on the first eigenvector, the second eigenvector, and the third eigenvector to obtain a spliced ​​target eigenvector includes:

[0180] The first eigenvector, the second eigenvector, and the third eigenvector are concatenated along the channel dimension to obtain the target eigenvector of a higher dimension.

[0181] The embodiments of this specification provide a system for vehicle image re-identification. The system inputs a preprocessed vehicle image into a first feature network to obtain multiple first feature maps of different levels and scales. The system then uses a second feature network and a third feature network to perform different processing operations on parts of the first feature maps to obtain different second feature maps and third feature maps. The system then performs a splicing operation on the first, second, and third feature maps to perform vehicle image re-identification using the spliced ​​feature maps.

[0182] In this way, different feature networks can be used to perform different processing on feature maps at different stages. The feature maps at different stages contain different detail features and semantic features, which fully utilizes the strengths of features at different stages. The final spliced ​​feature map contains feature representations with rich information, which can be used for vehicle re-identification tasks and improve the performance of vehicle re-identification. This feature fusion method can effectively improve the performance, recognition accuracy and robustness of the model.

[0183] Based on the same inventive concept, an embodiment of this specification further provides an electronic device, including at least one processor and a memory, wherein the memory stores a program and is configured to execute the following steps by the at least one processor:

[0184] Inputting the preprocessed vehicle image into the first feature network to obtain multiple first feature maps of different levels and scales;

[0185] Using a second feature network, performing feature fusion on part of the first feature map output by the first feature network to generate a second feature map;

[0186] Using a third feature network, cross-orthogonal regularization is performed on a portion of the first feature map output by the first feature network to generate a third feature map;

[0187] A splicing operation is performed on the first feature map, the second feature map, and the third feature map, so as to use the spliced ​​feature map to perform vehicle image re-identification.

[0188] Among them, other functions of the processor can also refer to the contents recorded in the above embodiments, which will not be repeated here.

[0189] Based on the same inventive concept, an embodiment of this specification further provides a computer-readable storage medium, including a program for use in conjunction with an electronic device, which can be executed by a processor to perform the following steps:

[0190] Inputting the preprocessed vehicle image into the first feature network to obtain multiple first feature maps of different levels and scales;

[0191] Using a second feature network, performing feature fusion on part of the first feature map output by the first feature network to generate a second feature map;

[0192] Using a third feature network, cross-orthogonal regularization is performed on a portion of the first feature map output by the first feature network to generate a third feature map;

[0193] A splicing operation is performed on the first feature map, the second feature map, and the third feature map, so as to use the spliced ​​feature map to perform vehicle image re-identification.

[0194] Among them, other functions of the processor can also refer to the contents recorded in the above embodiments, which will not be repeated here.

[0195] like Figure 8 As shown, the embodiment of this specification also provides a structural schematic diagram of a computer storage medium.

[0196] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0197] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0198] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0199] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0200] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0201] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0202] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0203] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0204] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0205] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0206] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0207] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0208] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for vehicle image re-identification, characterized in that: The method for vehicle image re-identification comprises: Inputting the preprocessed vehicle image into the first feature network to obtain multiple first feature maps of different levels and scales; Using a second feature network, performing feature fusion on part of the first feature map output by the first feature network to generate a second feature map; Using a third feature network, cross-orthogonal regularization is performed on a portion of the first feature map output by the first feature network to generate a third feature map; A splicing operation is performed on the first feature map, the second feature map, and the third feature map, so as to use the spliced ​​feature map to perform vehicle image re-identification.

2. The method according to claim 1, wherein The preprocessed vehicle image is input into the first feature network to obtain multiple first feature maps of different levels and scales, including: Inputting the preprocessed vehicle image into the initial stage module of the first feature network, so as to perform initial feature extraction on the vehicle image using the initial stage module to obtain an initial feature map; Inputting the initial feature map into the first stage module of the first feature network and outputting the first stage feature map; Inputting the first-stage feature map into the second-stage module of the first feature network and outputting the second-stage feature map; Inputting the second-stage feature map into the third-stage module of the first feature network and outputting the third-stage feature map; The third stage feature map is input into the fourth stage module of the first feature network, and a fourth stage feature map is output, where the first feature map includes the initial feature map, the first stage feature map, the second stage feature map, the third stage feature map and the fourth stage feature map.

3. The method according to claim 2, wherein The method further comprises: A batch normalization layer is embedded between the feature layer and the fully connected layer at the end of the first feature network.

4. The method according to claim 2, wherein Using the second feature network, performing feature fusion on part of the first feature map output by the first feature network to generate a second feature map, including: The second feature network is used to perform feature fusion on the first-stage feature map, the second-stage feature map, and the third-stage feature map to generate the second feature map that fuses shallow detail features and deep semantic features.

5. The method according to claim 2, wherein Using a third feature network, cross-orthogonal regularization is performed on part of the first feature map output by the first feature network to generate a third feature map, including: Utilizing the third feature network, extracting the second-stage feature map and the third-stage feature map; performing a unified operation on the scales of the second-stage feature map and the third-stage feature map; A cross-orthogonal regularization operation is performed on the second-stage feature map and the third-stage feature map with unified scales to generate the third feature map containing multi-scale features.

6. The method according to claim 2, wherein Performing a splicing operation on the first feature map, the second feature map, and the third feature map to perform vehicle image re-identification using the spliced ​​feature map, including: Converting the first feature map, the second feature map, and the third feature map into corresponding first feature vectors, second feature vectors, and third feature vectors, respectively; A splicing operation is performed on the first feature vector, the second feature vector, and the third feature vector to obtain a spliced ​​target feature vector, so as to use the target feature vector to perform vehicle image re-identification.

7. The method according to claim 6, wherein Converting the first feature map, the second feature map, and the third feature map into corresponding first feature vectors, second feature vectors, and third feature vectors, respectively, includes: Converting the fourth-stage feature map into the first feature vector through global average pooling and batch normalization layers; Convert the second feature map into the second feature vector through global average pooling; Performing a dimensionality reduction operation on the third feature map to obtain the third feature vector.

8. The method according to claim 6, wherein Performing a splicing operation on the first eigenvector, the second eigenvector, and the third eigenvector to obtain a spliced ​​target eigenvector includes: The first eigenvector, the second eigenvector, and the third eigenvector are concatenated along the channel dimension to obtain the target eigenvector of a higher dimension.

9. A system for vehicle image re-identification, characterized in that: The system for vehicle image re-identification comprises: A first processing module inputs the preprocessed vehicle image into a first feature network to obtain a plurality of first feature maps of different levels and scales; a feature fusion module, using a second feature network to perform feature fusion on part of the first feature map output by the first feature network to generate a second feature map; A second processing module, using a third feature network, performs cross-orthogonal regularization on part of the first feature map output by the first feature network to generate a third feature map; The splicing module performs a splicing operation on the first feature map, the second feature map, and the third feature map, so as to use the spliced ​​feature map to perform vehicle image re-identification.

10. A computer storage medium comprising a program for use in conjunction with an electronic device, the program being executable by a processor to perform the following steps: Inputting the preprocessed vehicle image into the first feature network to obtain multiple first feature maps of different levels and scales; Using a second feature network, performing feature fusion on part of the first feature map output by the first feature network to generate a second feature map; Using a third feature network, cross-orthogonal regularization is performed on a portion of the first feature map output by the first feature network to generate a third feature map; A splicing operation is performed on the first feature map, the second feature map, and the third feature map, so as to use the spliced ​​feature map to perform vehicle image re-identification.

Images