A vehicle re-identification system and method integrating multiple attributes and local features
By integrating multi-attribute and local features in the vehicle re-identification system, using the invariant feature and direction information of perspective angles, the problem of difficult vehicle recognition at different perspectives is solved, and higher recognition accuracy and detailed feature extraction is achieved .
Patent Information
- Application Number
- CN202210233646.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-10
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2042-03-10
AI Technical Summary
The existing vehicle re-identification methods fail to fully utilize the invariant characteristics of the vehicle's color, model and brand, making it difficult for the same vehicle to accurately identify it from different perspectives.
A vehicle re-identification system that combines multi-attributes and local features is adopted, including a general feature extraction module, a global feature module, an attribute recognition module and a direction-oriented DropBlock module. The global and attribute features of the vehicle are extracted through the modified ResNet-50 network. , and use direction information to generate mask Mask for feature extraction, and combine cross entropy and TripletLoss loss function for training.
It effectively improves the accuracy of vehicle re-identification, can accurately identify vehicles from different perspectives, extract more robust re-identification features, and enhances the recognition ability of vehicle direction, brand, model and color.
Smart Images
Figure CN114677557B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a vehicle re-identification system and method integrating multiple attributes and local features, belonging to the field of information technology, in particular to the field of vision-based vehicle identification technology. Background Art
[0002] With the continuous development of the Internet of Things, electronic cameras are everywhere in the main roads of the city. Intelligent transportation systems use vehicle re-identification technology, based on vehicle images in traffic monitoring videos, using computer vision technology to identify and process, retrieve images of the same vehicle from the vehicle image library, reduce the workload of manual retrieval, and quickly locate the target vehicle, which is of great significance to combating vehicle-related violations.
[0003] Although many vehicle re-identification methods have emerged, there are still many problems. The feature expressions of the current mainstream methods are not comprehensive enough. Some methods use the global attention method to perform multi-branch feature extraction, such as the Chinese patent with application number 202110977958.6. This patent proposes a global attention network model for vehicle re-identification, including a backbone network, a local branch that divides the feature map into two parts, and two global branches with global attention modules; but this patent extracts and operates the global features of the image, and divides the local features into horizontal and vertical parts for processing. For fine-grained vehicle re-identification, it only relies on the attention model in the image classification model to extract features, lacks the processing of local detail features, does not take into account the particularity of the vehicle re-identification task, and does not fully utilize the color, model and other easy-to-distinguish attributes of the vehicle for the auxiliary role of vehicle re-identification.
[0004] Another example: the Chinese patent with application number 202110756083.7 provides a vehicle re-identification method and system based on multi-perspective and convolutional attention modules. By adding a convolutional attention module, the model can focus on the effective features of the vehicle and ignore invalid features, thereby improving the model's recognition ability; through the calculation of common surfaces in the local module, the model has a stronger perspective anti-interference ability, which solves the problem of the same vehicle being difficult to distinguish from different perspectives; and realizes the function of re-identifying vehicles with similar appearances and the same vehicle from different perspectives. However, the patent divides the vehicle into four perspectives: top, front, rear, and side. It requires a large amount of detailed annotation of the vehicle's image information, and the semantic segmentation network also requires additional network computing costs. In addition, the patent does not fully utilize the perspective invariance features of the vehicle, such as the color and model of the vehicle.
[0005] Therefore, how to effectively extract and utilize the perspective-invariant features of the vehicle such as color, model and brand to improve the accuracy of vehicle re-identification has become an urgent problem that needs to be solved in the field of vehicle recognition technology. Summary of the invention
[0006] In view of this, the purpose of the present invention is to provide a vehicle re-identification system and method that integrates multiple attributes and local features, extracts more robust and comprehensive re-identification features for the vehicle, and effectively solves the problem that the same vehicle is difficult to identify due to large differences under different viewing angles.
[0007] In order to achieve the above object, the present invention proposes a vehicle re-identification system integrating multiple attributes and local features, the system comprising the following modules:
[0008] General feature extraction module: The function of this module is to extract general features from vehicle images of a set size. The specific structure of this module is as follows: a modified Resnet-50 network is used as the network structure of the general feature extraction module. The vehicle image first undergoes a 64-channel 7*7 convolution operation with a step size of 2 to obtain a 64-channel 112*112 feature map. Then, it undergoes a 3*3 size, step size of 2 maximum pooling to obtain a 56*56 size feature map. The following are three stages: Combination layer, these three stage combination layers are the same as the original ResNet-50 residual network; the three stage combination layers are composed of 3, 4, and 6 bottleneck layers respectively, each bottleneck layer includes three convolution layers with convolution kernel sizes of 1*1, 3*3, and 1*1, respectively, to perform convolution operations on the processed image; in the first three stage combination layers, the size of the feature map is halved and the number of channels is doubled; however, the fourth stage combination layer is different from the original ResNet-50 residual network, and the fourth stage combination layer is not downsampled, that is, the size of the feature map is not halved; finally, 1024 channels and a feature map of size 14*14 are obtained as the input of subsequent modules;
[0009] Global feature module: The function of this module is to extract global features of the global information of the vehicle based on the global features extracted by the general feature extraction module. The specific structure of this module is: it is composed of a global average pooling layer, a batch normalization layer BN and a fully connected layer connected in sequence. This module takes the 1024-channel 14*14 feature map obtained from the general feature extraction module as input, first passes through the global average pooling layer to obtain a 1024*1 feature vector, which is the global feature fg of the vehicle image; then fg is processed by the batch normalization layer BN to obtain a 1024*1 feature vector, and finally connected to a 1024*N fully connected layer, where N is the number of vehicle categories.
[0010] Attribute recognition module: The function of this module is to identify the attribute information of the vehicle, including the vehicle direction, brand, model and color, generate the corresponding mask according to the vehicle direction information, and extract the attribute features of the vehicle; the specific structure of this module is: it is composed of a Bottleneck layer connecting four attribute branches respectively, wherein each attribute branch is sequentially connected by a global average pooling layer and a fully connected layer; this module takes the 1024-channel 14*14 feature map obtained from the general feature extraction module as input, and obtains a 1024-channel 14*14 feature map after a Bottleneck layer, and then inputs the feature map into the four attribute branches; the vehicle direction attribute branch obtains a 1024*14 feature map after a global average pooling layer. The feature vector of the vehicle direction attribute is obtained by passing through a global average pooling layer, which is the feature vector of the vehicle direction attribute Fatto. Then, the feature vector is connected to a 1024*M1 fully connected layer, where M1 is the number of vehicle direction attributes. The brand attribute branch is subjected to a global average pooling layer to obtain a 1024*1 feature vector, which is the brand attribute feature vector Fattm. Then, the feature vector is connected to a 1024*M2 fully connected layer, where M2 is the number of brand attributes. The vehicle model attribute branch is subjected to a global average pooling layer to obtain a 1024*1 feature vector, which is the vehicle model attribute feature vector Fattt. Then, the feature vector is connected to a 1024*M3 fully connected layer, where M3 is the number of vehicle model attributes. The vehicle color attribute branch is subjected to a global average pooling layer to obtain a 1024*1 feature vector, which is the vehicle color attribute feature vector Fattc. Then, the feature vector is connected to a 1024*M4 fully connected layer, where M4 is the number of vehicle color attributes.
[0011] Direction-oriented DropBlock module: The function of this module is to combine the vehicle direction information, perform a mask operation, and extract the characteristics of the vehicle's direction-oriented DropBlock; the specific structure of this module is: it is composed of a Bottleneck layer, a global average pooling layer GAP, a batch normalization BN layer and a fully connected layer connected in sequence; this module takes the 1024-channel 14*14 feature map obtained from the general feature extraction module as input, and first passes through a Bottleneck layer to obtain a 1024-channel 14*14 feature map; then the 14*14-sized mask Mask matrix generated by the attribute recognition module is masked with the feature map obtained above, and then passes through a global average pooling layer to obtain a 1024*1 feature vector, which is the feature vector fl of the direction-oriented DropBlock, and the feature vector is processed by a batch normalization BN layer to obtain a 1024*1 feature vector, and finally connected to a 1024*N fully connected layer, where N is the number of vehicle categories.
[0012] According to the set training set, the general feature extraction module, the global feature module and the direction-oriented DropBlock module are trained using the cross entropy loss function and the TripletLoss loss function; the attribute recognition module is trained using the cross entropy loss function.
[0013] Each module of the system is trained according to the back-propagation algorithm. In each training process, the global feature module, the direction-oriented DropBlock module and the attribute recognition module respectively generate the update amount of each network parameter in the system, and then the update amount of each network parameter is accumulated for an update.
[0014] The vehicle direction information refers to the information after the vehicle direction is discretized, specifically: the vehicle head direction is taken as the positive direction and the first direction of the x-axis in the Cartesian coordinate system, and is divided into eight directions at intervals of 45°.
[0015] The specific content of generating the corresponding mask Mask according to the vehicle direction information is: first generate a matrix of a set size with all 1s, take the center of the matrix as the origin, make a straight line through the origin according to the vehicle direction, and then randomly translate the straight line up and down by -3 to +3 units, set the values of the matrix positions through which the straight line passes to 0, and the final 01 matrix is the mask Mask matrix; the set size refers to the size of the feature map obtained after the bottleneck layer in the direction-oriented DropBlock module.
[0016] The specific content of the mask operation is: the feature map obtained after the bottleneck layer in the direction-oriented DropBlock module is masked with the mask matrix, that is, the data of the feature map position corresponding to the point 1 in the mask matrix is retained, and the data of the feature map position corresponding to the point 0 in the mask matrix is set to 0.
[0017] The set training set refers to: a data set after preprocessing the collected vehicle image data; the preprocessing content is: processing the vehicle image size into 224*224, normalizing the three channels of the vehicle image, processing the vehicle image into multiple batches, each batch needs to contain positive examples, negative examples and anchor frames; each batch contains n vehicle categories, and each vehicle category contains k sample images; the n and k values are set according to the experimental equipment and network; for the same type of vehicle images, the images are positive examples of each other, and the vehicle images of different categories are negative examples of each other.
[0018] The present invention also proposes a vehicle re-identification method integrating multiple attributes and local features, the method comprising the following steps:
[0019] (1) Input the vehicle images in the image library into the vehicle re-identification system to obtain the re-identification features of the vehicle images in the image library, namely, global features, attribute features, and directional DropBlock features;
[0020] (2) Inputting the query vehicle image into the vehicle re-identification system, obtaining the re-identification features of the query vehicle image, namely: global features, attribute features, and directional DropBlock features;
[0021] (3) According to the obtained re-identification features, a set number of vehicle images having the greatest similarity with the vehicle image to be queried are found from the image library.
[0022] The calculation formula of the similarity described in step (3) is as follows:
[0023] S=α*|f g1 -f g2 | 2 +β*|f l1 -f l2 | 2 +γ1*|f attm1 -f attm2 | 2 +γ2 *|f attt1 -f attt2 | 2 +γ3*|f attc1 -f attc2 | 2
[0024] In the above formula, S represents the similarity, f g1 represents the global features of the vehicle image to be queried, f l1 The feature of the DropBlock representing the direction of the query vehicle image, f attm1 represents the brand attribute features of the vehicle image to be queried, f attt1 represents the vehicle model attribute features of the vehicle image to be queried, f attc1 represents the color attribute characteristics of the vehicle image to be queried; f g2 represents the global features of vehicle images in the image library, f l2 The feature of the DropBlock representing the orientation of the vehicle image in the image library, f attm2 represents the brand attribute features of the vehicle images in the image library, f attt2 represents the vehicle model attribute features of the vehicle images in the image library, f attc2represents the color attribute characteristics of the vehicle image in the image library; α, β, γ1, γ2, γ3 are weight parameters; α+β+ γ1+γ2+γ3=1.
[0025] The beneficial effects of the present invention are as follows: the present invention integrates the perspective invariant features such as the color, model and brand of the vehicle, and effectively solves the problem that the same vehicle is difficult to identify due to its large differences under different perspectives; in addition, the present invention utilizes the direction information of the vehicle to generate different vehicle masks for the different directions of the vehicle under different perspectives, and superimposes them with the original vehicle image to block part of the vehicle image for re-identification, so as to obtain more detailed features for re-identification, thereby extracting more robust re-identification features, and effectively improving the re-identification effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 It is a system module diagram of a vehicle re-identification system integrating multiple attributes and local features proposed by the present invention;
[0027] Figure 2 It is a schematic diagram of the structure of the general feature extraction module of the system proposed in the present invention;
[0028] Figure 3 It is a structural diagram of the global feature module of the system proposed in the present invention;
[0029] Figure 4 It is a structural diagram of the attribute recognition module of the system proposed in the present invention;
[0030] Figure 5 It is a schematic diagram of the structure of the direction-oriented DropBlock module of the system proposed in the present invention;
[0031] Figure 6 It is a flow chart of a vehicle re-identification method integrating multiple attributes and local features proposed by the present invention; DETAILED DESCRIPTION
[0032] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings.
[0033] See also Figure 1 , a vehicle re-identification system integrating multiple attributes and local features proposed by the present invention is introduced, and the system includes the following modules:
[0034] General feature extraction module: The function of this module is to extract general features from vehicle images of a set size;
[0035] See also Figure 2The specific structure of this module is as follows: the modified Resnet-50 network is used as the network structure of the general feature extraction module; the vehicle image first undergoes a 64-channel 7*7 convolution kernel with a step size of 2 to obtain a 64-channel 112*112 feature map; then it undergoes a 3*3 size, step size of 2 maximum pooling to obtain a 56*56 size feature map, followed by three stage combination layers, which are the same as the original ResNet-50 residual network; the three stage combination layers are composed of 3, 4, and 6 b The bottleneck layer consists of three convolution layers, each of which has three convolution kernels of 1*1, 3*3, and 1*1, to perform convolution operations on the processed image. In the first three stage combination layers, the feature map size is halved and the number of channels is doubled. However, the fourth stage combination layer is different from the original ResNet-50 residual network. The fourth stage combination layer does not downsample, that is, the feature map size is not halved. Finally, a 1024-channel, 14*14 feature map is obtained as the input of the subsequent modules.
[0036] Global feature module: The function of this module is to extract global features of vehicle global information based on the global features extracted by the general feature extraction module;
[0037] See also Figure 3 The specific structure of this module is: it is composed of a global average pooling layer GAP, a batch normalization layer BN and a fully connected layer FC connected in sequence; this module takes the 1024-channel 14*14 feature map obtained from the general feature extraction module as input, first passes through the global average pooling layer to obtain a 1024*1 feature vector, which is the global feature fg of the vehicle image; then fg is processed by the batch normalization layer BN to obtain a 1024*1 feature vector, and finally connected to a 1024*N fully connected layer, N is the number of vehicle categories;
[0038] Attribute recognition module: The function of this module is to recognize the attribute information of the vehicle, including the vehicle direction, brand, model and color, generate the corresponding mask according to the vehicle direction information, and extract the attribute characteristics of the vehicle;
[0039] See also Figure 4The specific structure of this module is: it is composed of a Bottleneck layer connecting four attribute branches respectively, wherein each attribute branch is connected in sequence by a global average pooling layer GAP and a fully connected layer FC; this module takes the 1024-channel 14*14 feature map obtained from the general feature extraction module as input, and obtains a 1024-channel 14*14 feature map after a Bottleneck layer, and then inputs the feature map into the four attribute branches; the vehicle direction attribute branch obtains a 1024*1 feature vector after a global average pooling layer, and the feature vector is the vehicle direction attribute feature vector Fatto, and then the feature vector is connected to a 1024*M1 fully connected layer, where M1 is the number of vehicle direction attributes; brand attribute The attribute branch is passed through the global average pooling layer to obtain a 1024*1 feature vector, which is the brand attribute feature vector Fattm, and then the feature vector is connected to a 1024*M2 fully connected layer, where M2 is the number of brand attributes; the vehicle model attribute branch is passed through the global average pooling layer to obtain a 1024*1 feature vector, which is the vehicle model attribute feature vector Fattt, and then the feature vector is connected to a 1024*M3 fully connected layer, where M3 is the number of vehicle model attributes; the vehicle color attribute branch is passed through the global average pooling layer to obtain a 1024*1 feature vector, which is the vehicle color attribute feature vector Fattc, and then the feature vector is connected to a 1024*M4 fully connected layer, where M4 is the number of vehicle color attributes;
[0040] Direction-oriented DropBlock module: This module combines the vehicle direction information, performs mask operation, and extracts the characteristics of the vehicle's direction-oriented DropBlock;
[0041] See also Figure 5 The specific structure of this module is: it is composed of a Bottleneck layer, a global average pooling layer GAP, a batch normalization BN layer and a fully connected layer connected in sequence; this module takes the 1024-channel 14*14 feature map obtained from the general feature extraction module as input, and first passes through a Bottleneck layer to obtain a 1024-channel 14*14 feature map; then the 14*14 mask Mask matrix generated by the attribute recognition module is masked with the feature map obtained above, and then passes through a global average pooling layer to obtain a 1024*1 feature vector, which is the feature vector fl of the direction-oriented DropBlock. The feature vector is processed by a batch normalization BN layer to obtain a 1024*1 feature vector, and finally connected to a 1024*N fully connected layer, where N is the number of vehicle categories.
[0042] According to the set training set, the general feature extraction module, the global feature module and the direction-oriented DropBlock module are trained using the cross entropy loss function and the TripletLoss (triplet loss) loss function; the attribute recognition module is trained using the cross entropy loss function.
[0043] Each module of the system is trained according to the back-propagation algorithm. In each training process, the global feature module, the direction-oriented DropBlock module and the attribute recognition module respectively generate the update amount of each network parameter in the system, and then the update amount of each network parameter is accumulated for an update.
[0044] The vehicle direction information refers to the information after the vehicle direction is discretized, specifically: the vehicle head direction is taken as the positive direction and the first direction of the x-axis in the Cartesian coordinate system, and is divided into eight directions at intervals of 45°.
[0045] The specific content of generating the corresponding mask Mask according to the vehicle direction information is: first generate a matrix of a set size with all 1s, take the center of the matrix as the origin, make a straight line through the origin according to the vehicle direction, and then randomly translate the straight line up and down by -3 to +3 units, set the values of the matrix positions through which the straight line passes to 0, and the final 01 matrix is the mask Mask matrix; the set size refers to the size of the feature map obtained after the bottleneck layer in the direction-oriented DropBlock module.
[0046] In the embodiment, the size of the mask is 14*14. For example, when the input is an image of the front of the vehicle in the x-axis direction, the first predicted direction is the x-axis, and then it is randomly moved up by one unit. The generated Mask matrix is counted from top to bottom, and the values at the 6th and 7th rows are 0, and the other values are 1, that is, a Mask matrix in the x-axis direction is obtained.
[0047] The specific content of the mask operation is: the feature map obtained after the bottleneck layer in the direction-oriented DropBlock module is masked with the mask matrix, that is, the data of the feature map position corresponding to the point 1 in the mask matrix is retained, and the data of the feature map position corresponding to the point 0 in the mask matrix is set to 0.
[0048] For example, the Mask matrix in the x-axis direction generated in the previous example is used for a mask operation with the feature map obtained after the bottleneck layer in the direction-oriented DropBlock module, that is, the values of the 6th and 7th rows of each feature map are set to 0, and the other parts remain unchanged.
[0049] The set training set refers to: a data set after preprocessing the collected vehicle image data; the content of the preprocessing is: processing the vehicle image size into 224*224 size, normalizing the three channels of the vehicle image, processing the vehicle image into multiple batches, each batch needs to contain positive examples, negative examples and anchor frames; each batch contains n vehicle categories, and each vehicle category contains k sample images; the n and k values are set according to the experimental equipment and network; for the same type of vehicle images, the images are positive examples of each other, and the vehicle images of different categories are negative examples of each other. In the embodiment, the main experimental equipment used by the inventor is 4 Nvidia Quadro RTX6000-24GB versions, and the Resnet-50 network is selected as the baseline of the general feature extraction module, then n=64, k=8.
[0050] See also Figure 6 The present invention also proposes a vehicle re-identification method integrating multiple attributes and local features, the method comprising the following steps:
[0051] (4) Inputting the vehicle images in the image library into the vehicle re-identification system, and obtaining the re-identification features of the vehicle images in the image library, namely: global features, attribute features, and features of the direction-oriented DropBlock;
[0052] (5) Inputting the query vehicle image into the vehicle re-identification system, obtaining the re-identification features of the query vehicle image, namely: global features, attribute features, and features of the direction-oriented DropBlock;
[0053] (6) Based on the obtained re-identification features, a set number of vehicle images having the greatest similarity with the vehicle image to be queried are found from the image library.
[0054] The calculation formula of the similarity described in step (3) is as follows:
[0055] S=α*|f g1 -f g2 | 2 +β*|f l1 -f l2 | 2 +γ1*|f attm1 -f attm2 | 2 +γ2 *|fattt1 -f attt2 | 2 +γ3*|f attc1 -f attc2 | 2
[0056] In the above formula, S represents the similarity, f g1 represents the global features of the vehicle image to be queried, f l1 The feature of the DropBlock representing the direction of the query vehicle image, f attm1 represents the brand attribute features of the vehicle image to be queried, f attt1 represents the vehicle model attribute features of the vehicle image to be queried, f attc1 represents the color attribute characteristics of the vehicle image to be queried; f g2 represents the global features of vehicle images in the image library, f l2 The feature of the DropBlock representing the orientation of the vehicle image in the image library, f attm2 represents the brand attribute features of the vehicle images in the image library, f attt2 represents the vehicle model attribute features of the vehicle images in the image library, f attc2 represents the color attribute characteristics of the vehicle images in the image library; α, β, γ1, γ2, γ3 are weight parameters; α+β+ γ1+γ2+γ3=1. In the embodiment, the inventor has obtained relatively good experimental results after multiple experiments at α=0.5, β=0.3, γ1=0.05, γ2=0.1, γ3=0.05.
[0057] The inventors have conducted a large number of experiments on the system and method of the present invention and obtained good experimental results, which shows that the system and method of the present invention are effective and feasible.
Claims
1. A vehicle re-identification system integrating multiple attributes and local features, characterized by: The system includes the following modules: General feature extraction module: The function of this module is to extract general features from vehicle images of a set size. The specific structure of this module is as follows: a modified Resnet-50 network is used as the network structure of the general feature extraction module. The vehicle image first undergoes a 64-channel 7*7 convolution kernel with a step size of 2 to obtain a 64-channel 112*112 feature map. Then, it undergoes a 3*3 maximum pooling with a step size of 2 to obtain a 56*56 feature map. Next, there are three stage combination layers, which are the same as the original ResNet-50 residual network. The first stage combination layer contains 3 bottles. The neck layer, the second stage combination layer contains 4 bottleneck layers, and the third stage combination layer contains 6 bottleneck layers; each bottleneck layer includes three convolution layers with kernel sizes of 1*1, 3*3, and 1*1, respectively, to perform convolution operations on the processed image; in the first three stage combination layers, the feature map size is halved and the number of channels is doubled; however, the fourth stage combination layer is different from the original ResNet-50 residual network, and the fourth stage combination layer does not downsample, that is, the feature map size is not halved; finally, 1024 channels and a feature map of size 14*14 are obtained as the input of subsequent modules; Global feature module: The function of this module is to extract global features of the global information of the vehicle based on the global features extracted by the general feature extraction module. The specific structure of this module is: it is composed of a global average pooling layer, a batch normalization layer BN and a fully connected layer connected in sequence. This module takes the 1024-channel 14*14 feature map obtained from the general feature extraction module as input, first passes through the global average pooling layer to obtain a 1024*1 feature vector, which is the global feature fg of the vehicle image; then fg is processed by the batch normalization layer BN to obtain a 1024*1 feature vector, and finally connected to a 1024*N fully connected layer, where N is the number of vehicle categories. Attribute recognition module: The function of this module is to identify the attribute information of the vehicle, including the vehicle direction, brand, model and color, generate the corresponding mask according to the vehicle direction information, and extract the attribute features of the vehicle; the specific structure of this module is: it is composed of a Bottleneck layer connecting four attribute branches respectively, wherein each attribute branch is sequentially connected by a global average pooling layer and a fully connected layer; this module takes the 1024-channel 14*14 feature map obtained from the general feature extraction module as input, and obtains a 1024-channel 14*14 feature map after a Bottleneck layer, and then inputs the feature map into the four attribute branches; the vehicle direction attribute branch obtains a 1024*1 feature vector after a global average pooling layer, which is the vehicle direction attribute feature vector Fatto, and then the feature vector Connect a 1024*M1 fully connected layer, where M1 is the number of vehicle direction attributes; the brand attribute branch passes through the global average pooling layer to obtain a 1024*1 feature vector, which is the brand attribute feature vector Fattm, and then connect the feature vector to a 1024*M2 fully connected layer, where M2 is the number of brand attributes; the vehicle model attribute branch passes through the global average pooling layer to obtain a 1024*1 feature vector, which is the vehicle model attribute feature vector Fattt, and then connect the feature vector to a 1024*M3 fully connected layer, where M3 is the number of vehicle model attributes; the vehicle color attribute branch passes through the global average pooling layer to obtain a 1024*1 feature vector, which is the vehicle color attribute feature vector Fattc, and then connect the feature vector to a 1024*M4 fully connected layer, where M4 is the number of vehicle color attributes; Direction-oriented DropBlock module: The function of this module is to combine the vehicle direction information, perform a mask operation, and extract the characteristics of the vehicle's direction-oriented DropBlock; the specific structure of this module is: it is composed of a Bottleneck layer, a global average pooling layer, a batch normalization BN layer and a fully connected layer connected in sequence; this module takes the 1024-channel 14*14 feature map obtained from the general feature extraction module as input, and first passes through a Bottleneck layer to obtain a 1024-channel 14*14 feature map; then the 14*14-sized mask Mask matrix generated by the attribute recognition module is masked with the feature map obtained above, and then passes through a global average pooling layer to obtain a 1024*1 feature vector, which is the feature vector fl of the direction-oriented DropBlock, and the feature vector is processed by a batch normalization BN layer to obtain a 1024*1 feature vector, and finally connected to a 1024*N fully connected layer, where N is the number of vehicle categories.
2. The vehicle re-identification system integrating multiple attributes and local features according to claim 1, characterized in that: According to the set training set, the general feature extraction module, the global feature module and the direction-oriented DropBlock module are trained using the cross entropy loss function and the TripletLoss loss function; the attribute recognition module is trained using the cross entropy loss function.
3. The vehicle re-identification system integrating multiple attributes and local features according to claim 2, characterized in that: Each module of the system is trained according to the back-propagation algorithm. In each training process, the global feature module, the direction-oriented DropBlock module and the attribute recognition module respectively generate the update amount of each network parameter in the system, and then the update amount of each network parameter is accumulated for an update.
4. The vehicle re-identification system integrating multiple attributes and local features according to claim 1, characterized in that: The vehicle direction information refers to the information after the vehicle direction is discretized, specifically: the vehicle head direction is taken as the positive direction and the first direction of the x-axis in the Cartesian coordinate system, and is divided into eight directions at intervals of 45°.
5. The vehicle re-identification system integrating multiple attributes and local features according to claim 1, characterized in that: The specific content of generating the corresponding mask Mask according to the vehicle direction information is: first generate a matrix of a set size with all 1s, take the center of the matrix as the origin, make a straight line through the origin according to the vehicle direction, and then randomly translate the straight line up and down by -3 to +3 units, set the values of the matrix positions through which the straight line passes to 0, and the final 01 matrix is the mask Mask matrix; the set size refers to the size of the feature map obtained after the bottleneck layer in the direction-oriented DropBlock module.
6. The vehicle re-identification system integrating multiple attributes and local features according to claim 1, characterized in that: The specific content of the mask operation is: the feature map obtained after the bottleneck layer in the direction-oriented DropBlock module is masked with the mask matrix, that is, the data of the feature map position corresponding to the point 1 in the mask matrix is retained, and the data of the feature map position corresponding to the point 0 in the mask matrix is set to 0.
7. The vehicle re-identification system integrating multiple attributes and local features according to claim 2, characterized in that: The set training set refers to: a data set after preprocessing the collected vehicle image data; the preprocessing content is: processing the vehicle image size into 224*224, normalizing the three channels of the vehicle image, processing the vehicle image into multiple batches, each batch needs to contain positive examples, negative examples and anchor frames; each batch contains n vehicle categories, and each vehicle category contains k sample images; the n and k values are set according to the experimental equipment and network; for the same type of vehicle images, the images are positive examples of each other, and the vehicle images of different categories are negative examples of each other.
8. A vehicle re-identification method integrating multiple attributes and local features, applied to the system according to any one of claims 1 to 7, characterized in that: The method comprises the following steps: (1) Input the vehicle images in the image library into the vehicle re-identification system to obtain the re-identification features of the vehicle images in the image library, namely, global features, attribute features, and directional DropBlock features; (2) Inputting the query vehicle image into the vehicle re-identification system, obtaining the re-identification features of the query vehicle image, namely: global features, attribute features, and directional DropBlock features; (3) According to the obtained re-identification features, a set number of vehicle images having the greatest similarity with the vehicle image to be queried are found from the image library.
9. The vehicle re-identification method integrating multiple attributes and local features according to claim 8, characterized in that: The calculation formula of the similarity described in step (3) is as follows: S=α*|f g1 -f g2 | 2 +β*|f l1 -f l2 | 2 +γ1*|f attm1 -f attm2 | 2 +γ2*|f attt1 -f attt2 | 2 +γ3*|f attc1 -f attc2 | 2 In the above formula, S represents the similarity, f g1 represents the global features of the vehicle image to be queried, f l1 The feature of the DropBlock representing the direction of the query vehicle image, f attm1 represents the brand attribute features of the vehicle image to be queried, f attt1 represents the vehicle model attribute features of the vehicle image to be queried, f attc1 Represents the color attribute features of the vehicle image to be queried; f g2 represents the global features of vehicle images in the image library, f l2 The feature of the DropBlock representing the orientation of the vehicle image in the image library, f attm2 represents the brand attribute features of the vehicle images in the image library, f attt2 represents the vehicle model attribute features of the vehicle images in the image library, f attc2 Represents the color attribute characteristics of the vehicle image in the image library; α, β, γ1, γ2, γ3 are weight parameters; α+β+γ1+γ2+γ3=1.