A vehicle re-identification method and device guided by camera topology map

Abstract

The present invention discloses a vehicle re-identification method and device guided by a camera topology map. The method comprises: constructing a training set to obtain vehicle feature representations; constructing a camera topology map based on the vehicle feature representations; constructing a topological relationship between the feature representations of any two vehicles based on the camera topology map and inputting the relationship into a graph convolutional network to obtain final aggregated features; fusing the final aggregated features with the vehicle feature representations, and inputting the fusion result into a fully connected layer for class prediction; constructing a target loss function, training the graph convolutional network, and stopping the training until the target loss function value is minimized to obtain a trained graph convolutional network; and performing vehicle re-identification using the trained graph convolutional network. The present invention has the advantages of improving the accuracy of re-identification.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and more particularly to a vehicle re-identification method and device guided by a camera topology map. Background Art

[0002] Vehicle re-identification (Re-ID) aims to retrieve images of vehicles of interest from a gallery of images captured by non-overlapping surveillance cameras. This is an active and challenging task that has attracted considerable attention due to its wide range of applications in areas such as social security, smart cities, and intelligent transportation. Despite its remarkable success, it still faces severe challenges such as intra-camera occlusion, cross-camera illumination, and viewpoint variations, which limit its application in complex real-world scenarios.

[0003] Existing technologies have proposed different approaches to address the three challenges mentioned above. Representative methods can be divided into three categories: 1) Viewpoint learning-based methods, which learn two metrics of similarity and difference of viewpoints in two feature spaces and propose a viewpoint-aware network (VANet) for vehicle re-identification; 2) Component learning-based methods, which propose a dual-path adaptive attention model to capture key points related to vehicle re-identification (AAVER) components; and 3) Path learning-based methods, which construct spatiotemporal constraints and optimize the matching results of vehicle re-identification, using spatiotemporal information as physical constraints to reduce the complexity of the matching algorithm. However, these works mainly focus on mining information within a single image, and therefore lack the interaction between different images.

[0004] In recent years, graph convolutional networks (GCNs) have become popular. They generalize the capabilities of convolutional neural networks (CNNs) by performing convolution operations on graph-structured data. Traditional GCN models are widely used in computer vision tasks, such as 1) pose estimation, where the Semantic Graph Convolutional Network (SemGCN) captures pose information such as local and global node relationships; 2) action recognition, where the Action Structure Graph Convolutional Network (AS-GCN) is proposed to extract useful spatial and temporal information for action recognition; 3) person re-identification, where a similarity-guided graph neural network is proposed to incorporate rich image library similarity information into the training process; and 4) vehicle re-identification, where a Parsing-Guided Cross-Component Reasoning Network (PCRNet) is proposed to learn discriminative feature representations and model the correlation between components. Vehicle re-identification based on graph convolutional networks has gradually become a research hotspot in the industry.

[0005] Chinese patent publication number CN112396027A discloses a vehicle re-identification method based on a graph convolutional neural network, comprising: constructing a network model for vehicle re-identification, using a convolutional neural network to extract global and local features of the vehicle image to be re-identified, using the graph convolutional neural network to obtain structured features, and using the structured features to calculate the loss function of the network model; training the network model based on the loss function; and using the graph convolutional neural network to mine the structured information between local features and local features, and between local features and global features, thereby obtaining a better and more comprehensive feature expression and improving the accuracy of vehicle re-identification. However, in complex camera system scenarios, where multiple different cameras are capturing images, the method does not consider the differences between the images captured by different cameras and the connections between adjacent cameras. As a result, the captured feature vectors cannot accurately represent vehicle information, and the accuracy of vehicle re-identification is low. Summary of the Invention

[0006] The technical problem to be solved by the present invention is how to improve the accuracy of vehicle re-identification in a scenario where multiple different cameras capture images.

[0007] The present invention solves the above technical problems through the following technical means: a vehicle re-identification method guided by a camera topology map, the method comprising:

[0008] Step 1: Build a training set and obtain vehicle feature representation;

[0009] Step 2: Construct a camera topology map based on vehicle feature representation;

[0010] Step 3: Based on the camera topology graph, the topological relationship between the feature representations of any two vehicles is constructed and input into the graph convolutional network to obtain the final aggregated features;

[0011] Step 4: Fuse the final aggregated features with the vehicle feature representation and input the fusion result into the fully connected layer for class prediction;

[0012] Step 5: Construct the target loss function and train the graph convolutional network until the target loss function value is minimized, and then stop training to obtain the trained graph convolutional network;

[0013] Step 6: Use the trained graph convolutional network to perform vehicle re-identification.

[0014] Beneficial effects: The present invention first constructs a training set to obtain vehicle feature representation, and also constructs a camera topology map to input the topological relationship into the graph convolutional network to obtain aggregated features, and then fuses the two features to obtain fused features. The class prediction results are finally obtained based on the fused features. The entire feature recognition process not only considers the original visual features, that is, the vehicle feature representation, but also obtains aggregated features based on the camera topology map. Therefore, when multiple different cameras capture images, it can characterize the differences in images captured by different cameras and the connections between adjacent cameras, so that the captured feature vectors can accurately express vehicle information, and the accuracy of vehicle re-identification is high.

[0015] Furthermore, the step 1 includes:

[0016] Constructing a training set Among them, x i represents the i-th image, N T Indicates the total number of images in the training set, y i Indicates its identity tag, Indicates its camera tag;

[0017] The training set is input into the vehicle representation model ResNet-50 to extract the vehicle feature representation, which is represented as: {h1,h2,...,h N}, h N Represents the feature representation of the Nth vehicle.

[0018] Furthermore, the step 2 includes:

[0019] According to the vehicle feature representation, different cameras are used as nodes, and edges are constructed based on the various relationships between cameras, thereby constructing a camera topology graph G = (V, E), where V represents the camera node, Indicates C T camera nodes, E is the edge set set in the camera topology graph, E={E system ,E position ,E orientation ,E individual}, E system ,E position ,E orientation ,E individual Represent the edge sets constructed based on the relationship between camera system, position, direction and identity respectively; then the camera topology graph based on camera system, position, direction and identity is represented as: G system ,G position ,G orientation ,G individual .

[0020] Furthermore, the step three includes:

[0021] The feature representation h of any two vehicles i and h j The topological relationship A ij Expressed as:

[0022]

[0023] in, Represents the edge between the i-th camera label and the j-th camera label in the camera topology graph G.

[0024] Furthermore, the working process of the graph convolutional network in step 3 is as follows:

[0025] By formula Calculate the mask matrix, where topk represents the topk algorithm, Sim i Represents the feature similarity between the i-th image and the j-th image, : represents all samples, (Sim i ,:) indicates the comparison between the i-th sample and all samples;

[0026] Based on the mask matrix through formula h′ i =σ(∑ j Mh j norm(Mask⊙A) ij ) obtains the aggregated features, where σ represents the ReLU activation function, M represents the learnable transformation matrix, norm represents the normalization function, and ⊙ represents the element-wise product;

[0027] By formula Perform weighted update on the aggregated features to obtain the final aggregated features, where: It's a camera The learnable weight vector, The dth row of M is represented by The d-th element of is scaled.

[0028] Furthermore, the step 4 includes:

[0029] By formula f i =Concat(h i ,h″ i ) Connect the vehicle feature representation and the final aggregate feature to obtain the final vehicle feature {f1,f2,...,f N}, h i represents the feature representation of the i-th vehicle, h″ i represents the final aggregated features of the i-th vehicle, f N Represents the final vehicle features of the Nth vehicle; f i Insert the fully connected layer to obtain the class prediction results.

[0030] Furthermore, the step five includes:

[0031] By formula Construct the first loss function; where y i represents the identity label of the i-th image, FC represents the fully connected layer, ‖·‖ represents the L2 norm distance, and f i,p and f i,n Represents the i-th image x in each mini-batch i The hardest positive and negative features, m represents the triple distance;

[0032] By formula

[0033]

[0034] Construct the second loss function, where S i represents the number of positive samples of the i-th picture, and Softplus represents the function of obtaining non-negative probability;

[0035] By formula Construct the target loss function;

[0036] Adjust the parameters of the graph convolutional network and train the graph convolutional network until the target loss function value is minimized, and then stop training to obtain a trained graph convolutional network.

[0037] The present invention also provides a vehicle re-identification device guided by a camera topology map, the device comprising:

[0038] Feature representation module, used to construct training sets and obtain vehicle feature representations;

[0039] A topology building module for building a camera topology map based on vehicle feature representation;

[0040] The feature aggregation module is used to construct the topological relationship between the feature representations of any two vehicles based on the camera topology graph and input it into the graph convolutional network to obtain the final aggregated features;

[0041] The class prediction module is used to fuse the final aggregated features with the vehicle feature representation, and the fusion result is input into the fully connected layer for class prediction;

[0042] The model training module is used to construct the target loss function and train the graph convolutional network until the target loss function value is minimized, thus obtaining a trained graph convolutional network.

[0043] The re-identification module is used to perform vehicle re-identification using the trained graph convolutional network.

[0044] Furthermore, the feature representation module is also used to:

[0045] Constructing a training set Among them, x i represents the i-th image, N T Indicates the total number of images in the training set, y i Indicates its identity tag, Indicates its camera tag;

[0046] The training set is input into the vehicle representation model ResNet-50 to extract the vehicle feature representation, which is represented as: {h1,h2,...,h N}, h N Represents the feature representation of the Nth vehicle.

[0047] Furthermore, the topology building module is further configured to:

[0048] According to the vehicle feature representation, different cameras are used as nodes, and edges are constructed based on the various relationships between cameras, thereby constructing a camera topology graph G = (V, E), where V represents the camera node, Indicates C T camera nodes, E is the edge set set in the camera topology graph, E={E system ,E position ,E orientation ,E individual}, E system ,E position ,E orientation ,E individual Represent the edge sets constructed based on the relationship between camera system, position, direction and identity respectively; then the camera topology graph based on camera system, position, direction and identity is represented as: G system ,G position ,G orientation ,G individual .

[0049] Furthermore, the feature aggregation module is further configured to:

[0050] The feature representation h of any two vehicles i and h j The topological relationship A ij Expressed as:

[0051]

[0052] in, Represents the edge between the i-th camera label and the j-th camera label in the camera topology graph G.

[0053] Furthermore, the working process of the graph convolutional network in the feature aggregation module is as follows:

[0054] By formula Calculate the mask matrix, where topk represents the topk algorithm, Sim i Represents the feature similarity between the i-th image and the j-th image, : represents all samples, (Sim i ,:) indicates the comparison between the i-th sample and all samples;

[0055] Based on the mask matrix through formula h′ i =σ(∑ j Mh j norm(Mask⊙A) ij ) obtains the aggregated features, where σ represents the ReLU activation function, M represents the learnable transformation matrix, norm represents the normalization function, and ⊙ represents the element-wise product;

[0056] By formula Perform weighted update on the aggregated features to obtain the final aggregated features, where: It's a camera The learnable weight vector, The dth row of M is represented by The d-th element of is scaled.

[0057] Furthermore, the class prediction module is further configured to:

[0058] By formula f i =Concat(h i ,h″ i ) Connect the vehicle feature representation and the final aggregate feature to obtain the final vehicle feature {f1,f2,...,f N}, h i represents the feature representation of the i-th vehicle, h″ i represents the final aggregated features of the i-th vehicle, f N Represents the final vehicle features of the Nth vehicle; f i Insert the fully connected layer to obtain the class prediction results.

[0059] Furthermore, the model training module is also used to:

[0060] By formula Construct the first loss function; where y i represents the identity label of the i-th image, FC represents the fully connected layer, ‖·‖ represents the L2 norm distance, and f i,p and f i,n Represents the i-th image x in each mini-batch i The hardest positive and negative features, m represents the triple distance;

[0061] By formula

[0062]

[0063] Construct the second loss function, where S i represents the number of positive samples of the i-th picture, and Softplus represents the function of obtaining non-negative probability;

[0064] By formula Construct the target loss function;

[0065] Adjust the parameters of the graph convolutional network and train the graph convolutional network until the target loss function value is minimized, and then stop training to obtain a trained graph convolutional network.

[0066] The advantages of the present invention are: the present invention first constructs a training set to obtain vehicle feature representation, and also constructs a camera topology map to input the topological relationship into the graph convolutional network to obtain aggregated features, and then fuses the two features to obtain fused features. The class prediction result is finally obtained based on the fused features. The entire feature recognition process not only considers the original visual features, that is, the vehicle feature representation, but also obtains aggregated features based on the camera topology map. Therefore, when multiple different cameras capture images, the differences in images captured by different cameras and the connections between adjacent cameras can be characterized, so that the captured feature vectors can accurately express vehicle information, and the accuracy of vehicle re-identification is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is a graph of the existing strong recognition baseline model on the VeRi-776 dataset;

[0068] Figure 2 A schematic diagram of a vehicle re-identification method guided by a camera topology map provided by an embodiment of the present invention;

[0069] Figure 3 A schematic diagram of generating a camera topology map from a real-world traffic scene in a vehicle re-identification method guided by a camera topology map provided by an embodiment of the present invention, wherein: Figure 3 (a) is a schematic diagram of a CCTV camera system. Figure 3 (b) is the corresponding camera topology;

[0070] Figure 4 A camera topology map based on camera system, position, direction and identity in a vehicle re-identification method guided by a camera topology map provided by an embodiment of the present invention, wherein: Figure 4 (a) is the camera topology diagram based on the camera system, Figure 4 (b) is the camera topology map based on the camera position, Figure 4 (c) is the camera topology map based on the camera direction, Figure 4 (d) is the camera topology map based on individual cameras. DETAILED DESCRIPTION

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0072] Example 1

[0073] like Figure 1 The phenomenon diagram of the existing strong recognition benchmark model on the VeRi-776 dataset shows the following three phenomena: (1) The Rank-1 performance of the entire camera system is much higher than that of each camera, e.g. Figure 1 As shown in (1), the five-pointed star in the figure represents the Rank-1 performance of the entire camera system. This shows that the Rank-1 performance of the existing technology method is exaggerated because it only retrieves easy positive samples under the entire camera system and does not accurately hit the positive samples under each camera. (2) The mAP performance of the entire camera system is much lower than the performance under each camera, such as Figure 1 As shown in (2), the five-pointed star in the figure represents the mAP performance of the entire camera system. This shows that the positive samples under each camera are more clustered than the positive samples under the entire camera system. (3) Eliminating the top-ranked samples will significantly reduce the re-identification performance, such as Figure 1 (3) shows that the re-ID performance obtained by traditional methods is suboptimal and susceptible to camera interference. In addition, it is well known that the information of each identity under each camera is limited. If the information of the vehicle can be aggregated under the entire camera system, then its information is sufficient and robust.

[0074] Therefore, if Figure 2 As shown, the present invention introduces a camera topology-guided vehicle re-identification method for vehicle re-identification to fully explore easy positive samples and hard positive samples under the entire camera system. The method includes:

[0075] S1: Construct a training set and obtain vehicle feature representation; the specific process is as follows:

[0076] Constructing a training set Among them, x i represents the i-th image, N T Indicates the total number of images in the training set, y i Indicates its identity tag, Indicates its camera tag;

[0077] The training set is input into the vehicle representation model ResNet-50 to extract vehicle feature representation, which is represented as follows: h N Represents the feature representation of the Nth vehicle.

[0078] S2: Construct a camera topology map based on vehicle feature representation; the specific process is as follows:

[0079] like Figure 3 As shown in Figure 1, according to the vehicle feature representation, different cameras are used as nodes, and edges are constructed based on the various relationships between cameras, thereby constructing a camera topology graph G = (V, E), where V represents the camera node, Indicates C T camera nodes, E is the edge set set in the camera topology graph, E={E system ,E position ,E orientation ,E individual}, E system ,E position ,E orientation ,E individual Represent the edge sets constructed based on the relationship between camera system, position, direction and identity respectively; then the camera topology graph based on camera system, position, direction and identity is represented as: G system ,G position ,G orientation ,G individual This embodiment is a camera topology diagram constructed based on a closed-circuit television camera system, where Figure 3 (a) is a schematic diagram of a CCTV camera system. Figure 3 (b) is the corresponding camera topology diagram.

[0080] Figure 4 The camera topology graph based on the camera system, position, orientation and identity (individual) is given, where G system Represents the camera topology based on the camera system. It is the default setting, and each adjacent node is connected in sequence, such as Figure 4 As shown in (a).

[0081] G position : represents the camera topology based on camera positions. First, the cameras at consecutive intersections are defined as spatially adjacent nodes. According to the CCTV camera system ( Figure 3 (b)) camera positions, camera5, camera7 and camera8 are considered as adjacent nodes, and there are edges between these adjacent nodes, such as Figure 4 (b) is shown. system Compared with the camera relationship of G positionThe camera relationship of G is easier because it requires positive samples from adjacent cameras to present consistent feature representations. Since a continuously moving vehicle can be captured by two adjacent cameras, G position It is consistent with the vehicle driving logic. position It interacts with the positive samples under the adjacent cameras.

[0082] G orientation : represents the camera topology based on the camera orientation. The more consistent the camera orientation, the more consistent the appearance of the positive samples. Figure 4 As shown in (c), the solid lines represent the edges determined by the positional relationship between the cameras, and the dotted lines represent the edges determined by the direction relationship between the cameras. Camera3 and camera4 are adjacent cameras, but since their camera directions are different, there is no edge determined by the direction relationship between them. position Compared with the camera relationship of G orientation The camera relationship of is easier because it ignores irrelevant nodes based on the camera direction. It is worth noting that the present invention defines cameras whose two directions are orthogonal as adjacent cameras, such as Figure 4 (c) camera5 and camera7. G orientation It interacts with the positive samples under the camera with the same direction.

[0083] G individual : represents the camera topology graph based on individual cameras. The video sequence of the target vehicle can be captured by the same camera. Figure 4 As shown in (d), any camera will have an edge on itself. individual The camera relationship of G is the easiest, because the images within a class captured by the same camera often have large information overlap. individual It interacts with the positive samples under the same camera.

[0084] Learning the relationship between camera system, position, orientation and identity helps to reduce the range of feature interaction in the feature learning and evaluation phases. These four subgraphs are used to construct the camera topology graph. In the camera topology graph G = (V, E), the edge between two cameras can be represented as E ij , and the larger the value, the stronger the relationship between the cameras. In the four subgraphs above, if there is an edge between the nodes, the value is 1, otherwise it is 0. By using these four topological relationships between cameras, the ultimate goal is to obtain hierarchical and aggregated topological features. Such topological features complement visual features, making the final features more comprehensive and robust.

[0085] S3: Based on the camera topology graph, the topological relationship between the feature representations of any two vehicles is constructed and input into the graph convolutional network to obtain the final aggregated features. The specific process is as follows:

[0086] In order to embed the topological relationship into the feature representation, the topological relationship between cameras is converted into sample pairs. Using the camera topology map guided by the CCTV camera system, an adjacency matrix is ​​established between the visual features, that is, the above vehicle feature representations. The feature representation h of any two vehicles i and h i The topological relationship A ij Expressed as:

[0087]

[0088] in, Represents the edge between the i-th camera label and the j-th camera label in the camera topology graph G.

[0089] As can be seen from the above formula, the feature relationship between samples is represented by the camera relationship between them. This is because the stronger the camera relationship between samples, the greater the overlap between vehicle images. However, this process includes many irrelevant samples and increases the computational burden.

[0090] In order to discard irrelevant samples and reduce the amount of computation, a mask matrix is ​​introduced. Assuming that two vehicle images are visually adjacent in the feature space, they are likely to be related. To this end, a k-nearest neighbor mask is calculated from the visual similarity. It will process the top k similarity values ​​of each row. Specifically, through the formula Calculate the mask matrix, where topk represents the topk algorithm, Simi represents the feature similarity between the i-th image and the j-th image; : represents all samples, (Sim i ,:) indicates comparison with Sim i and all samples; the topk algorithm is an existing algorithm, which mainly refers to finding the largest K numbers in an unordered sequence of N numbers. In this embodiment, the Sim i Find the top k similarity data with all samples. The specific content of the algorithm will not be described here.

[0091] Based on the mask matrix through formula h′ i =σ(∑ j Mh j norm(Mask⊙A) ij ) obtains the aggregated features, where σ represents the ReLU activation function, M represents the learnable transformation matrix, norm represents the normalization function, and ⊙ represents the element-wise product; the mask matrix Mask is added to the weighted transformation matrix to achieve feature aggregation that only occurs in adjacent cameras, which increases the attention to more relevant images.

[0092] Although the above formula obtains more robust aggregation features while reducing computational complexity, this aggregation process will introduce unnecessary camera noise. To solve this problem, a learnable camera storage matrix is ​​designed. and the weighted transformation matrix To store the transformation matrix of different cameras. Specifically, through the formula Perform weighted update on the aggregated features to obtain the final aggregated features, where: It's a camera The learnable weight vector of the camera The storage matrix, The dth row of M is represented by The d-th element of is scaled.

[0093] S4: The final aggregated features are fused with the vehicle feature representation, and the fusion result is input into the fully connected layer for class prediction. The specific process is as follows:

[0094] In the camera topology-based graph convolutional network, visual features are transformed into topological features, which are the final aggregated features, through adjacency relationships and specific transformation matrices. The camera topology-based graph convolutional network is used to learn cross-camera representations to obtain more discriminative vehicle features. The camera topology-based graph convolutional network only aggregates manageable neighbor nodes and learns different weight matrices for different cameras. It retains the ability of traditional graph convolutional networks to interact with graph nodes, while introducing the learning of different camera topological relationships. In addition, through the formula f i =Concat(h i ,h″ i ) Connect the vehicle feature representation and the final aggregated feature to obtain the final vehicle feature h i represents the feature representation of the i-th vehicle, h″ i represents the final aggregated features of the i-th vehicle, f N Represents the final vehicle features of the Nth vehicle; f i Put it into the fully connected layer to obtain the class prediction result. Figure 2 As shown, in actual applications, the vehicle feature representation can also be input into the hidden layer and the final aggregated feature can also be input into the hidden layer, and then the two are combined through formula f i =Concat(h i ,h″ i ) for fusion.

[0095] S5: Construct the target loss function and train the graph convolutional network until the target loss function value is minimized, and then stop training to obtain the trained graph convolutional network. The specific process is as follows:

[0096] By formula Construct the first loss function; where y i represents the identity label of the i-th image, FC represents the fully connected layer, ‖·‖ represents the L2 norm distance, and f i,p and f i,n Represents the i-th image x in each mini-batch i The most difficult positive and negative features, m represents the triplet distance; although the first loss function is widely used in the field of vehicle re-identification, it has limitations and cannot consider the topological relationship between samples.

[0097] Therefore, this paper proposes a new topological cross entropy loss based on the topological relationship in the topological cross entropy loss, which promotes the clustering of positive samples from strong to weak, optimizes the topological relationship between positive samples, and trains the entire network in an end-to-end manner. This topological cross entropy loss is also the key to aggregating vehicles under adjacent cameras, which makes the representation aggregation process more effective and efficient. Specifically, through the formula

[0098]

[0099] Construct the second loss function, where S i represents the number of positive samples of the i-th picture, and Softplus represents the function of obtaining non-negative probability;

[0100] By formula Construct the target loss function;

[0101] Adjust the parameters of the graph convolutional network and train the graph convolutional network until the target loss function value is minimized, and then stop training to obtain a trained graph convolutional network.

[0102] S6: Real-time vehicle image acquisition is input into ResNet-50 to obtain vehicle feature representation, a camera topology map is constructed, and the camera topology map is input into the trained graph convolutional network. The trained graph convolutional network is used to perform vehicle re-identification, and its recognition result is fused with the vehicle feature representation and input into the fully connected layer to obtain the predicted class result.

[0103] Through the above technical solution, the present invention first constructs a training set to obtain vehicle feature representation, and also constructs a camera topology map to input the topological relationship into the graph convolutional network to obtain aggregated features, and then fuses the two features to obtain fused features. The class prediction result is finally obtained based on the fused features. The entire feature recognition process not only considers the original visual features, that is, the vehicle feature representation, but also obtains aggregated features based on the camera topology map. Therefore, when multiple different cameras capture images, the differences in images captured by different cameras and the connections between adjacent cameras can be characterized, so that the captured feature vectors can accurately express vehicle information, and the accuracy of vehicle re-identification is high.

[0104] Example 2

[0105] Based on Example 1, Example 2 of the present invention further provides a vehicle re-identification device guided by a camera topology map, the device comprising:

[0106] Feature representation module, used to construct training sets and obtain vehicle feature representations;

[0107] A topology building module for building a camera topology map based on vehicle feature representation;

[0108] The feature aggregation module is used to construct the topological relationship between the feature representations of any two vehicles based on the camera topology graph and input it into the graph convolutional network to obtain the final aggregated features;

[0109] The class prediction module is used to fuse the final aggregated features with the vehicle feature representation, and the fusion result is input into the fully connected layer for class prediction;

[0110] The model training module is used to construct the target loss function and train the graph convolutional network until the target loss function value is minimized, thus obtaining a trained graph convolutional network.

[0111] The re-identification module is used to perform vehicle re-identification using the trained graph convolutional network.

[0112] Specifically, the feature representation module is further used to:

[0113] Constructing a training set Among them, x i represents the i-th image, N T Indicates the total number of images in the training set, y i Indicates its identity tag, Indicates its camera tag;

[0114] The training set is input into the vehicle representation model ResNet-50 to extract the vehicle feature representation, which is represented as: {h1,h2,...,h N}, h N Represents the feature representation of the Nth vehicle.

[0115] Specifically, the topology building module is further used to:

[0116] According to the vehicle feature representation, different cameras are used as nodes, and edges are constructed based on the various relationships between cameras, thereby constructing a camera topology graph G = (V, E), where V represents the camera node, Indicates C T camera nodes, E is the edge set set in the camera topology graph, E={E system ,E position ,E orientation ,Eindividual}, E system ,E position ,E orientation ,E individual Represent the edge sets constructed based on the relationship between camera system, position, direction and identity respectively; then the camera topology graph based on camera system, position, direction and identity is represented as: G system ,G position ,G orientation ,G individual .

[0117] Specifically, the feature aggregation module is further used to:

[0118] The feature representation h of any two vehicles i and h j The topological relationship A ij Expressed as:

[0119]

[0120] in, Represents the edge between the i-th camera label and the j-th camera label in the camera topology graph G.

[0121] More specifically, the working process of the graph convolutional network in the feature aggregation module is as follows:

[0122] By formula Calculate the mask matrix, where topk represents the topk algorithm, Sim i Represents the feature similarity between the i-th image and the j-th image, : represents all samples, (Sim i ,:) indicates comparison with Sim i and all samples;

[0123] Based on the mask matrix through formula h′ i =σ(∑ j Mh j norm(Mask⊙A) ij ) obtains the aggregated features, where σ represents the ReLU activation function, M represents the learnable transformation matrix, norm represents the normalization function, and ⊙ represents the element-wise product;

[0124] By formula Perform weighted update on the aggregated features to obtain the final aggregated features, where: It's a camera The learnable weight vector, The dth row of M is represented by The d-th element of is scaled.

[0125] Specifically, the class prediction module is further used to:

[0126] By formula f i =Concat(h i ,h″ i ) Connect the vehicle feature representation and the final aggregate feature to obtain the final vehicle feature {f1,f2,...,f N}, h i represents the feature representation of the i-th vehicle, h″ i represents the final aggregated features of the i-th vehicle, f N Represents the final vehicle features of the Nth vehicle; f i Insert the fully connected layer to obtain the class prediction results.

[0127] Specifically, the model training module is also used to:

[0128] By formula Construct the first loss function; where y i represents the identity label of the i-th image, FC represents the fully connected layer, ‖·‖ represents the L2 norm distance, and f i,p and f i,n Represents the i-th image x in each mini-batch i The hardest positive and negative features, m represents the triple distance;

[0129] By formula

[0130]

[0131] Construct the second loss function, where S i represents the number of positive samples of the i-th picture, and Softplus represents the function of obtaining non-negative probability;

[0132] By formula Construct the target loss function;

[0133] Adjust the parameters of the graph convolutional network and train the graph convolutional network until the target loss function value is minimized, and then stop training to obtain a trained graph convolutional network.

[0134] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A vehicle re-identification method guided by a camera topology map, characterized in that: The method comprises: Step 1: Construct a training set and obtain vehicle feature representation; Construct a training set ,in, represents the i-th image, Represents the total number of images in the training set, Indicates its identity tag, Indicates its camera label; the training set is input into the vehicle representation model ResNet-50 to extract the vehicle feature representation, and the vehicle feature representation is: , Represents the feature representation of the Nth vehicle; Step 2: Construct a camera topology graph based on vehicle feature representation; Based on the vehicle feature representation, different cameras are used as nodes, and edges are constructed based on the various relationships between cameras to construct a camera topology graph. ,in, Represents the camera node, , Indicates the camera nodes, is the edge set set in the camera topology graph, , Represent the edge sets constructed based on the relationship between camera system, position, direction and identity respectively; then the camera topology graph based on camera system, position, direction and identity is represented as: ; Step 3: Based on the camera topology graph, construct the topological relationship between the feature representations of any two vehicles and input it into the graph convolutional network to obtain the final aggregated features; the feature representations of any two vehicles Topological relationship Expressed as: in, represents the edge between the i-th camera label and the j-th camera label in the camera topology graph G; The working process of graph convolutional network is: By formula Compute the mask matrix, where express algorithm, represents the feature similarity between the i-th image and the j-th image, : represents all samples, Express contrast and all samples; Based on the mask matrix through the formula Get the aggregated features, where represents the ReLU activation function, represents the learnable transformation matrix, represents the normalization function, represents element-wise product; By formula Perform weighted update on the aggregated features to obtain the final aggregated features, where: It's a camera The learnable weight vector, express The dth row of The d-th element of is scaled; Step 4: Fuse the final aggregated features with the vehicle feature representation and input the fusion result into the fully connected layer for class prediction; Step 5: Construct the target loss function and train the graph convolutional network until the target loss function value is minimized, and then stop training to obtain the trained graph convolutional network; Step 6: Use the trained graph convolutional network to perform vehicle re-identification.

2. The vehicle re-identification method guided by camera topology map according to claim 1, characterized in that: The fourth step includes: By formula Connect the vehicle feature representation and the final aggregated feature to obtain the final vehicle feature , represents the feature representation of the i-th vehicle, represents the final aggregated features of the i-th vehicle, Represents the final vehicle characteristics of the Nth vehicle; Insert the fully connected layer to obtain the class prediction results.

3. The vehicle re-identification method guided by camera topology map according to claim 1, characterized in that: The step five includes: By formula Construct the first loss function; where, represents the fully connected layer, represents the L2 norm distance, and Represents the i-th image in each mini-batch The hardest positive and negative features, m represents the triple distance; By formula Construct the second loss function, where Indicates the number of positive samples of the i-th picture, represents a function for obtaining non-negative probabilities; By formula Construct the target loss function; Adjust the parameters of the graph convolutional network and train the graph convolutional network until the target loss function value is minimized, and then stop training to obtain a trained graph convolutional network.

4. A vehicle re-identification device guided by a camera topology map, characterized in that: The device comprises: Feature representation module, used to construct training sets and obtain vehicle feature representation; construct training sets ,in, represents the i-th image, Represents the total number of images in the training set, Indicates its identity tag, Indicates its camera label; the training set is input into the vehicle representation model ResNet-50 to extract the vehicle feature representation, and the vehicle feature representation is: , Represents the feature representation of the Nth vehicle; A topology building module is used to build a camera topology graph based on vehicle feature representation. Based on the vehicle feature representation, different cameras are used as nodes, and edges are built based on the various relationships between cameras to build a camera topology graph. ,in, Represents the camera node, , Indicates the camera nodes, is the edge set set in the camera topology graph, , Represent the edge sets constructed based on the relationship between camera system, position, direction and identity respectively; then the camera topology graph based on camera system, position, direction and identity is represented as: ; The feature aggregation module is used to construct the topological relationship between the feature representations of any two vehicles based on the camera topology graph and input it into the graph convolutional network to obtain the final aggregated features; the feature representations of any two vehicles Topological relationship Expressed as: in, represents the edge between the i-th camera label and the j-th camera label in the camera topology graph G; The working process of graph convolutional network is: By formula Compute the mask matrix, where express algorithm, represents the feature similarity between the i-th image and the j-th image, : represents all samples, Express contrast and all samples; Based on the mask matrix through the formula Get the aggregated features, where represents the ReLU activation function, represents the learnable transformation matrix, represents the normalization function, represents element-wise product; By formula Perform weighted update on the aggregated features to obtain the final aggregated features, where: It's a camera The learnable weight vector, express The dth row of The d-th element of is scaled; The class prediction module is used to fuse the final aggregated features with the vehicle feature representation, and the fusion result is input into the fully connected layer for class prediction; The model training module is used to construct the target loss function and train the graph convolutional network until the target loss function value is minimized, thus obtaining a trained graph convolutional network. The re-identification module is used to perform vehicle re-identification using the trained graph convolutional network.

Images