A method for re-identifying vehicles obscured by video

Abstract

This invention discloses a method for re-identifying vehicles occluded in videos. The method includes: inputting a vehicle image into a 3D-CNN backbone model for feature extraction; processing the extracted features through an adaptive edge transfer graph convolutional layer, and outputting... x out Features, and on x out Multi-level cascading of features yields relational features. x A ; Relationship characteristics x A The input cross-level alignment layer obtains similar topological features; similarity is calculated for the similar topological features, and the calculated result is compared with a preset threshold. If it is greater than the threshold, the vehicle images are the same; otherwise, they are different. This invention proposes a vehicle occlusion re-identification method. It extracts features from different objects through a backbone network and uses image comparison topology to obtain topological information for the occluded part of the target. By jointly processing some features of the object, the training efficiency of the model is improved, and the accuracy of vehicle occlusion re-identification is effectively improved.

Description

Technical Field

[0001] This invention relates to a method for re-identifying vehicles obscured by video, belonging to the field of image processing technology. Background Technology

[0002] Traditional vehicle re-identification relies heavily on existing vehicle images, requiring processing of raw data to facilitate network learning and recognition. This is labor-intensive and resource-wasting. To address this, methods for vehicle re-identification directly from video have emerged, meeting practical needs and improving efficiency. However, complex road conditions, diverse vehicle types, numerous non-overlapping cameras, and partial occlusion caused by vehicles on the road significantly complicate vehicle re-identification tasks. Retrieving the target vehicle from a massive video image database manually is difficult, time-consuming, and often results in missed opportunities for tracking and solving cases. Vehicle occlusion has also become a new problem in the field of vehicle re-identification. Therefore, utilizing computer vision technology to replace manual searching can quickly and accurately search, locate, and track occluded target vehicles in massive video image databases, playing a crucial role in road traffic safety and public safety. Summary of the Invention

[0003] The purpose of this invention is to provide a method for re-identifying vehicles with video occlusion, so as to solve the problem that it is difficult to achieve the identification of a large number of partially occluded vehicles by relying solely on human labor.

[0004] A method for re-identifying vehicles obscured by video, the method comprising:

[0005] Vehicle images are input into a 3D-CNN backbone model for feature extraction;

[0006] The extracted features are processed by an adaptive edge transfer graph convolutional layer, and the output x is obtained. out Features, and for x out Feature cascading yields relational feature x A ;

[0007] Relation feature x A Similar topology features are obtained by inputting cross-level alignment layers;

[0008] The similarity of similar topological features is calculated, and the result is compared with a preset threshold. If the result is greater than the threshold, the vehicle images are the same; otherwise, they are different.

[0009] Furthermore, the feature extraction method includes:

[0010] 3D-CNN model for vehicle images The processing yields temporal static features (t, b), coarse-grained global features (s, a), and fine-grained local features (s, b); from time... t The static content of the dimension, and the content from space h l ×w l Coarse-grained and fine-grained features of dimensions;

[0011] For each feature Reshape to feature volume c l t l ×h l w l Features Where c l Let be the number of channels, and then pass them through the attention decomposition module to obtain new features.

[0012] Furthermore, the x out Methods for obtaining features include:

[0013] The difference feature matrix values ​​are obtained by analyzing the differences between features;

[0014] Multiply the difference feature matrix values ​​by the predefined normalization matrix to obtain a new adjacency matrix;

[0015] The remaining features are then concatenated and integrated through three unconnected fully connected layers to obtain feature x. out .

[0016] Furthermore, the relational feature x A The methods for obtaining the information include:

[0017] The relation feature x is obtained by cascading multiple adaptive edge transfer map convolutional layers to form a difference feature matrix of a high-order relation module. A .

[0018] Furthermore, the adaptive edge-transfer graph convolutional layer is used to suppress the message passing of meaningless features.

[0019] Furthermore, the similarity acquisition includes:

[0020] Given the two corresponding graphs P1 = (V1, E1) and P2 = (V2, E2) for images X1 and X2, let U ∈ [0, 1] be the matching matrix between V1 and V2. ia For the corresponding node v in the two graphs 1i and v 2a The degree of matching between them is used to construct a square-symmetric positive matrix M∈R. KK×KK K is the number of key points, such that M ia and M ibThe optimal matching U was obtained by measuring the matching degree of each pair (i,j)∈E1 and (a,b)∈E2. * ;

[0021] Two sets of feature nodes x1 and x2 are embedded into a fully connected layer and a ReLU layer in a hidden space. The affinity matrix U is obtained by matching the graphs of x1 and x2. k×k After passing through a fully connected layer f, the output is x1. f and x2 f By using an adaptive edge propagation graph convolutional layer and a cross-level feature alignment layer, the topological features are obtained as follows: Similarity is obtained based on topological feature information.

[0022] Furthermore, the method for obtaining the difference feature matrix includes: reshaping the local features and global features into a K×K×c matrix. l Then, a subtraction operation is performed to obtain the difference feature matrix between local features and global features.

[0023] Furthermore, the formula for calculating similarity is as follows:

[0024]

[0025] Where σ(·) is the sigmoid activation function, |·| is the element matching operation, and f is the fully connected layer.

[0026] Furthermore, the loss of the cross-level feature alignment layer can be expressed as:

[0027]

[0028] When X1 and X2 are the same vehicle, y = 1; otherwise, y = 0.

[0029] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: This invention proposes a vehicle occlusion re-identification method, which extracts features from different objects through a backbone network and uses image comparison topology to obtain topology information for the occluded part of the target. By jointly processing some features of the object, the training efficiency of the model is improved, and the accuracy of vehicle occlusion re-identification is effectively improved. Attached Figure Description

[0030] Figure 1 This is a structural diagram of the feature extraction module of the present invention;

[0031] Figure 2 This is a diagram of the adaptive edge transfer graph convolutional layer structure of the present invention;

[0032] Figure 3 This is a structural diagram of the cross-level feature alignment layer of the present invention. Detailed Implementation

[0033] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0034] As shown in the figure, this invention discloses a method for re-identifying vehicles obscured by video, the method comprising:

[0035] Vehicle images are input into a 3D-CNN backbone model for feature extraction;

[0036] The extracted features are processed by an adaptive edge transfer graph convolutional layer, and the output x is obtained. out Features, and for x out Feature cascading yields relational feature x A ;

[0037] Relation feature x A Similar topology features are obtained by inputting cross-level alignment layers;

[0038] The similarity of similar topological features is calculated, and the result is compared with a preset threshold. If the result is greater than the threshold, the vehicle images are the same; otherwise, they are different.

[0039] In this embodiment, the present invention is divided into three stages, as follows:

[0040] The first stage involves extracting image features by constructing a semantic extraction module. A 3D-CNN backbone model is used to extract semantic features, resulting in three features.

[0041] The second stage processes the extracted features, proposing an Adaptive Edge-transfer Graph Convolutional layer (AEGC) to transmit information about the relationships between nodes. This layer can dynamically learn the direction and degree of connection, automatically suppressing the propagation of meaningless features, and outputting a processed x. out Features, through multi-level cascading, yield relational features x. A .

[0042] The third stage proposes a cross-level feature alignment layer (CFA) to address occlusion, which jointly learns and embeds topological information into features to obtain similarity calculations.

[0043] Phase 1: Construct a semantic extraction module to extract semantic features of key vehicle regions, generating features including local features, global features, and temporal static features.

[0044] The construction of the semantic extraction module in this stage is divided into three steps.

[0045] Step 1: 3D-CNN model on vehicle images The processing yields three features: temporal static features (t, b), coarse-grained global features (s, a), and fine-grained local features (s, b). These features originate from static content along the time dimension (t) and from spatial dimensions (h). l ×w l Coarse-grained and fine-grained features of dimensions.

[0046] Step 2: For each feature Reshape to feature volume c l t l ×h l w l Features Where c l The number of channels is given, and each feature is passed through an Attention Decomposition Module (ADM) to obtain new features.

[0047] The second stage: To suppress message passing of meaningless features in the extracted semantic features and promote message passing of semantic features, an Adaptive Edge-transfer Graph Convolutional layer (AEGC) is constructed to dynamically learn the direction and extent of message passing. Semantic features of different keypoint regions are treated as nodes. This stage is constructed in two steps.

[0048] Step 1: Update the edge weights A of all nodes in the graph by using the differences between the global and local features obtained in the first stage. tre .

[0049] Step 2: By pre-setting a normalized matrix W and three fully connected layers f1, f2, and f3 for easier dimensionality calculation, the three feature sets are integrated to obtain the output x. out .

[0050] The third stage: To fully utilize the vehicle topology information learned by the graph matching algorithm, a Cross-level Feature Alignment layer (CFA) is proposed. It takes two graphs as input, uses graph matching to learn the correspondence between nodes in the two graphs, and transmits messages by treating the learned correspondence as an adjacency matrix. Therefore, relevant features can be enhanced, and alignment information can be embedded into the features. This stage consists of three steps.

[0051] Step 1: Based on the two corresponding graphs P1 = (V1, E1) and P2 = (V2, E2) for images X1 and X2, let U ∈ [0, 1] be the matching value between V1 and V2.

[0052] Assign matrix, U ia For the corresponding node v in the two graphs 1i and v 2a The matching degree between them. Considering the second-order similarity of the edges, a square-symmetric positive matrix M∈R is constructed. KK×KK K is the number of key points, such that M ia and M ib The matching degree of each pair (i,j)∈E1 and (a,b)∈E2 was measured. For pairs that do not form an edge, their corresponding entries in the matrix were set to 0. Diagonal entries contain node boundary scores, while off-diagonal entries contain edge boundary scores. Therefore, the optimal matching U can be obtained. * .

[0053] Step 2: A novel Cross-level Feature Alignmentlayer (CFA) is proposed. Two sets of feature nodes x1 and x2 are embedded into a fully connected layer and a ReLU layer in a hidden space. In the third stage (Step 1), the graphs of x1 and x2 are matched to obtain the affinity matrix U. k×k After passing through a fully connected layer f, the output is x1. f and x2 f Through the AEGC layer and CFA layer in stage two, their topological features are obtained as follows:

[0054] Step 3: Obtain their similarity through topological information. Stochastic gradient descent (SGD) is used for end-to-end training when optimizing the objective function.

[0055] 1. Feature extraction stage

[0056] Step 1: Refine the time-static features and define r. tb For the time static resolution, the decomposition function G αβ (·) represents the pooling function:

[0057] G tb =pool(r tb ,1,1)

[0058] Step 2: The feature volume is then decomposed along the spatial dimension to extract coordinate and fine-level information. For frames where the car is occluded or spatially misaligned, fine-level features capture the "car-part" of the frame. sa For spatial coarse-grained resolution, r sbFor fine-grained spatial resolution, the spatial decomposition function is:

[0059] G sa =pool(1, r) sa ,r sa )

[0060] G sb =pool(1, r) sb ,r sb )

[0061] 2. Construction phase of ADM module

[0062] Step 1: To decompose the three features of the vehicle image, use the decomposition function G. αβ (·), which is designed as a pooling function, is used after the input features pass through the channel reduction layer H. αβ (·), which possesses A convolutional kernel of size 1 is used to extract specific information: In this design, n is set to 16. The following three functions are defined to obtain the composite function for each feature output:

[0063]

[0064]

[0065]

[0066] Step 2: The composite function output in the above formula Size is c l t l ×h l w l By calculating the variance matrix C αβ Let's summarize the input features:

[0067]

[0068] Set the parameter k to 4, and use the softmax function σ(·) to obtain the decomposition mask M. αβ The formula is:

[0069] M αβ =σ(C αβ )

[0070] Step 3: Use the explode mask M αβ The text can obtain new feature representations. The formula is:

[0071]

[0072] Step 4: L ceFor standard cross-entropy, L tri Given a triple loss function, where L is the overall loss function, the loss function can be expressed as:

[0073] L = L ce +L tri

[0074] 3. Construction of the AEGC layer

[0075] Step 1: First, reshape the local and global features into K×K×c l Then, a subtraction operation is performed to obtain the difference feature matrix A between the local features and the global features. tre The formula is:

[0076]

[0077] Step 2: Analyze the difference feature matrix A tre The input features are multiplied by a predefined normalized matrix W to obtain a new adjacency matrix. Then, the input features are concatenated and integrated through three unconnected fully connected layers f1, f2, and f3 to obtain x. out .

[0078]

[0079] F uses multiple AEGC layers to cascade into a higher-order relational module. A Obtain relation features x A for:

[0080]

[0081] Step 3: L ce For standard cross-entropy, L tri Given a triple loss function, where L is the overall loss function, the loss function can be expressed as:

[0082] L = L ce +L tri

[0083] 4. Construction of the CFA layer

[0084] Step 1: By following Step 1 in the third stage, the optimal matching U can be obtained. * The formula is:

[0085]

[0086] Step 2: Embed x1 and x2 into a fully connected layer and a ReLU layer to obtain two sets of hidden features. and Step one yields the matching matrix U, which is then passed from the fully connected layer f via a dot product with the feature matrix. The formula can be expressed as:

[0087]

[0088]

[0089] Step 3: The relationship features between a pair of images X1 and X2 can be obtained through the AEGC layer. F C (·) represents a CFA layer, and topological features can be obtained by cascading multiple layers. for:

[0090]

[0091] After obtaining the topological features, their similarity can be determined. Perform the calculation:

[0092]

[0093] Where σ(·) is the sigmoid activation function, |·| is the element matching operation, and f is the fully connected layer.

[0094] Step 4: The loss of the CFA layer can be expressed as:

[0095]

[0096] When X1 and X2 are the same vehicle, y = 1; otherwise, y = 0. Stochastic gradient descent (SGD) is used for end-to-end network training.

[0097] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for video occlusion vehicle re-identification, characterized in that, The method comprises: inputting a vehicle image into a 3D-CNN backbone network model for feature extraction; The extracted features are processed by an adaptive edge transfer graph convolution layer to output features, and the features are obtained through multi-level cascading of the relationship features features ; relational features input cross-level alignment layer to obtain similar topological features; calculating similarity of similar extension features, comparing the calculated result with a preset threshold value, if the result is greater than the threshold value, the vehicle images are the same, otherwise they are different; The The feature acquisition method comprises: obtaining a difference feature matrix value through the difference between the features; multiplying the difference feature matrix value with a predefined normalization matrix to obtain a new adjacency matrix; The remaining features are spliced and integrated through three unconnected fully connected layers to obtain the features ; The relationship feature The acquisition method comprises: The difference feature matrix obtained by cascading multiple adaptive edge transfer graph convolution layers into a high-order relationship module obtains relationship features ; The similarity acquisition comprises: According to the image and the image corresponding two graphs and , set as and between the matching matrix, the matching degree between the corresponding nodes of two graphs and , establish a square symmetric positive matrix , as the number of key points, make and measure the matching degree of each pair and get the optimal matching ; Two groups of feature nodes and are embedded into a hidden space full connection layer and ReLU layer, and the affinity matrix is obtained by matching the graph and . , the output is obtained by the full connection layer . and , the extended features are obtained by the adaptive edge transfer graph convolution layer and the cross-level feature alignment layer , and the similarity is obtained according to the extended feature information .

2. The method for video occlusion vehicle re-identification according to claim 1, characterized in that, The feature extraction method comprises: a 3D-CNN model for vehicle images The processing obtains time static features (t, b), coarse-grained global features (s, a), and fine-grained local features (s, b); the time t dimensional static content, and the coarse-grained and fine-grained features from the spatial dimension; for each feature is reshaped into a feature volume with a size of , wherein is the number of channels, and new features are obtained through an attention decomposition module respectively .​ 3.The method of video occlusion vehicle re-identification according to claim 1, characterized in that, The adaptive edge transfer graph convolution layer is used to suppress meaningless feature message passing. 4.The method of video occlusion vehicle re-identification according to claim 1, characterized in that, The difference feature matrix acquisition method comprises the following steps: And do subtraction operation, get the difference feature matrix between local features and global features.

5. The method for video occlusion vehicle re-identification according to claim 1, characterized in that, The formula for similarity is as follows: where is a sigmoid activation function, is an element-wise match operation, is a fully connected layer.

6. The method for video occlusion vehicle re-identification according to claim 1, characterized in that, The loss of the cross-level feature alignment layer can be expressed as: ； wherein and are the same car, , otherwise .

Images