Fine-grained feature matching vehicle re-identification method for license plate shielding scene

Abstract

The invention discloses a fine-grained feature matching vehicle re-identification method for a license plate shielding scene, and the method comprises a vehicle detection stage based on illumination perception and adaptive prediction, and a fine-grained Transform feature matching stage based on implicit semantic prototype perception. Comprising an illumination sensing module and an adaptive prediction module. The illumination sensing module is used for realizing image adaptive enhancement under different illumination conditions; the adaptive prediction module suppresses a redundant prediction frame through a self-attention weight mechanism; in the fine-grained Transform feature matching stage based on implicit semantic prototype perception, an implicit semantic prototype learning module and a fine-grained Transform matching module are included; the implicit semantic prototype learning module unsupervised decouples vehicle structural components through a group of learnable semantic prototypes to generate a soft semantic distribution diagram; and the fine-grained Transform matching module is used for introducing semantic affinity bias in cross attention. According to the method, the accuracy and stability of vehicle detection and identity matching in a complex environment can be improved.

Description

Technical Field

[0001] This invention relates to a fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios, belonging to the application technology of computer vision in the field of intelligent transportation. Background Technology

[0002] Vehicle re-identification, a key technology in intelligent transportation systems, aims to achieve continuous tracking and retrieval of specific vehicles within a multi-camera monitoring network. A typical vehicle re-identification system comprises two core stages: vehicle detection and feature matching. With the development of deep learning technology, vehicle re-identification methods based on convolutional neural networks and Transformers have made significant progress, demonstrating good performance in controllable scenarios.

[0003] Existing methods are mainly divided into global feature-based methods and local feature-based methods. Global feature-based methods extract feature representations of the entire vehicle through deep networks for matching, and perform well in scenarios where the vehicle is upright and unobstructed. Local feature-based methods achieve fine-grained matching by detecting key points or explicit component regions (such as headlights and windows), which improves robustness to occlusion and viewpoint changes to some extent. However, when facing complex real-world scenarios, existing methods still have significant limitations:

[0004] First, in the vehicle detection stage, existing detection algorithms are prone to missed detections and false detections under adverse conditions such as low light at night, strong shadows, and rain or snow. Meanwhile, in congested traffic scenarios, severe occlusion between vehicles can lead to redundant or missing detection boxes. Traditional non-maximum suppression strategies are ill-suited to the complexities of multiple overlapping targets, severely impacting the input quality of subsequent re-identification processes.

[0005] Second, in the feature matching stage, existing methods struggle to handle challenges such as license plate occlusion, partial modifications, and drastic changes in viewpoint. Traditional handcrafted feature methods are sensitive to changes in lighting and scale, lacking robustness; while deep learning methods often rely excessively on obvious markers like license plates, resulting in significant performance degradation when these features are invisible. Furthermore, existing methods lack the ability to autonomously discover the implicit structural semantics of vehicles, making it difficult to learn discriminative component-level feature representations in unsupervised environments, thus limiting their ability to distinguish between vehicles of the same model.

[0006] In summary, existing technologies suffer from the following core problems: 1. The detection stage lacks reliability in complex environments, making it difficult to guarantee the input quality for subsequent processing; 2. The matching stage suffers from poor robustness and over-reliance on easily lost global features, lacking the autonomous and effective utilization of stable vehicle structural components and personalized detailed features. Therefore, there is an urgent need for an end-to-end vehicle re-identification solution that can simultaneously improve robustness and accuracy in complex scenarios in both the detection and matching stages. Summary of the Invention

[0007] Purpose of the Invention: To address the problems of low detection accuracy, insufficient utilization of stable vehicle structural features, and difficulty in handling severe occlusion and lighting changes in existing vehicle re-identification methods in complex scenarios, this invention proposes a fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios. By constructing an end-to-end joint learning framework, an illumination perception and redundant box suppression mechanism is introduced in the detection stage to improve detection robustness in complex environments. In the matching stage, an implicit semantic prototype learning and prototype-aware matching mechanism is proposed, enabling the network to learn the semantic representation of vehicle components unsupervised and guide feature matching at the semantic level, thereby achieving accurate vehicle identification across viewpoints and across occlusions.

[0008] Technical solution: To achieve the above objectives, the technical solution adopted by this invention is as follows:

[0009] A fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios includes a vehicle detection stage based on illumination perception and adaptive prediction, and a fine-grained Transformer feature matching stage based on implicit semantic prototype perception.

[0010] The vehicle detection stage based on illumination perception and adaptive prediction includes an illumination perception module and an adaptive prediction module; the illumination perception module is used to achieve adaptive image enhancement under different illumination conditions; the adaptive prediction module suppresses redundant prediction boxes through a self-attention weight mechanism.

[0011] The fine-grained Transformer feature matching stage based on implicit semantic prototype awareness includes an implicit semantic prototype learning module and a fine-grained Transformer matching module. The implicit semantic prototype learning module unsupervisedly decouples vehicle structural components through a set of learnable semantic prototypes to generate a soft semantic assignment map to enhance feature discriminativeness and structural consistency. The fine-grained Transformer matching module introduces a semantic affinity bias in cross-attention, making the matching process focus more on regions with high semantic consistency and improving the robustness of matching under occlusion and viewpoint changes.

[0012] Specifically, the method includes the following steps:

[0013] Step 1: Given an input image I0, input image I0 is fed into the YOLOv13 backbone network built based on a deep convolutional neural network. The shallow extraction layer of the YOLOv13 backbone network outputs the intermediate feature map F. in ;

[0014] Step 2, transfer the intermediate feature map F in The input is fed into the illumination sensing module, which generates a set of illumination sensing modulation parameters. Based on these parameters, the intermediate feature map F is processed.in Perform a channel-by-channel affine transformation to obtain the illumination enhancement feature map F. enhanced ;

[0015] Step 3: Enhance the illumination feature map F enhanced Returning to the YOLOv13 backbone network to continue forward propagation, it participates in the feature processing flow of the global correlation enhancement module of the YOLOv13 backbone network, and outputs the initial prediction box set P in the detection head of the YOLOv13 backbone network.

[0016] Step 4: Input the initial set of predicted bounding boxes P into the adaptive prediction module. A self-attention weight mechanism is used to achieve global contextual interaction between the predicted bounding boxes, suppressing redundant predicted bounding boxes and obtaining a refined set of predicted bounding boxes P. refined ;

[0017] Step 5: Based on the refined set of prediction boxes P refined Cropped vehicle image I is extracted from input image I0;

[0018] Step 6: Crop the two vehicle images to be matched. A I B Simultaneously, the input is fed into a CNN backbone network with shared weights, and a feature pyramid network is connected to the back end of the CNN backbone network. Through a top-down path and lateral connections, features from different levels are fused, thereby obtaining coarse-grained features containing rich semantic information. and fine-grained features that retain detailed information

[0019] Step 7: Transfer coarse-grained features The input is fed into the implicit semantic prototype learning module, and the output is a reconstructed feature map F after semantic refinement. A ′, F B '、Soft semantic allocation graph A A A B ;

[0020] Step 8: Reconstruct the feature map F A ′, F B '、Soft semantic allocation graph A A A B The fine-grained Transformer matching module is input to perform dense feature interaction and matching to obtain the vehicle cropping image I. A and I B A series of high-precision matching point pairs are used to calculate the vehicle cropping image I based on the matching point pairs. A and I B Similarity score between AB In order to make re-identification decisions.

[0021] Specifically, in Step 1, the YOLOv13 backbone network consists of a set of DS-C3k2 modules based on depthwise separable convolutions, used to capture hierarchical feature representations from low-level texture to high-level semantics. The intermediate feature map F is output from the shallow layers of the YOLOv13 backbone network (specifically located in the second DS-C3k2 module). in The reason for choosing shallow features is that they retain rich spatial details and low-level visual features (such as edges, corners, textures, etc.), which are crucial for subsequent vehicle detection and are particularly sensitive to changes in lighting, shadows, and noise interference.

[0022] Specifically, in Step 2, the illumination sensing module is a light quantum network. Its core idea is to characterize the dynamic modulation feature response by learning the illumination conditions, thereby enhancing the useful signal under low light and suppressing the response in overexposed areas. The illumination sensing module generates a set of illumination sensing modulation parameters to modify the intermediate feature map F. in Perform a channel-by-channel affine transformation to obtain the illumination enhancement feature map F. enhanced The execution process of the light sensing module includes the following steps:

[0023] (21) Use global average pooling operation to process intermediate feature maps along the spatial dimension. Compression is performed to generate a global illumination sensing vector.

[0024]

[0025] Among them: U={u c}, u c The global illumination sensing vector U represents the average response intensity of the c-th channel feature across the entire spatial range. It effectively encodes the statistical information of global illumination conditions, providing a basis for subsequent feature modulation. in [:i,j] represents the intermediate feature map F in In the vector, we take the eigenvalues ​​of all channels at spatial position (i, j); C represents the number of channels, and H and W represent the intermediate feature maps F, respectively. in Height and width;

[0026] (22) Input the global illumination sensing vector U into two independent fully connected layers to generate scale modulation parameters respectively. Translation modulation parameters

[0027] γ=σ(W γ ·U+b γ )

[0028] β=W β ·U+B β

[0029] in: and Represents the learnable weight matrix. and σ(·) represents the learnable bias vector, and σ(·) represents the Sigmoid activation function. It ensures that the value of the scale modulation parameter γ is between (0, 1) to achieve smooth scaling adjustment. The scale modulation parameter γ and the translation modulation parameter β are used to control the magnification of the feature channel and the baseline offset, respectively.

[0030] (23) Using the scaling modulation parameter γ and the translation modulation parameter β to modify the intermediate feature map F in Perform a channel-wise affine transformation to generate an illumination enhancement feature map.

[0031] F enhanced =γ⊙F in +β

[0032] Where: ⊙ indicates multiplication by channel;

[0033] For intermediate feature map F in By performing a channel-by-channel affine transformation, the illumination sensing module can adaptively enhance important feature channels under low light conditions according to illumination conditions, while suppressing overexposed or noise-sensitive channels, which significantly improves the robustness of feature representation under complex illumination conditions.

[0034] Specifically, in Step 4, the initial set of predicted bounding boxes P is input into the adaptive prediction module. A self-attention weight mechanism is used to achieve global contextual interaction between the predicted bounding boxes, suppressing redundant predicted bounding boxes at the source and thus solving the problem of multiple detections in occluded scenes. This includes the following steps:

[0035] (41) Feature extraction and dimensionality reduction: For a prediction box p∈P, it is mapped to a feature vector f through a lightweight multilayer perceptron:

[0036]

[0037] Among them: MLP embed (·) denotes a lightweight multilayer perceptron, consisting of two fully connected layers and a ReLU activation function. The first fully connected layer increases the dimension to 256, and the second fully connected layer reduces the dimension to D.

[0038] (42) Top-K selection: To avoid excessive computational overhead, the top K predicted boxes with the highest object confidence scores are selected from all predicted boxes based on their object confidence scores. The corresponding feature vectors form the feature matrix F. TopKThis screening step can retain the predicted bounding boxes that are most likely to be the real targets while ensuring processing efficiency.

[0039] (43) Self-attention interaction: The feature matrix F TopK The input is fed into a standard Transformer encoding layer, and the feature matrix F is transformed through linear projection. TopK These are mapped to the query matrix Q, key matrix K, and value matrix V, respectively:

[0040] Q = F TopK ·W Q

[0041] K = F TopK ·W K

[0042] V = F TopK ·W V

[0043] Among them: W Q W K W V Let Q represent the learnable projective weight matrices of the query matrix Q, the key matrix K, and the value matrix V, respectively.

[0044] Calculate the self-attention weight matrix A0:

[0045]

[0046] Calculate context-aware features F attn :

[0047] F attn =A0·V

[0048] Feature F is enhanced through residual connections and layer normalization output. out :

[0049] F out =LayerNorm(F TopK +Dropout(F attn ))

[0050] Where: LayerNorm(·) represents the layer normalization operation, and Dropout(·) represents the random deactivation operation;

[0051] This step enables each prediction box to perceive the state of other prediction boxes, thereby learning to adjust its own expression in the global context;

[0052] (44) Refined prediction: Enhanced feature F out Input to Lightweight Prediction Head MLP refine The refined set of prediction boxes P is obtained. refined:

[0053] P refine =MLP refine (F out )

[0054] Among them: MLP refine (·) indicates a lightweight prediction head, consisting of three fully connected layers, with the middle layer using ReLU activation and the output layer using Sigmoid activation.

[0055] Specifically, in Step 5, the refined prediction box set P is obtained. refined It requires a series of rigorous post-processing steps to transform it into the final vehicle cropped image. This process is a crucial step in regressing the abstract numerical values ​​output by the neural network into concrete image data; the predicted bounding box set P refined The prediction box in the image is represented as p′∈P refined The process of cropping the vehicle image I from the input image I0 based on the predicted bounding box p′ includes the following steps:

[0056] (51) By performing a denormalization operation on the center point coordinates, the relative center coordinates are mapped back to the input image at scale I0:

[0057] x abs =x′×W0

[0058] y abs =y′×H0

[0059] (52) Map the relative width and height back to the input image I0 scale by performing a width and height inverse normalization operation:

[0060] w abs =w′×W0

[0061] h abs =h′×H0

[0062] (53) Convert the center-width-height representation to the top-left-bottom-right corner representation commonly used by image processing libraries (such as OpenCV):

[0063]

[0064] Where: round(·) represents rounding the floating-point coordinates to integer pixel coordinates, which is a necessary step in the matrix slicing operation of the image; (x′, y′, w′, h′) represent the bounding box parameters of the predicted box p′, (x abs y abs w abs h abs) represents the bounding box parameters of the predicted box p′ after denormalization, and (x1, y1) and (x2, y2) represent the integer pixel coordinates of the top left and bottom right corners of the predicted box β′.

[0065] The bounding box parameters (x′, y′, w′, h′) of the predicted box β′ are relative coordinates normalized to the interval [0, 1]. This design makes the model scale invariant, but it must be converted back to the absolute pixel coordinates of the original image through an inverse normalization operation before physical operations can be performed.

[0066] (54) Based on the integer pixel coordinates of the prediction box p′, the corresponding rectangular region [y1:y2:x1:x2,:] is cropped from the input image I0 to obtain the vehicle cropped image I; [y1:y2:x1:x2,:] means that the slice is from row y1 to row y2 in the height dimension and from column x1 to column x2 in the width dimension.

[0067] The cropped vehicle image I is a new, independent 3D tensor and the final output of the entire vehicle detection stage. The cropped vehicle image I will be sent to the feature matching stage. Through the implicit semantic prototype learning module and the prototype-aware fine-grained Transformer matching module, fine-grained features are extracted from the cropped vehicle image and matched efficiently and robustly with vehicle features in the image library, thus completing the vehicle re-identification task.

[0068] Specifically, in Step 6, the vehicle cropping image I is simultaneously acquired through a CNN backbone network and a feature pyramid network. A I B coarse-grained characteristics and fine-grained features Includes the following steps:

[0069] (61) Cropping the vehicle image I A I B Common input to the CNN backbone network (such as ResNet-50); vehicle cropped image I A Intermediate layer feature maps are generated through the intermediate layers of the CNN backbone network. The final layer feature map is generated by the final layer of the CNN backbone network. Vehicle cropped image I B Intermediate layer feature maps are generated through the intermediate layers of the CNN backbone network. The final layer feature map is generated by the final layer of the CNN backbone network.

[0070] (62) The feature pyramid network has a five-layer structure, named P2, P3, P4, P5, and P6 from low to high; the feature maps of the middle layers are... and final layer feature map The input is fed into the feature pyramid network, and the output of layer P5 is used as a coarse-grained feature map. The output of layer P3 is used as a fine-grained feature map. intermediate layer feature map and final layer feature map The input is fed into the feature pyramid network, and the output of layer P5 is used as a coarse-grained feature map. The output of layer P3 is used as a fine-grained feature map.

[0071] Specifically, in Step 7, the following steps are used: Represents coarse-grained feature map or Will The input is fed into the implicit semantic prototype learning module to obtain the reconstructed feature map F′, including the following steps:

[0072] (71) Define a learnable semantic prototype vector e k K represents the semantic prototype of the k-th vehicle structural component, and K1 is the number of semantic prototypes. The semantic prototype is randomly initialized before training, and its specific semantics (such as corresponding to components like wheels, headlights, and windows) do not depend on any manual annotation, but are learned autonomously by the network through data-driven methods during training.

[0073] (72) For semantic prototypes The eigenvector f at spatial location (i, j) ij Calculate its relationship with the semantic prototype e k The cosine similarity is calculated, and the probability distribution of spatial position (i, j) belonging to each semantic prototype is obtained through the Softmax function:

[0074]

[0075] Where: s ij，k Represents the eigenvector f ij With semantic prototype e k The initial similarity between them, a ij，k Represents the eigenvector f ij Belongs to the semantic prototype e k The probability of;

[0076] (73) The feature vectors at each spatial location in the equation belong to the semantic prototype e. k The probability {a} ij，k} constitutes the semantic assignment vector A k Semantic assignment vector A k It is a two-dimensional response graph that intuitively represents the different spatial locations and semantic prototypes e in the cropped vehicle image I. kThe degree of correlation, will {A k Let} be denoted as soft semantic allocation graph A;

[0077] (74) Using the soft semantic assignment map A as weights, the semantic prototype vector E is weighted and summed to generate a new reconstructed feature map F′ that has undergone semantic refinement and enhancement:

[0078]

[0079] Where: ⊙ represents channel-by-channel multiplication; this operation is equivalent to re-describing the input features with a set of learned "standard parts" (semantic prototypes), which strengthens features related to stable structures and suppresses noise and irrelevant details.

[0080] (75) The coarse-grained feature map The soft semantic allocation graph is denoted as A. A A B coarse-grained features The reconstructed feature map is denoted as F. A ′, F B ′.

[0081] Specifically, in Step 8, the feature map F is reconstructed. A ′, F B '、Soft semantic allocation graph A A A B The input is a fine-grained Transformer matching module, which performs self-attention and cross-attention interactions to obtain the vehicle cropping image I. A and I B A series of matching point pairs between them, including the following steps:

[0082] (81) Reconstruct the feature map F A ′, F B Expanding the feature map in spatial dimensions transforms it into a sequence of feature vectors, essentially merging the height and width dimensions into a single dimension and adding a sinusoidal positional encoding G, thus reconstructing the feature map F. A ′, F B Convert ' to X A X B :

[0083] X A =Flatten(F A ′)+G

[0084] X B =Flatten(F B ′)+G

[0085] Where Flatten(·) represents the mathematical operation of converting a two-dimensional spatial feature map into a one-dimensional sequence;

[0086] (82) The feature sequence X after feature preprocessing A X B Recorded as As the initial input to the context-aware module, the context-aware module consists of N C The nth layer is composed of a series of self-attention and cross-attention composite layers, and the input of the nth composite layer is... The output is The output of the last composite layer Let Z be the context-aware feature. A Z B ;

[0087] (83) Calculate the feature similarity matrix The element s at position (i, j) ij Z represents A (i) and Z B Similarity between (j):

[0088]

[0089] Where: τ is the temperature parameter used to adjust the sharpness of the Softmax distribution, and Z... A (i) represents the context-aware feature Z A The vector at position i, Z B (j) represents the context-aware feature Z. B The vector at the j-th position, <Z A (i), Z B (j)> indicates Z A (i) and Z B The inner product of (j);

[0090] (84) Use double soft maximum calculation to calculate the matching confidence matrix P C The element c at position (i, j) ij Z represents A (i) and Z B Confidence level between (j):

[0091] c ij =Softmax(S(i,·)) j ·Softmax(S(·,j)) i

[0092] Where: S(i,·) represents all elements in the i-th row of the feature similarity matrix S, S(·,j) represents all elements in the j-th column of the feature similarity matrix S, and Softmax(S(i,·)) j Z represents A (i) Matched ZB The probability of (j), Softmax(S(·,j)) i Z represents B (j) Matches Z A The probability of (i);

[0093] This step performs Softmax in both the row and column directions, then takes the product at corresponding positions, and the result c ij It can be regarded as Z A (i) and Z B (j) is the soft probability of mutual nearest neighbors;

[0094] (85) Set the confidence threshold θ C If c ij ≥θ c If Z satisfies the mutual nearest neighbor criterion, then Z is considered to satisfy the mutual nearest neighbor criterion. A (i) and Z B (j) are the most likely matching pairs of points, which is naturally reflected by the double soft maximum matrix, and (Z) A (i), Z B (j) Add to the set of matching point pairs M C ;

[0095] (86) For each matching point pair (Z) A (i), Z B (j))∈M C Map its location to fine-grained features Above, with the mapped position as the center, in fine-grained features Each local window of size w×w is cropped, and the features within the two local windows are input into a lightweight Transformer module for local feature interaction and transformation to obtain a refined feature representation.

[0096] (87) with With the central feature vector, Calculate the central feature vector for the window. With window The correlation of all feature vectors within the context is analyzed and a heatmap is generated. The expectation of the probability distribution represented by the generated heatmap is then calculated to obtain the context-aware feature Z. B The middle corresponds to the context-aware feature Z. A The sub-pixel precise position j′ of position i is given by (i, j′, c). ij Add the refined set of matching point pairs M f ;

[0097] (88) Calculate vehicle cropping image I based on refined matching point pairs A and I BSimilarity score between AB :

[0098]

[0099] Where: |M f | represents the refined set of matching point pairs M. f The number of matching point pairs contained therein;

[0100] Similarity score S AB It comprehensively reflects the structural similarity of vehicles at the component level, thereby achieving accurate and robust vehicle re-identification.

[0101] Beneficial Effects: The fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios provided by this invention has the following advantages compared with existing technologies: 1. The illumination adaptive module in this invention improves the model's detection capability under low-light conditions such as nighttime and shadows; 2. The adaptive prediction module in this invention suppresses redundant boxes from the source, effectively solving the problem of multiple detection in occlusion scenarios; 3. The implicit semantic prototype learning module proposed in this invention enables the network to learn the implicit semantic representation of vehicle components autonomously in an unsupervised manner without any component-level annotation. It can discover and utilize the common vehicle structures that are stable under different viewpoints and illuminations, enhancing the robustness of feature representation and the interpretability of the model; 4. The prototype-aware matching module designed in this invention injects high-level semantic information into the low-level feature matching process in the form of affinity bias, guiding the matching algorithm to prioritize establishing correspondences between semantically consistent components, improving the matching accuracy and robustness under complex challenges such as severe occlusion, sudden changes in viewpoint, and changes in illumination. Attached Figure Description

[0102] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention;

[0103] Figure 2 This is a schematic diagram of the framework of a vehicle detection algorithm based on illumination perception and adaptive prediction.

[0104] Figure 3 This is a structural block diagram of the light sensing module;

[0105] Figure 4 Here is a block diagram of the adaptive prediction module;

[0106] Figure 5 A schematic diagram of the framework of a fine-grained Transformer feature matching algorithm based on implicit semantic prototype awareness.

[0107] Figure 6 This is a structural diagram of the implicit semantic prototype learning module;

[0108] Figure 7This is a block diagram of the fine-grained Transformer matching module. Detailed Implementation

[0109] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0110] This invention provides a fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios, enabling robust vehicle identification in complex environments. For example... Figure 2 As shown, the vehicle detection phase employs an algorithm framework based on illumination perception and adaptive prediction, including, for example... Figure 3 The light sensing module (LAM) shown and as shown Figure 4 The Adaptive Prediction Module (APM) shown below: After multi-scale feature extraction of the input image, the illumination perception module enhances illumination invariance, and then the adaptive prediction module suppresses redundant prediction boxes, finally outputting a cropped vehicle image. Figure 5 As shown, the feature matching stage employs a fine-grained Transformer architecture based on implicit semantic prototype awareness, including, for example... Figure 6 The Implicit Semantic Prototype Learning Module (ISPM) shown and such Figure 7 The prototype-aware fine-grained Transformer matching module (PATM) is shown. After the features of the cropped vehicle image are extracted by the CNN backbone network, the implicit semantic prototype learning module decouples the vehicle structural components in an unsupervised manner. Then, the semantic information is injected into the feature matching process in the form of affinity bias through the prototype-aware matching mechanism. Finally, the matching point pairs are output and the similarity score is calculated to complete the re-identification decision.

[0111] like Figure 1 The diagram shown is a flowchart of the implementation of the present invention. The steps are described in detail below.

[0112] Step S01: Input the original traffic scene image.

[0113] Given an input image H0 and W0 represent the height and width of the input image I0, respectively, and 3 represents the three RGB color channels. The input image I0 is fed into the YOLOv13 backbone network built on a deep convolutional neural network for multi-scale feature extraction; the YOLOv13 backbone network consists of a set of DS-C3k2 modules based on depthwise separable convolution, used to capture hierarchical feature representations from low-level texture to high-level semantics.

[0114] Step S02: Process the input image using a vehicle detection algorithm based on illumination perception and adaptive prediction, and output a cropped vehicle image.

[0115] Design a... Figure 2The vehicle detection algorithm shown includes a lighting perception module and an adaptive prediction module. The specific process of the vehicle detection algorithm is explained below.

[0116] Step 21: Output intermediate feature maps from the shallow extraction layer (second DS-C3k2 module) of the YOLOv13 backbone network. Where C represents the number of channels (typical value C = 256), and H and W represent the intermediate feature maps F, respectively. in Height and width (usually) ).

[0117] Step 22: Transfer the intermediate feature map F in The input is fed into a specially designed illumination sensing module, which generates a set of illumination sensing modulation parameters for the intermediate feature map F. in Perform a channel-by-channel affine transformation to obtain the illumination enhancement feature map F. enhanced The specific processing procedure is as follows.

[0118] Step 221: Apply Global Average Pooling (GAP) along the spatial dimension to the intermediate feature map F in Compression is performed to generate a global illumination sensing vector.

[0119]

[0120] Among them: U={u c}, u c F represents the average response intensity of the c-th channel feature across the entire spatial range; in [:i,j] represents the intermediate feature map F in In the vector, we take the eigenvalues ​​of all channels at spatial position (i, j).

[0121] Step 222: Input the global illumination sensing vector U into two independent fully connected layers to generate scale modulation parameters respectively. Translation modulation parameters

[0122] γ=σ(W γ ·U+b γ )

[0123] β=W β ·U+B β

[0124] in: and Represents the learnable weight matrix. and Let represent the learnable bias vector, and σ(·) represent the Sigmoid activation function.

[0125] Step 223: Use the scaling modulation parameter γ and the shift modulation parameter β to modify the intermediate feature map F. in Perform a channel-wise affine transformation to generate an illumination enhancement feature map.

[0126] F enhanced =γ⊙F in +β

[0127] Where: ⊙ represents channel-by-channel multiplication (i.e., multiplying each channel scalar of the scale modulation parameter γ by the intermediate feature map F). in The corresponding spatial location feature values ​​of the channel, plus the offset scalar of the channel corresponding to the translation modulation parameter β.

[0128] Step 23: Enhance the illumination feature map F enhanced Returning to the YOLOv13 backbone network to continue forward propagation, it participates in the feature processing flow of the global correlation enhancement module of the YOLOv13 backbone network, and outputs the initial set of predicted bounding boxes in the detection head of the YOLOv13 backbone network.

[0129] Step 24: Input the initial set of prediction boxes P into the adaptive prediction module to realize global context interaction between prediction boxes, suppress redundant detection boxes from the source, and effectively solve the problem of multiple predictions in occluded scenes.

[0130] Step 241: Feature extraction and dimensionality reduction, generating high-dimensional feature representations for each prediction box.

[0131] For a prediction box p∈P, p=(x,y,w,h,obj) score cls score The feature vector f is mapped to a D-dimensional feature vector through a lightweight multilayer perceptron.

[0132]

[0133] Where: (x, y, w, h) represent the bounding box parameters of the predicted box p, and obj score cls represents the object confidence of the prediction box p. score The predicted bounding box p represents the class probability distribution; D represents the feature dimension (typical value D = 128); Lightweight Multilayer Perceptron (MLP) embed It consists of two fully connected layers and a ReLU activation function. The first fully connected layer increases the dimension to 256, and the second fully connected layer reduces the dimension to D.

[0134] Step 242, Top-K filtering.

[0135] Based on object confidence level obj scoreFrom all predicted bounding boxes, the TopK (typically TopK = 500) predicted bounding boxes with the highest object confidence are selected, and the corresponding D-dimensional feature vectors f form a feature matrix.

[0136] Step 243, Self-attention interaction.

[0137] The feature matrix F obtained after filtering TopK The input is fed into a standard Transformer encoding layer, and the feature matrix F is transformed through linear projection. TopK These are mapped to the query matrix Q, the key matrix K, and the value matrix V, respectively:

[0138] Q = F TopK ·W Q

[0139] K = F TopK ·W K

[0140] V = F TopK ·W V

[0141] Where: Q represents the information request sent by each prediction box to other prediction boxes, K represents the identity of each prediction box (used to respond to queries), and V represents the original feature information to be transmitted by each prediction box; and The corresponding learnable projective weight matrix (usually d) k =d v =D).

[0142] Calculate the self-attention weight matrix A0:

[0143]

[0144] Calculate context-aware features F attn :

[0145]

[0146] Feature F is enhanced through residual connections and layer normalization output. out :

[0147] F out =LayerNorm(F TopK +Dropout(F attn ))

[0148] Where: LayerNorm(·) represents the layer normalization operation, which is used to standardize the feature vector of each prediction box to stabilize the training process and accelerate convergence; Dropout(·) represents the random deactivation operation, which enhances the robustness of the model and prevents overfitting by randomly masking a portion of the self-attention output during the training phase.

[0149] Step 244, Refined Prediction.

[0150] Enhance feature F out Input to Lightweight Prediction Head MLP refine The refined set of prediction boxes P is obtained. refined :

[0151] P refine =MLP refine (F out )

[0152] Among them: Lightweight Predictive Head MLP refine It consists of three fully connected layers, with the middle layer using ReLU activation and the output layer using Sigmoid activation to ensure an appropriate numerical range.

[0153] To distinguish it from the initial set of predicted bounding boxes P, the refined set of predicted bounding boxes P refined The predicted bounding box in the image is denoted as p′, the bounding box parameters of the predicted bounding box p′ are denoted as (x′, y′, w′, h′), and the object confidence is denoted as obj. score The category probability distribution is represented as cls. score ′.

[0154] Step 25: Set the refined prediction box set P refined After being output from the vehicle detection stage based on illumination perception and adaptive prediction, it needs to be converted into the final cropped vehicle image I through a series of rigorous post-processing steps; this process is the key step in reverting the abstract numerical values ​​output by the neural network to specific image data.

[0155] Step 251: Post-processing and coordinate transformation.

[0156] By performing inverse normalization of the center point coordinates, the relative center coordinates are mapped back to the input image at scale I0:

[0157] x abs =x′×W0

[0158] y abs =y′×H0

[0159] By performing inverse normalization of width and height, the relative width and height are mapped back to the input image at scale I0:

[0160] w abs=w′×W0

[0161] h abs =h′×H0

[0162] Convert the center-width-height representation to the top-left-bottom-right corner representation commonly used by image processing libraries (such as OpenCV):

[0163]

[0164] Where: round(·) indicates rounding the floating-point coordinates to integer pixel coordinates, (x abs y abs w abs h abs (x1, y1) represents the bounding box parameters of the predicted bounding box p′ after denormalization, and (x2, y2) represents the integer pixel coordinates of the top left and bottom right corners of the predicted bounding box p′.

[0165] Step 251: To ensure the system only processes high-confidence vehicle targets, a filtering threshold is set to ensure the validity and reliability of the predicted boxes, resulting in a set of valid predicted boxes (ValidBoxes).

[0166] ValidBoxes = {box∈P} refined |obj score ′>θ}

[0167] Where: box represents the valid predicted bounding box retained after filtering, θ is a preset threshold (usually between 0.2 and 0.5); object confidence score obj score Only predicted bounding boxes above this threshold are considered to be actual targets, while those below this threshold are considered background or noise and are discarded.

[0168] Step 253: Traverse the set of valid prediction boxes. For each valid prediction box, crop the corresponding rectangular region [y1:y2:x1:x2,:] from the input image I0 to obtain the vehicle cropped image I; [y1:y2:x1:x2,:] means slicing from row y1 to row y2 in the height dimension and from column x1 to column x2 in the width dimension, while retaining all 3 color channels.

[0169] Step S03: End-to-end training of the vehicle detection stage based on illumination perception and adaptive prediction.

[0170] The training dataset used in the vehicle detection phase should contain a large number of vehicle images from real traffic scenes, covering various lighting conditions (e.g., daytime, nighttime, shadows, rain, snow), different degrees of occlusion (including partial and severe occlusion), and various viewpoint variations. The dataset should include high-quality bounding box annotations (vehicle locations) and vehicle identification labels (e.g., publicly available datasets such as VehicleID, VeRi-776, or self-built datasets). To further enhance the model's generalization ability, data augmentation strategies can be employed during training, including but not limited to: random brightness / contrast adjustment, simulated rain and snow noise, random cropping, and horizontal flipping. These augmentations help improve the model's robustness in complex environments, ensuring that the lighting perception module and adaptive prediction module have good adaptability in real-world scenes. It is recommended that the training data consist of no fewer than 100,000 vehicle images, covering at least 500 vehicle identification categories, to ensure that the model can fully learn the common structural features and individual differences of vehicles.

[0171] During the vehicle detection phase, the end-to-end training process is carried out after the entire network forward propagation is completed. The Hungarian matching algorithm is used for end-to-end training. This training strategy forces a one-to-one match between the prediction and the real target, which can fundamentally solve the redundancy problem caused by one real target corresponding to multiple prediction boxes in traditional methods.

[0172] Step S04: Input the cropped vehicle image to be matched into the CNN backbone network with shared weights to extract multi-scale feature maps.

[0173] To ensure input consistency and optimize computational efficiency for the feature matching network, an image size standardization step is added before inputting the cropped vehicle images into the CNN backbone network: all cropped vehicle images are uniformly rescaled to H′×W′=512×512 using image scaling algorithms such as bilinear interpolation. This design ensures that the feature matching network can receive inputs of uniform size, avoiding training instability caused by inconsistent input sizes.

[0174] Take two cropped vehicle images to be matched. A I B This constitutes a vehicle cropped image pair (I A I B ); crop the vehicle image pair (I A I B The input is fed into a CNN backbone network with shared weights. A Feature Pyramid Network (FPN) is then connected to the back end of the CNN backbone network. Through a top-down path and lateral connections, feature maps from different levels are fused together, and coarse-grained feature maps containing rich semantic information are obtained. and fine-grained feature maps that retain detailed information

[0175] The CNN backbone network generates feature maps of different resolutions at different depths, where intermediate layers of the CNN backbone network generate intermediate layer feature maps. Its size is the vehicle cropped image I A of (Right now This feature map retains good spatial details, preserving more local details such as edges and textures, which can be used for subsequent fine-tuning of matching positions; the final layer of the CNN backbone network generates the final layer feature map. Its size is the vehicle cropped image I A of (Right now This feature map has the largest receptive field and the richest semantic information, making it suitable for capturing the overall structure of a vehicle and the relationships between its components.

[0176] For vehicle cropped image I B Similarly, the intermediate layer feature map is obtained. Final layer feature map

[0177] intermediate layer feature map and final layer feature map The input is fed into the feature pyramid network, where high-level semantic features and low-level detailed features are fused through a top-down path and lateral connections. The feature pyramid network has a five-layer structure, named P2, P3, P4, P5, and P6 from lowest to highest. The feature maps from the middle layers are then processed... and final layer feature map The input is fed into the feature pyramid network, and the output of layer P5 is used as a coarse-grained feature map. The output of layer P3 is used as a fine-grained feature map.

[0178] For vehicle cropped image I B Similarly, coarse-grained feature maps are obtained. Fine-grained feature map

[0179] Step S05: Input the multi-scale features into the implicit semantic prototype learning module to generate a soft semantic assignment map and a semantically enhanced reconstructed feature map.

[0180] Design a... Figure 5 The implicit semantic prototype learning module shown is a differentiable processing layer connected after the CNN backbone network and before the fine-grained Transformer matching module; its input is the coarse-grained features extracted by the CNN backbone network. and fine-grained features The output is the semantically refined reconstructed feature map F. A ′, FB ′ and soft semantic allocation graph A A A B Coarse-grained features and fine-grained features The input is fed into the implicit semantic prototype learning module, which learns the implicit semantic representation of vehicle components autonomously in an unsupervised manner.

[0181] Step 51: Define and initialize the semantic prototype.

[0182] Define a learnable semantic prototype vector e k K1 represents the semantic prototype of the k-th vehicle structural component, where K1 is the number of semantic prototypes (typical value K1 = 16).

[0183] Step 52: Generate a soft semantic assignment graph.

[0184] For semantic prototypes The eigenvector f at spatial location (i, j) ij Calculate its relationship with the semantic prototype e k The cosine similarity is calculated, and the probability distribution of spatial position (i, j) belonging to each semantic prototype is obtained through the Softmax function:

[0185]

[0186] Where: s ij，k Represents the eigenvector f ij With semantic prototype e k The initial similarity between them, a ij，k Represents the eigenvector f ij Belongs to the semantic prototype e k The probability of.

[0187] The feature vectors at each spatial location in the equation belong to the semantic prototype e. k The probability {a} ij，k} constitutes the semantic assignment vector A k Semantic assignment vector A k It is a two-dimensional response graph that intuitively represents the different spatial locations and semantic prototypes e in the cropped vehicle image I. k The degree of correlation, will {A k Let A be the soft semantic assignment graph.

[0188] Step 53: Prototype-guided feature reconstruction.

[0189] Using the soft semantic assignment map A as weights, the semantic prototype vector E is weighted and summed to generate a new reconstructed feature map F′ that has undergone semantic refinement and enhancement:

[0190]

[0191] Where: ⊙ represents channel-by-channel multiplication; this operation is equivalent to re-describing the input features with a set of learned "standard parts" (semantic prototypes), which strengthens features related to stable structures and suppresses noise and irrelevant details.

[0192] Step S06: Input the reconstructed feature map, fine-grained feature map, and soft semantic assignment map into the fine-grained Transformer matching module to perform self-attention and cross-attention interaction, obtain high-precision matching point pairs, and calculate similarity scores.

[0193] Step 61: Input the reconstructed feature map and soft semantic assignment map into the fine-grained Transformer matching module to perform self-attention and cross-attention interaction, and introduce semantic affinity bias.

[0194] Design a... Figure 3 The prototype-aware fine-grained Transformer matching module shown takes the reconstructed feature map F as input. A ′, F B '、Soft semantic allocation graph A A A B .

[0195] Step 611: Feature preprocessing and location encoding.

[0196] Reconstruct the feature map F A ′, F B Unfolding the feature map in the spatial dimension, it is transformed into a sequence of feature vectors, that is, merging the height and width dimensions into one dimension, and adding a sinusoidal positional encoding G (a two-dimensional extension following DETR) to inject spatial positional information, thus reconstructing the feature map F. A ′, F B Convert ' to X A X B :

[0197] X A =Flatten(F A ′)+G

[0198] X B =Flatten(F B ′)+G

[0199] Where Flatten(·) represents the mathematical operation of converting a two-dimensional spatial feature map into a one-dimensional sequence.

[0200] Step 612: Context awareness is achieved through alternating layers of self-attention and cross-attention.

[0201] The feature sequence X after feature preprocessing A X B Recorded as As the initial input to the context-aware module, the context-aware module consists of N C (usually N) C =4) self-attention and cross-attention composite layers are connected in series, and the input of the nth composite layer is The output is The output of the last composite layer Let Z be the context-aware feature. A Z B The nth composite layer contains two inputs. and two outputs Processing is achieved through a self-attention layer and two cross-attention layers that share the same set of learnable projective weight matrices; the self-attention layer is used to realize feature interactions within the image, with the input being... The output is The cross-attention layer is used to realize feature interaction between images, and the input is... The output is

[0202] Step 6121 Each self-attention layer is used to aggregate contextual information within the image:

[0203]

[0204] Where: SelfAttn(·) represents self-attention operation, and LinearAttention(·) represents linear attention operation; express The query point, key point, and value in the text. express The query point, key point, and value in the text. This represents the learnable projective weight matrix in the nth composite layer;

[0205]

[0206] Where: φ(·)=ELU(·)+1 is the feature mapping function, and ELU(·) represents the exponential linear unit activation function; the self-attention layer uses a linear attention mechanism, which mainly processes the feature interaction within a single image, effectively capturing the spatial context relationship of vehicle parts through linear complexity, and ensuring computational efficiency.

[0207] Step 6122, Input two cross-attention layers, the first cross-attention layer with... For query, with Key-value calculation exist Representation in context The second cross-attention layer For query, with Key-value calculation exist Representation in context

[0208]

[0209] in: express The query point, key point, and value in the text. express The query points, key points, and values ​​in the text.

[0210] Step 6123, in calculation Inquiry point and Key points When determining the matching degree, a semantic affinity bias matrix B is introduced. sem The element at position (i, j) By extracting the soft semantic allocation graph A A A B The vector corresponding to the spatial location is obtained by calculation:

[0211]

[0212] Where: λ0 is a learnable scale parameter, Represents the soft semantic allocation graph A A The corresponding extracted The vector at the i-th spatial location, Represents the soft semantic allocation graph A B The corresponding extracted The vector at the j-th spatial location.

[0213] In calculation Inquiry point and Key points When determining the matching degree, the semantic affinity bias matrix B is used. sem transpose vector B semT .

[0214] Step 6124: In the cross-attention layer, calculate the attention score:

[0215]

[0216] Where: CrossAttn(·) represents the cross-attention operation, DX The dimension of the input feature.

[0217] The context-aware module means that if the query point and key points If they are assigned to the same semantic prototype (i.e., their semantic assignment vectors are extremely similar), they will receive an additional positive incentive when matching. This makes the matching process no longer a blind low-level appearance matching, but rather it is carried out at the level of "semantic components" discovered autonomously by the network. It prioritizes establishing correspondences between semantically consistent regions (such as wheel to wheel, headlight to headlight), thereby significantly improving the matching accuracy and robustness in complex scenarios such as changes in perspective and large differences in appearance.

[0218] Step 62: Perform coarse-grained matching and fine-grained matching to obtain high-precision matching point pairs.

[0219] Step 621: Calculate the feature similarity matrix The element s at position (i, j) ij Z represents A (i) and Z B Similarity between (j):

[0220]

[0221] Where: τ is the temperature parameter used to adjust the sharpness of the Softmax distribution, and Z... A (i) represents the context-aware feature Z A The vector at position i, Z B (j) represents the context-aware feature Z. B The vector at the j-th position, <Z A (i), Z B (j)> indicates Z A (i) and Z B The inner product of (j).

[0222] Step 622: Use double-soft maximum calculation to calculate the matching confidence matrix P. C The element c at position (i, j) ij Z represents A (i) and Z B Confidence level between (j):

[0223] c ij =Softmax(S(i,·)) j ·Softmax(S(·,j)) i

[0224] Where: S(i,·) represents all elements in the i-th row of the feature similarity matrix S, S(·,j) represents all elements in the j-th column of the feature similarity matrix S, and Softmax(S(i,·)) j Z represents A (i) Matched Z B The probability of (j), Softmax(S(·,j)) i Z represents B (j) Matches Z A The probability of (i).

[0225] Step 623: Set the confidence threshold θ C If c ij ≥θ C Then Z is considered A (i) and Z B (j) are the most likely matching pairs of points, which is naturally reflected by the double soft maximum matrix, and (Z) A (i), Z B (j) Add to the set of matching point pairs M C .

[0226] Step 624: For each matching point pair (Z) A (i), Z B (j))∈M C Map its location to fine-grained features Above, with the mapped position as the center, in fine-grained features Each local window is cropped to a size of w×w (e.g., w=5). The features within both local windows are then input into a lightweight Transformer module (containing N). f Layers, typically N f =1) Perform local feature interactions and transformations to obtain refined feature representations.

[0227] Step 625, with With the central feature vector, Calculate the central feature vector for the window. With window The correlation of all feature vectors within the context is analyzed and a heatmap is generated. The expectation of the probability distribution represented by the generated heatmap is then calculated to obtain the context-aware feature Z. B The middle corresponds to the context-aware feature Z. A The sub-pixel precise position j′ of position i is given by (i, j′, c). ij′ Add the refined set of matching point pairs M f .

[0228] Step 63: Calculate the similarity score between the two cropped vehicle images based on the matching point pairs, and make a vehicle re-identification decision based on the score.

[0229] The vehicle cropping image I is calculated based on the refined matching point pairs. A and I B Similarity score between AB To make re-identification decisions:

[0230]

[0231] Where: |M f | represents the refined set of matching point pairs M. f The number of matching point pairs contained therein.

[0232] Step S07: End-to-end training of fine-grained Transformer feature matching stage based on implicit semantic prototype awareness.

[0233] The feature matching network is a deep learning model that requires end-to-end training. It includes a CNN backbone, a semantic prototype module, and a fine-grained Transformer matcher, all of which need to be jointly trained end-to-end. Its training objective is to enable the model to simultaneously learn how to decouple the semantic components of a vehicle, establish accurate component-level correspondences, and output highly discriminative re-identification similarity scores. The model is trained on a dataset containing a large number of vehicle image pairs. Each sample in this dataset contains: a pair of vehicle images from different viewpoints (I... A I B The labels indicate whether the image pair belongs to the same vehicle (positive sample pairs for the same vehicle, negative sample pairs for different vehicles); the camera pose parameters and depth map provided for each positive sample pair are used to calculate the pixel-level true correspondence required for the above loss function. The entire network is trained by minimizing a total loss function that combines multi-task learning. The total loss function is defined as a weighted sum of coarse-grained matching loss, fine-grained matching loss, and prototype consistency loss.

[0234] During training, the optimizer (AdamW) calculates the gradient of the total loss with respect to all learnable parameters (including CNN weights, semantic prototype vector E, fine-grained Transformer module parameters, scale parameter λ0, etc.) using the backpropagation algorithm, and updates these parameters to minimize the total loss. This process iterates until the model converges, ultimately resulting in an end-to-end re-identification system capable of autonomously discovering vehicle parts, performing accurate matching, and outputting high-discriminative similarity scores.

[0235] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios, characterized in that: This includes a vehicle detection stage based on illumination perception and adaptive prediction, and a fine-grained Transformer feature matching stage based on implicit semantic prototype perception. The vehicle detection stage based on illumination perception and adaptive prediction includes an illumination perception module and an adaptive prediction module; the illumination perception module is used to achieve adaptive image enhancement under different illumination conditions. The adaptive prediction module suppresses redundant prediction boxes through a self-attention weight mechanism. The fine-grained Transformer feature matching stage based on implicit semantic prototype awareness includes an implicit semantic prototype learning module and a fine-grained Transformer matching module. The implicit semantic prototype learning module generates a soft semantic assignment graph by unsupervised decoupling of vehicle structural components through a set of learnable semantic prototypes. The fine-grained Transformer matching module introduces semantic affinity bias in cross-attention.

2. The fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios according to claim 1, characterized in that: The method includes the following steps: Step 1: Given an input image I0, input image I0 is fed into the YOLOv13 backbone network built based on a deep convolutional neural network. The shallow extraction layer of the YOLOv13 backbone network outputs the intermediate feature map F. in ; Step 2, transfer the intermediate feature map F in The input is fed into the illumination sensing module, which generates a set of illumination sensing modulation parameters. Based on these parameters, the intermediate feature map F is processed. in Perform a channel-by-channel affine transformation to obtain the illumination enhancement feature map F. enhanced ; Step 3: Enhance the illumination feature map F enhanced Returning to the YOLOv13 backbone network to continue forward propagation, it participates in the feature processing flow of the global correlation enhancement module of the YOLOv13 backbone network, and outputs the initial prediction box set P in the detection head of the YOLOv13 backbone network. Step 4: Input the initial set of predicted bounding boxes P into the adaptive prediction module. A self-attention weight mechanism is used to achieve global contextual interaction between the predicted bounding boxes, suppressing redundant predicted bounding boxes and obtaining a refined set of predicted bounding boxes P. refined ; Step 5: Based on the refined set of prediction boxes P refined Cropped vehicle image I is extracted from input image I0; Step 6: Crop the two vehicle images to be matched. A I B Simultaneously, the input is fed into a CNN backbone network with shared weights, and a feature pyramid network is connected to the back end of the CNN backbone network. Through a top-down path and lateral connections, features from different levels are fused, thereby obtaining coarse-grained features containing rich semantic information. and fine-grained features that retain detailed information Step 7: Transfer coarse-grained features The input is fed into the implicit semantic prototype learning module, and the output is a reconstructed feature map F after semantic refinement. A ′, F B '、Soft semantic allocation graph A A A B ; Step 8: Reconstruct the feature map F A ′, F B '、Soft semantic allocation graph A A A B Fine-grained characteristics Input the fine-grained Transformer matching module to obtain the vehicle cropped image I. A and I B A series of matching point pairs are used to calculate the vehicle cropping image I based on the matching point pairs. A and I B Similarity score between AB .

3. The fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios according to claim 1, characterized in that: In Step 1, the YOLOv13 backbone network consists of a set of DS-C3k2 modules based on depthwise separable convolutions, used to capture hierarchical feature representations from low-level textures to high-level semantics.

4. The fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios according to claim 2, characterized in that: In Step 2, the illumination sensing module is a light quantum network used to generate a set of illumination sensing modulation parameters and to process the intermediate feature map F. in Perform a channel-by-channel affine transformation to obtain the illumination enhancement feature map F. enhanced It includes the following steps: (21) Use global average pooling operation to process intermediate feature maps along the spatial dimension. Compression is performed to generate a global illumination sensing vector. Among them: U={u c }, u c F represents the average response intensity of the c-th channel feature across the entire spatial range; in [:i,j] represents the intermediate feature map F in In the vector, we take the eigenvalues ​​of all channels at spatial position (i, j); C represents the number of channels, and H and W represent the intermediate feature maps F, respectively. in Height and width; (22) Input the global illumination sensing vector U into two independent fully connected layers to generate scale modulation parameters respectively. Translation modulation parameters γ=σ(W γ ·U+b γ ) β=W β ·U+b β in: and Represents the learnable weight matrix. and σ(·) represents the learnable bias vector, and σ(·) represents the Sigmoid activation function. (23) Using the scaling modulation parameter γ and the translation modulation parameter β to modify the intermediate feature map F in Perform a channel-wise affine transformation to generate an illumination enhancement feature map. F enhanced =γ⊙F in +b Where: ⊙ indicates multiplication by channel.

5. The fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios according to claim 2, characterized in that: In Step 4, the initial set of predicted bounding boxes P is input into the adaptive prediction module. A self-attention weight mechanism is used to realize global contextual interaction between the predicted bounding boxes, suppressing redundant predicted bounding boxes and obtaining a refined set of predicted bounding boxes P. refined It includes the following steps: (41) Feature extraction and dimensionality reduction: For a prediction box p∈P, it is mapped to a feature vector f through a lightweight multilayer perceptron: Among them: MLP embed (·) denotes a lightweight multilayer perceptron, consisting of two fully connected layers and a ReLU activation function. The first fully connected layer increases the dimension to 256, and the second fully connected layer reduces the dimension to D. (42) Top-K selection: Based on the object confidence of the predicted bounding boxes, select the top K predicted bounding boxes with the highest object confidence from all predicted bounding boxes, and the corresponding feature vectors form the feature matrix F. TopK ; (43) Self-attention interaction: The feature matrix F TopK The input is fed into a standard Transformer encoding layer, and the feature matrix F is transformed through linear projection. TopK These are mapped to the query matrix Q, key matrix K, and value matrix V, respectively: Q=F TopK ·W Q K=F TopK ·W K V=F TopK ·W V Among them: W Q W K W V Let Q represent the learnable projective weight matrices of the query matrix Q, the key matrix K, and the value matrix V, respectively. Calculate the self-attention weight matrix A0: Calculate context-aware features F attn : F attn =A0·V Feature F is enhanced through residual connections and layer normalization output. out : F out =LayerNorm(F TopK +Dropout(F attn )) Where: LayerNorm(·) represents the layer normalization operation, and Dropout(·) represents the random deactivation operation; (44) Refined prediction: Enhanced feature F out Input to Lightweight Prediction Head MLP refine The refined set of prediction boxes P is obtained. refined : P refine =MLP refine (F out ) Among them: MLP refine (·) indicates a lightweight prediction head, consisting of three fully connected layers, with the middle layer using ReLU activation and the output layer using Sigmoid activation.

6. The fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios according to claim 2, characterized in that: In Step 5, the refined prediction box set P refined The prediction box in the image is represented as p′∈P refined The process of cropping the vehicle image I from the input image I0 based on the predicted bounding box p′ includes the following steps: (51) Map the relative center coordinates back to the input image at scale I0 by performing a denormalization operation on the center point coordinates: x abs =x′×W0 and abs =y′×H0 (52) Map the relative width and height back to the input image I0 scale by performing a width and height inverse normalization operation: In abs =w′×W0 h abs =h′×H0 (53) Convert the center-width-height representation to the top-left-bottom-right corner representation: Where: round(·) indicates rounding the floating-point coordinates to integer pixel coordinates, (x′, y′, w′, h′) represents the bounding box parameters of the predicted box p′, (x abs y abs w abs h abs (x1, y1) represents the bounding box parameters of the predicted box p′ after denormalization, and (x2, y2) represents the integer pixel coordinates of the top left and bottom right corners of the predicted box p′. (54) Based on the integer pixel coordinates of the prediction box p′, the corresponding rectangular region [y1:y2:x1:x2,:] is cropped from the input image I0 to obtain the vehicle cropped image I; [y1:y2:x1:x2,:] means that the slice is from row y1 to row y2 in the height dimension and from column x1 to column x2 in the width dimension.

7. The fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios according to claim 2, characterized in that: In Step 6, the vehicle cropping image I is simultaneously acquired through a CNN backbone network and a feature pyramid network. A I B coarse-grained characteristics and fine-grained features Includes the following steps: (61) Cropping the vehicle image I A I B Common input to the CNN backbone network; vehicle cropped image I A Intermediate layer feature maps are generated through the intermediate layers of the CNN backbone network. The final layer feature map is generated by the final layer of the CNN backbone network. Vehicle cropped image I B Intermediate layer feature maps are generated through the intermediate layers of the CNN backbone network. The final layer feature map is generated by the final layer of the CNN backbone network. (62) The feature pyramid network has a five-layer structure, named P2, P3, P4, P5, and P6 from low to high; the feature maps of the middle layers are... and final layer feature map The input is fed into the feature pyramid network, and the output of layer P5 is used as a coarse-grained feature map. The output of layer P3 is used as a fine-grained feature map. intermediate layer feature map and final layer feature map The input is fed into the feature pyramid network, and the output of layer P5 is used as a coarse-grained feature map. The output of layer P3 is used as a fine-grained feature map.

8. The fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios according to claim 2, characterized in that: In Step 7, using Represents coarse-grained feature map or Will The input is fed into the implicit semantic prototype learning module to obtain the reconstructed feature map F′, including the following steps: (71) Define a learnable semantic prototype vector e k K represents the semantic prototype of the k-th vehicle structural component, where K1 is the number of semantic prototypes. (72) For The eigenvector f at spatial location (i, j) ij Calculate its relationship with the semantic prototype e k The cosine similarity is calculated, and the probability distribution of spatial position (i, j) belonging to each semantic prototype is obtained through the Softmax function: Where: s ij，k Represents the eigenvector f ij With semantic prototype e k The initial similarity between them, a ij,k Represents the eigenvector f ij Belongs to the semantic prototype e k The probability of; (73) The feature vectors at each spatial location in the equation belong to the semantic prototype e. k The probability {a} ij，k } constitutes the semantic assignment vector A k Semantic assignment vector A k It is a two-dimensional response graph that intuitively represents the different spatial locations and semantic prototypes e in the cropped vehicle image I. k The degree of correlation, will {A k Let} be denoted as soft semantic allocation graph A; (74) Using the soft semantic assignment map A as weights, the semantic prototype vector E is weighted and summed to generate a new reconstructed feature map F′ that has undergone semantic refinement and enhancement: Where: ⊙ indicates multiplication by channel; (75) The coarse-grained feature map The soft semantic allocation graph is denoted as A. A A B coarse-grained features The reconstructed feature map is denoted as F. A ′, F B ′.

9. The fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios according to claim 2, characterized in that: Step 8 involves reconstructing the feature map F. A ′, F B '、Soft semantic allocation graph A A A B The input is a fine-grained Transformer matching module, which performs self-attention and cross-attention interactions to obtain the vehicle cropping image I. A and I B A series of matching point pairs between them, including the following steps: (81) Reconstruct the feature map F A ′, F B Expanding the feature map in spatial dimensions transforms it into a sequence of feature vectors, essentially merging the height and width dimensions into a single dimension and adding a sinusoidal positional encoding G, thus reconstructing the feature map F. A ′, F B ′ converted to X A X B : X A =Flatten(F A ′)+G X B =Flatten(F B ′)+G Where Flatten(·) represents the mathematical operation of converting a two-dimensional spatial feature map into a one-dimensional sequence; (82) The feature sequence X after feature preprocessing A X B Recorded as As the initial input to the context-aware module, the context-aware module consists of N C The nth layer is composed of a series of self-attention and cross-attention composite layers, and the input of the nth composite layer is... The output is The output of the last composite layer Let Z be the context-aware feature. A Z B ; (83) Calculate the feature similarity matrix The element s at position (i, j) ij Z represents A (i) and Z B Similarity between (j): Where: τ is the temperature parameter used to adjust the sharpness of the Softmax distribution, and Z... A (i) represents the context-aware feature Z A The vector at position i, Z B (j) represents the context-aware feature Z. B The vector at the j-th position, <Z A (i), Z B (j)> indicates Z A (i) and Z B The inner product of (j); (84) Use double soft maximum calculation to calculate the matching confidence matrix P C The element c at position (i, j) ij Z represents A (i) and Z B Confidence level between (j): c ij =Softmax(S(i,·))j·Softmax(S(·,j))i Where: S(i,·) represents all elements in the i-th row of the feature similarity matrix S, S(·,j) represents all elements in the j-th column of the feature similarity matrix S, and Softmax(S(i,·)) j Z represents A (i) Matched Z B The probability of Z(j) is denoted by Softmax(S(·,j))i. B (j) Matches Z A The probability of (i); (85) Set the confidence threshold θ C If c ij ≥θ C Then (Z) A (i), Z B (j) Add to the set of matching point pairs M C ; (86) For each matching point pair (Z) A (i), Z B (j))∈M C Map its location to fine-grained features Above, with the mapped position as the center, in fine-grained features Each local window of size w×w is cropped, and the features within the two local windows are input into a lightweight Transformer module for local feature interaction and transformation to obtain a refined feature representation. (87) with With the central feature vector, Calculate the central feature vector for the window. With window The correlation of all feature vectors within the context is analyzed and a heatmap is generated. The expectation of the probability distribution represented by the generated heatmap is then calculated to obtain the context-aware feature Z. B The middle corresponds to the context-aware feature Z. A The sub-pixel precise position j′ of position i is given by (i, j′, c). ij′ Add the refined set of matching point pairs M f ; (88) Calculate vehicle cropping image I based on refined matching point pairs A and I B Similarity score between AB : Where: |M f | represents the refined set of matching point pairs M. f The number of matching point pairs contained therein.

10. The fine-grained feature matching vehicle re-identification method for license plate occlusion scenarios according to claim 9, characterized in that: In step (82), the nth composite layer contains two inputs. and two outputs Processing is achieved through a self-attention layer and two cross-attention layers that share the same set of learnable projective weight matrices; the self-attention layer is used to realize feature interactions within the image, with the input being... The output is The cross-attention layer is used to realize feature interaction between images, and the input is... The output is The processing procedure for the nth composite layer is as follows: (821) The internal contextual information of the image is aggregated independently through self-attention layers: Where SelfAttn(·) represents self-attention operation, and LinearAttention(·) represents linear attention operation; express The query point, key point, and value in the text. express The query point, key point, and value in the text. This represents the learnable projective weight matrix in the nth composite layer; Where: φ(·) is the feature mapping function; the self-attention layer uses a linear attention mechanism, which mainly processes the feature interaction within a single image, effectively capturing the spatial context relationship of vehicle parts through linear complexity, and ensuring computational efficiency; (822) will Input two cross-attention layers, the first cross-attention layer with... For query, with Key-value calculation exist Representation in context The second cross-attention layer For query, with Key-value calculation exist Representation in context in: express The query point, key point, and value in the text. express The query points, key points, and values ​​in the text; (823) In calculation Inquiry point and Key points When determining the matching degree, a semantic affinity bias matrix B is introduced. sem The element at position (i, j) By extracting the soft semantic allocation graph A A A B The vector corresponding to the spatial location is obtained by calculation: Where: λ0 is a learnable scale parameter, Represents the soft semantic allocation graph A A The corresponding extracted The vector at the i-th spatial location, Represents the soft semantic allocation graph A B The corresponding extracted The vector at the j-th spatial location; In calculation Inquiry point and Key points When determining the matching degree, the semantic affinity bias matrix B is used. sem transpose vector B semT ; (824) Calculate the attention score in the cross-attention layer: Where: CrossAttn(·) represents the cross-attention operation, D X The dimension of the input feature.

Images