A cross-modal vehicle re-identification method based on group-log-polar coordinate space perception guidance

Abstract

The application discloses a cross-modal vehicle re-identification method based on group logarithmic polar coordinate space perception guidance, and comprises the following steps: acquiring a visible light input image and an infrared input image of a vehicle to be identified; performing group logarithmic polar coordinate space perception feature extraction to obtain a visible light feature map and an infrared feature map; performing intra-modal graph perception optimization to obtain a visible light sub-block feature set and an infrared sub-block feature set; performing fine-grained cross-modal semantic alignment to obtain a cross-modal soft matching matrix; performing weighted reconstruction on the visible light sub-block feature set or the infrared sub-block feature set to obtain a cross-modal aligned sub-block feature set, and then performing vehicle re-identification. The application relates to the technical field of computer vision, and can improve the robustness of feature extraction in a dynamic remote sensing scene, enhance the structural consistency of intra-modal representation, and realize fine alignment of cross-modal local features, so that the retrieval accuracy and the generalization ability in a complex environment are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically a cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception. Background Technology

[0002] Cross-modal vehicle re-identification in remote sensing refers to the technology of performing cross-temporal and spatial identity matching and retrieval of the same vehicle target between heterogeneous modal images (visible light images and infrared images) acquired by UAVs or other high-altitude remote sensing platforms. This technology can overcome the performance bottleneck of a single sensor in extreme environments and effectively serve core scenarios such as all-weather intelligent traffic monitoring, smart city governance, and low-light night patrols.

[0003] Existing cross-modal vehicle re-identification technologies mainly follow three technical routes: the first is a robust feature learning method, which improves the backbone network's adaptability to scale, rotation, viewpoint drift, and background perturbations through network architecture reconstruction or enhancement design; the second is a heterogeneous representation alignment method, which reduces the macroscopic distribution differences between visible light and infrared modes due to different imaging mechanisms by modality-invariant mapping or distribution metric constraints in the pixel domain or global feature space; and the third is a structured and local modeling method, which establishes the correlation between local regions through local attention or graph neural networks to improve local detail discrimination and spatial topological consistency.

[0004] The first type of prior art closest to this invention is a robust feature extraction scheme based on geometric transformation and scale / rotation modeling. Its typical process is as follows: inputting a cross-modal vehicle image, extracting a primary feature map using a traditional convolutional neural network (such as ResNet); then, by stacking multi-scale convolutional kernels, introducing a Spatial Transformer Network (STN), dilated convolution, or traditional log-polar coordinate transformation, establishing adaptive adjustment capabilities for scale and rotation changes at the feature layer; finally, outputting a feature representation with a certain degree of geometric robustness for cross-modal matching. The second type of prior art closest to this invention is a cross-modal feature learning scheme based on global macroscopic distribution alignment. The typical process is as follows: a backbone network with two branches or partially shared parameters is used to extract features from visible light images and infrared images respectively; during the training phase, global modality alignment loss (such as maximum mean difference MMD, Wasserstein distance), class center constraint or cross-modal contrastive learning loss is used to forcibly map the two heterogeneous features to the same potential shared feature space; during the retrieval phase, the similarity is directly ranked by the distance of the global feature vectors generated by global average pooling (GAP).

[0005] However, under the dynamic imaging conditions of UAV remote sensing, the above-mentioned existing technologies have the following technical shortcomings: Defect 1: Poor robustness to feature extraction in dynamic and complex scenes, easily leading to recognition failure due to geometric distortion. Drones frequently change altitude and viewpoint during flight, causing drastic scaling and multi-angle rotation distortion of the target vehicle in the image. Existing mainstream networks use regular rectangular convolutional kernels, lacking inherent equivariance to geometric transformations. Using multi-scale stacking or spatial transformation networks to address this would cause a surge in parameters. This mismatch between the underlying spatial sampling mechanism and dynamic geometric deformation directly leads to severe distortion of the extracted features, losing the fundamental stability of cross-modal matching. Ultimately, existing robustness enhancement strategies often only statically process the single dimension of "scale" or "rotation," failing to unify and efficiently model both at the underlying spatial sampling mechanism level.

[0006] Defect 2: The internal representation of a single modality is highly susceptible to environmental noise interference, resulting in extremely low feature consistency. Visible light images are easily affected by sudden changes in illumination and shadow occlusion, while infrared images are easily affected by thermal radiation blurring and texture loss. Single-modal images themselves contain high-frequency noise. However, most existing technologies extract local features in isolation, severing the spatial topological relationships between various vehicle components (such as the roof, front, and wheels). They lack mechanisms for self-repair and noise reduction using contextual structural information, leading to noisy or information-deficient local features being directly passed to subsequent alignment networks, causing a chain reaction of feature distortion.

[0007] Defect 3: Severe semantic misalignment exists across modal feature spaces, resulting in low matching accuracy and retrieval precision. Due to the fundamentally different imaging mechanisms of visible light and infrared light, the local structures of the same vehicle in both modalities often cannot be aligned one-to-one at the feature level. Existing technologies generally use global average pooling for coarse-grained global feature alignment, which essentially forces spatially dissimilar regions (such as the front of the vehicle in visible light and background clutter in infrared light) to be closer together. This logic of "forced alignment without pairing" smooths out the fine-grained structural differences of the vehicle, creating a "semantic gap" that is difficult to eliminate, leading to a large number of mismatches and causing retrieval accuracy to encounter a bottleneck.

[0008] In summary, existing technologies lack a layered collaborative processing mechanism: in the feature extraction stage, it is difficult to overcome the rotation and scale distortion caused by the fluctuation of the UAV's perspective and altitude; in the single-modal processing stage, it fails to utilize the spatial topological association between local components to eliminate noise interference; in the cross-modal alignment stage, it blindly performs global coarse-grained narrowing without establishing a component-level mapping mechanism of "first local precise pairing, then fine feature alignment", which ultimately severely restricts the retrieval accuracy of cross-modal vehicle re-identification and its generalization ability in complex environments. Summary of the Invention

[0009] To address the shortcomings of the existing technologies, this invention provides a cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception. This method improves the robustness of feature extraction in dynamic remote sensing scenarios, enhances the structural consistency of intra-modal representations, and achieves fine-grained local alignment across modalities without significantly increasing the model parameter scale. This significantly improves retrieval accuracy and generalization ability in complex environments. Specifically, the objectives of this invention include the following three aspects, each corresponding to one of the deficiencies described in the background technology: To address the first defect, this invention abandons traditional rectangular sampling and introduces a group logarithmic polar coordinate space perception mechanism: the logarithmic polar coordinate transforms the radial "scale scaling" of the image into a "translation operation" in logarithmic space, while simultaneously achieving equivariant extraction of "rotational changes" by modeling the angle set as a cyclic group; this mechanism follows the visual law of "near is fine and far is coarse", allowing the receptive field to expand exponentially with the hierarchy without increasing the model parameters, so that no matter how the UAV changes its altitude and viewpoint, the network can adaptively output robust features that have both scale invariance and multi-angle rotation invariance.

[0010] To address the second defect, this invention introduces a modal intra-graph perception feature optimization mechanism: the single-modal feature map is divided into horizontal and vertical sub-blocks and a graph adjacency matrix is ​​constructed using cosine similarity. Information is transmitted between nodes through graph convolution. When a local sub-block (such as the car body) is damaged due to reflection or shadow, semantic information can be aggregated from highly correlated adjacent nodes (such as the roof and windows) for reconstruction compensation. This fundamentally filters local noise and ensures the purity and structural integrity of single-modal features before cross-modal alignment.

[0011] To address the third defect, this invention proposes a fine-grained cross-modal semantic alignment mechanism based on optimal transmission, following the logic of "precise pairing first, then feature alignment": by solving the optimal transmission soft matching matrix between visible light and infrared local sub-blocks through Sinkhorn iteration, the physical components that truly correspond between heterogeneous modes are intelligently found and reconstructed accordingly with weights, thereby avoiding the forced close-up of unrelated regions and preserving the fine-grained discrimination information unique to each mode to the greatest extent.

[0012] To achieve the above objectives, this invention provides a cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception, comprising the following steps: Step 1: Acquire visible light and infrared images of the vehicle to be identified in the same scene, and preprocess the visible light and infrared images to obtain visible light and infrared input images with uniform pixel resolution. Step 2: Perform group logarithmic polar coordinate spatial perception feature extraction on the visible light input image and the infrared input image respectively to obtain visible light feature map and infrared feature map with scale invariance and rotation invariance; Step 3: Perform modal in-map sensing optimization on the visible light feature map and the infrared feature map respectively to obtain the visible light sub-block feature set and the infrared sub-block feature set; Step 4: Perform fine-grained cross-modal semantic alignment on the visible light sub-block feature set and the infrared sub-block feature set to obtain a cross-modal soft matching matrix; Step 5: Based on the cross-modal soft matching matrix, the visible light sub-block feature set or the infrared sub-block feature set is weighted and reconstructed to obtain the cross-modal aligned sub-block feature set, and then the vehicle is re-identified based on the cross-modal aligned sub-block feature set.

[0013] In one embodiment, in step 2, the group logarithmic polar coordinate space-aware feature extraction has the following characteristics: The visible light input image and the infrared input image are respectively input into a deep convolutional neural network with a group logarithmic polar coordinate space perception mechanism; In the deep convolutional neural network, the anchor pixels in the visible light input image and the infrared input image are sampled by multi-angle rotation in non-uniform space, and the multi-angle feature groups obtained by multi-angle rotation sampling are adaptively aggregated based on an attention mechanism to obtain the visible light feature map and the infrared feature map.

[0014] In one embodiment, the multi-angle rotation sampling of the non-uniform space specifically includes: Construct a structure with a radius centered on the anchor pixel. The local neighborhood is defined, and the local neighborhood is divided into... Radial levels and Each angle and level determines A log-polar coordinate subregion; Perform in each of the said logarithmic polar coordinate sub-regions Rotation sampling in each angular direction.

[0015] In one embodiment, the attention-based adaptive aggregation specifically includes: Obtain the same logarithmic polar coordinate sub-region Rotation sampling Aspect features Then, all angle sub-features are stacked according to the angle dimension to obtain stacked features. ; Stacked features Input a lightweight angle attention branch and obtain an unnormalized response. :

[0016] in, , For learnable parameters, It is a non-linear activation function; The normalized weights are obtained using the Softmax function. :

[0017] Feature aggregation based on normalized weights yields logarithmic polar coordinate sub-regions. Corresponding sampling features .

[0018] In one embodiment, step 3, the modal intragraph sensing optimization process is as follows: The unimodal feature map is uniformly divided along the height dimension. Each horizontal feature sub-block uniformly divides the single-modal feature map along the width dimension into several horizontal feature sub-blocks. A vertical feature sub-block, wherein the single-modal feature map is the visible light feature map or the infrared feature map; The horizontal and vertical feature sub-blocks are used as nodes in the topology graph, i.e., the total number of nodes. ; Calculate the cosine similarity between the features of corresponding sub-blocks of any two nodes in the topology graph, and use the cosine similarity as a weight to construct a dimension. Single-mode inner adjacency matrix ; For each node in the topology graph, based on the single-modal intra-adjacency matrix Relation inference is performed by applying multi-layer graph convolution operations, i.e., each node is based on the single-modal intra-adjacency matrix. The weights in the graph adaptively aggregate the semantic features of other nodes to obtain the graph convolutional features corresponding to each node; By performing residual connection between the original sub-block features corresponding to each node and the graph convolution features, the graph-optimized sub-block features of each node are obtained. By combining the graph-optimized sub-block features of all nodes, the visible light sub-block feature set or the infrared sub-block feature set is obtained.

[0019] In one embodiment, step 4, the fine-grained cross-modal semantic alignment process is as follows: Calculate the distance between any two sub-block features between the visible light sub-block feature set and the infrared sub-block feature set, and construct a fully connected cross-modal alignment cost matrix; The cross-modal sub-block matching problem is modeled as an optimal transmission problem. Based on the Sinkhorn algorithm with entropy regularization, the cross-modal alignment cost matrix is ​​subjected to differentiable row and column normalization iteration. After convergence after a preset number of iterations, the cross-modal soft matching matrix is ​​output. The elements in the cross-modal soft matching matrix represent the association probability that the visible photo sub-block and the infrared sub-block belong to the same physical component.

[0020] In one embodiment, step 5, the weighted reconstruction process is as follows: For each visible photon sub-block feature in the visible photon sub-block feature set, a weighted aggregation is performed based on the infrared sub-block feature set according to the association probability corresponding to the cross-modal soft matching matrix to obtain the infrared reconstructed feature corresponding to the visible photon sub-block feature. Then, the infrared reconstructed feature is residually fused or spliced ​​with the corresponding visible photon sub-block feature to obtain the cross-modal aligned sub-block feature set; or For each infrared sub-block feature in the infrared sub-block feature set, the visible light sub-block feature set is weighted and aggregated according to the association probability corresponding to the cross-modal soft matching matrix to obtain the visible light reconstructed feature corresponding to the infrared sub-block feature. Then, the visible light reconstructed feature and the corresponding infrared sub-block feature are residually fused or spliced ​​to obtain the cross-modal aligned sub-block feature set.

[0021] In one embodiment, step 5, the vehicle re-identification process is as follows: Global aggregation is performed on the feature set of sub-blocks after cross-modal alignment to obtain global features. ; global features The data is fed into a BNNeck or normalization layer to obtain the target embedding features; The candidate embedding features of each candidate image in the candidate image library are obtained. Based on the similarity or distance between the target embedding features and the candidate embedding features, the candidate images are sorted from high to low similarity or from small to large distance. The candidate image ranked first is the same target.

[0022] The target embedding features are mapped to each candidate image in the candidate image library.

[0023] In one embodiment, the cross-modal vehicle re-identification method guided by group log-polar coordinate space perception further includes end-to-end training and optimization of the group log-polar coordinate space perception feature extraction, the modal intra-graph perception optimization, and the fine-grained cross-modal semantic alignment using a multi-objective joint loss function.

[0024] In one embodiment, the multi-objective joint loss function is a weighted combination of global identity classification loss, global infrared-visible cross-modal triplet loss, fine-grained cross-modal optimal transmission alignment loss, intramodal structure preservation regularization loss, negative sample constraint loss, and entropy regularization constraint term.

[0025] Compared with the prior art, the present invention has the following beneficial technical effects: 1. This invention adopts a group logarithmic polar coordinate space perception mechanism to transform radial scale scaling into logarithmic spatial translation and angular rotation into cyclic group equivariance. Therefore, robust underlying features with both scale invariance and multi-angle rotation invariance can be obtained without significantly increasing the number of parameters. This overcomes the problem of difficulty in balancing "parameter efficiency-geometric robustness" in existing methods and solves the defect of recognition failure caused by geometric distortion in dynamic remote sensing scenarios. 2. This invention constructs a local relational topology graph within a modality by dividing it into horizontal and vertical heterogeneous parts and uses graph convolution for relational reasoning and residual repair. This enables damaged local sub-blocks to complete self-repair by using the contextual information of highly correlated adjacent sub-blocks, effectively filtering out high-frequency noise and information loss within a single modality, significantly enhancing the structural consistency and noise resistance of the single-modality representation, and providing a clean and complete feature foundation for subsequent cross-modal alignment. 3. Based on optimal transmission, this invention achieves fine-grained alignment at the local sub-block level by "first precise pairing, then feature alignment". It constrains local regions that truly have the same physical semantics. Therefore, it eliminates semantic overlap and excessive smoothing caused by coarse-grained global pooling from the underlying logic, bridges the cross-modal semantic gap, and preserves fine-grained discriminative information to the greatest extent. This significantly improves the ranking stability, hit accuracy, and generalization ability in complex environments of cross-modal retrieval. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0027] Figure 1 This is a flowchart of a cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception in an embodiment of the present invention.

[0028] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0030] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0031] For ease of understanding, the key technical terms involved in this invention are explained as follows: Remote sensing cross-modal vehicle re-identification refers to the technology of matching and retrieving the identity of the same vehicle between visible light images and infrared images acquired by a remote sensing platform; Group-log polar coordinate space perception refers to a feature learning method that maps local neighborhoods to log polar coordinate space according to distance and direction and combines group action to model direction. Group action refers to using algebraic structures such as cyclic groups to describe the parameter sharing and index transformation relationships between different directions, thereby improving the ability to model rotations. Intramodal graph-aware optimization refers to the optimization process of constructing a graph of local sub-blocks within a single modality and enhancing the structural relationships between sub-blocks through graph propagation. Fine-grained cross-modal semantic alignment refers to an alignment method that establishes correspondences for samples of different modalities and implements consistency constraints at the local sub-block level; Optimal transfer refers to the method of establishing a matching relationship between two distributions with the minimum total cost while satisfying edge constraints; Sinkhorn iteration refers to the optimal transfer algorithm with entropy regularization, which can efficiently obtain the soft matching matrix; A soft matching matrix refers to a matching result that allows a local region to establish a correspondence with multiple candidate regions according to different weights. Triplet loss refers to a loss function that improves retrieval and discrimination capabilities by constraining the distance relationships between anchor samples, similar samples, and dissimilar samples. mAP, or mean precision, is used to measure the overall ranking quality of a retrieval task. Rank-1 refers to the proportion of the first search result in the query sample that is a correct match.

[0032] This embodiment discloses a cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception. It takes into account the robustness of feature extraction, the completeness of intramodal representation, and the consistency of intermodal representation. It aims to solve the problems of decreased recognition accuracy, insufficient robustness, and limited generalization ability caused by the combined effects of dynamic imaging conditions and heterogeneous modal differences, and achieve high-precision vehicle retrieval under complex remote sensing conditions with varying perspectives and scales.

[0033] refer to Figure 1 The cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception in this embodiment includes the following steps: Step 1: Acquire visible light and infrared images of the vehicle to be identified in the same scene, and preprocess the visible light and infrared images to obtain visible light and infrared input images with uniform pixel resolution. Step 2: Perform group logarithmic polar coordinate spatial perception feature extraction on the visible light input image and the infrared input image respectively to obtain visible light feature map and infrared feature map with scale invariance and rotation invariance; Step 3: Perform modal in-map sensing optimization on the visible light feature map and the infrared feature map respectively to obtain the visible light sub-block feature set and the infrared sub-block feature set; Step 4: Perform fine-grained cross-modal semantic alignment on the visible light sub-block feature set and the infrared sub-block feature set to obtain the cross-modal soft matching matrix; Step 5: Based on the cross-modal soft matching matrix, the visible light sub-block feature set or the infrared sub-block feature set is weighted and reconstructed to obtain the cross-modal aligned sub-block feature set, and then the vehicle is re-identified based on the cross-modal aligned sub-block feature set.

[0034] In the specific implementation of step 1, visible light images of the same vehicle in a dynamic remote sensing scene from an unmanned aerial vehicle are acquired. and infrared images and visible light images Infrared images After preprocessing such as boundary padding, random cropping, horizontal flipping, and / or scaling, a visible light input image and an infrared input image with a uniform pixel resolution (e.g., 256×256) are obtained.

[0035] In the specific implementation of step 2, a robust feature learning process is performed at the bottom layer using a deep convolutional neural network (CNN) backbone architecture with a group logarithmic polar coordinate spatial awareness mechanism. In the traditional regular rectangular convolutional layer or feature sampling layer of the backbone network, this embodiment no longer uses the traditional square sampling sliding window for any anchor pixel. Instead, a logarithmic polar coordinate system is introduced for spatial awareness reconstruction, thereby performing multi-angle rotation sampling of anchor pixels in non-uniform space in the visible light input image and infrared input image, and performing adaptive aggregation of the multi-angle feature groups obtained by multi-angle rotation sampling based on an attention mechanism to obtain visible light feature maps and infrared feature maps.

[0036] The spatial perception reconstruction using a logarithmic polar coordinate system specifically involves dividing the local neighborhood around the anchor pixel into radial and angular levels, with the radius of the radial level... The interval is divided using a geometric series (exponential series), that is... , Based on the radius, For expansion bases greater than 1, For hierarchical indexes; for angular hierarchies, it's in... Divide the area into equal-angled uniform sections, and divide it into... Each region is divided into angular sectors. This division method allows the sampling receptive field to expand exponentially with the increase of radial levels, forming a biomimetic visual topology of "dense at the center and sparse at the edges (thinner and coarser at the distance)". Within each logarithmic polar coordinate sampling region, along the set... Multi-angle rotation sampling is performed in equal-angle directions to generate a set of samples for each local region. This embodiment uses a multi-angle feature set of multiple components. To uniformly handle the severe rotational distortion caused by UAV attitude drift within a limited parameter scale, this embodiment will... The set of rotation operations at each angle is modeled as an algebraic cyclic group. By utilizing the group-equivariant convolution property in group theory, when the input image undergoes rotational transformations, the network's output features only undergo corresponding permutation cycles in the group dimension, without causing deformation of the feature semantic content.

[0037] In the specific implementation process, multi-angle rotation sampling in non-uniform space includes: For any anchor pixel, construct a structure with radius as the center of the anchor pixel. The local neighborhood is divided into... Radial levels and There are several angle levels, among which the first... The radial boundary can be represented as , , The distance from the pixel to the anchor pixel satisfy At that time, the pixel was assigned to the first... A radial level; and simultaneously, based on its angular position, it is assigned to the corresponding first level. Each angle sector; Therefore, the local neighborhood of each anchor pixel is divided into Each logarithmic polar coordinate sub-region is then executed within each logarithmic polar coordinate sub-region. Rotation sampling in each angular direction preserves both radial scale hierarchy information and angular direction information simultaneously.

[0038] Based on the multi-angle feature group obtained by multi-angle rotation sampling, this embodiment designs a lightweight adaptive angle attention module. This module dynamically generates a set of weight coefficients by capturing the discriminative properties between different angle components, and uses these weight coefficients to perform weighted summation on the multi-angle feature group to achieve adaptive feature fusion. (Using logarithmic polar coordinates as a sub-region) For example, the specific implementation process is as follows: First, obtain the same logarithmic polar coordinate sub-region. Rotation sampling Aspect features Then, all angle sub-features are stacked according to the angle dimension to obtain stacked features. ; Then, stack the features The input to the lightweight angle attention branch yields an unnormalized response. This lightweight angle attention branch can consist of global average pooling, a fully connected layer, and a nonlinear activation function, used to estimate the discrimination contribution of different rotation angles to the local region of the current vehicle. The unnormalized response... Specifically, it is expressed as follows:

[0039] in, , For learnable parameters, It is a non-linear activation function; Subsequently, the normalized weights are obtained using the Softmax function. Specifically, it is expressed as:

[0040] in, ,and ; Finally, feature aggregation is performed based on normalized weights to obtain log-polar coordinate sub-regions. Corresponding sampling features .

[0041] This embodiment extracts features using group-logarithmic polar coordinate space perception. The spatial mapping of logarithmic polar coordinates successfully transforms the radial "scaling" of the image into a "translation operation" in logarithmic space. Combined with angular aggregation of cyclic groups, the final output visible light feature map is obtained. and infrared feature map It possesses both high scale invariance and multi-angle rotation invariance.

[0042] Because visible light images contain large areas of shadow and time-varying illumination clutter, while infrared images suffer from severe thermal blurring and loss of texture details, single-modal features are often spatially discontinuous or contain local noise. To enhance the completeness and structural stability of the representation within a single modality before cross-modal alignment, this embodiment utilizes graph convolutional networks to infer and repair the spatial topological relationships within a single modality, namely, the intra-modal graph perception optimization in step 3. This mainly includes three parts: heterogeneous spatial partitioning of horizontal and vertical feature maps, construction of intra-modal local correlation topological graphs, graph convolutional relationship inference, and residual connection for feature repair.

[0043] During the process of partitioning the heterogeneous space of the horizontal and vertical feature maps, the single-modal feature map (i.e., the visible light feature map) is... or infrared feature map Two independent partitioning strategies were executed respectively: Horizontal partitioning: Dividing the unimodal feature map uniformly along the height dimension. Each horizontal feature sub-block has the physical meaning of powerfully depicting the hierarchical semantic structure of the vehicle target in the vertical direction of space, such as the roof, windows, body, and wheels. Vertical partitioning: Dividing the single-modal feature map uniformly along its width dimension into... The vertical feature sub-blocks are physically significant because they emphasize the spatial symmetry and asymmetrical structure of the vehicle in the horizontal direction, such as the discriminative differences between the front and rear of the vehicle.

[0044] The specific implementation process for constructing the intramodal local association topology graph is as follows: A horizontal feature sub-block and Each vertical feature sub-block serves as a node in the topological graph, representing the total number of nodes. To characterize the inherent spatial and semantic relationships between these vehicle components, this embodiment calculates the cosine similarity between the features of corresponding sub-blocks of any two nodes in the topological graph, and dynamically constructs the dimension using the cosine similarity as the weight. Single-mode inner adjacency matrix Single-modal internal adjacency matrix Each element value in the table represents the semantic correlation between different parts of the vehicle.

[0045] In this embodiment, the number of horizontal divisions Number of vertical divisions These are pre-defined, fixed hyperparameters that remain unchanged during system training and inference. They are used to control the granularity of sub-block partitioning of the feature map in both the horizontal and vertical directions. For example, they can be set to... The system then obtains four horizontal sub-block features and four vertical sub-block features, which are then unified as nodes in the graph structure. Therefore, the total number of nodes is 8, corresponding to a single-modal internal adjacency matrix. Size is .

[0046] The implementation process of graph convolutional relation inference and feature repair residual connection is as follows: For each node (i.e., a local component of a vehicle) in the topological graph, based on the single-modal internal adjacency matrix... Applying multi-layer graph convolution operations for relation reasoning, i.e., each node is based on a single-modal intra-adjacency matrix. The weights in the graph network adaptively aggregate the semantic features of other nodes to obtain the graph convolutional features corresponding to each node. For example, when the "vehicle body" sub-block in a visible light image is distorted due to strong light reflection, it can introduce complementary contextual spatial structure information from the spatially intact "roof" and "window" sub-blocks through the graph convolutional network, thereby adaptively repairing the damaged features. After the graph convolution output, the original sub-block features corresponding to each node are residually connected with the graph convolutional features to obtain the graph-optimized sub-block features of each node. This prevents the oversmoothing effect of the deep graph network. By combining the graph-optimized sub-block features of all nodes, the visible light sub-block feature set or the infrared sub-block feature set is obtained.

[0047] After completing robust feature extraction and intramodal feature optimization, visible light and infrared features have strong individual stability. However, a "semantic gap" still exists between them due to the heterogeneity of physical imaging. To address this, this embodiment performs fine-grained cross-modal semantic alignment on the visible light sub-block feature set and the infrared sub-block feature set in step 4. The specific implementation process is as follows: Let the graph-optimized visible photon block feature set be... And the infrared sub-block feature set after graph optimization is Calculate the distance (e.g., Euclidean distance or cosine distance) between any two features in the visible light sub-block feature set and the infrared sub-block feature set, and construct a fully connected cross-modal alignment cost matrix. ,matrix elements in Reflects the first The characteristics of the first visible photon block are similar to those of the second... Mismatch degree of infrared sub-block features in latent space; To overcome the problems of spatial misalignment, visibility / hiding, or local deformation of vehicle components across different modes from the perspective of UAV remote sensing, this embodiment models the cross-modal sub-block matching problem as an optimal transport problem, and uses the Sinkhorn algorithm with entropy regularization to solve the cross-modal alignment cost matrix. Perform differentiable row and column normalization iterations; After convergence through a preset number of iterations, the cross-modal soft matching matrix is ​​output. ,matrix Each element in This forms a probability distribution, representing the first... The characteristics of the first visible photon block are similar to those of the second... The soft matching mechanism, which determines the probability of infrared sub-block features belonging to the same physical component, changes the rigid approach of forcibly bringing the same spatial position closer together. It can intelligently achieve precise component-level pairing between the "visible light front" and the "infrared front that is misaligned due to the rotation of the viewing angle".

[0048] After obtaining the cross-modal soft matching matrix Then, the visible light sub-block feature set or the infrared sub-block feature set can be reconstructed by weighting to obtain the cross-modal aligned sub-block feature set. This fine-grained strategy of "precise pairing first, then feature alignment" ensures that the subsequent loss function, when bringing heterogeneous modal features closer together, constrains local regions that truly have the same physical semantics, thus eliminating the semantic overlap and over-smoothing problems caused by coarse-grained global pooling from the underlying logic.

[0049] In this embodiment, the weighted reconstruction process is as follows: For each visible photon feature in the visible photon feature set, weighted aggregation is performed based on the infrared feature set according to the association probability corresponding to the cross-modal soft matching matrix to obtain the infrared reconstructed feature of the corresponding visible photon feature. Then, the infrared reconstructed feature is residually fused or spliced ​​with the corresponding visible photon feature to obtain the cross-modal aligned visible photon feature set. For example, for the first... Visible photon block features Its corresponding infrared reconstruction features can be expressed as ,in For the first The infrared sub-block features are then used to reconstruct the infrared features and the corresponding visible light sub-block features. Residual fusion or stitching mapping is then performed between these infrared features and the corresponding visible light sub-block features to obtain the cross-modal aligned first infrared sub-block features. Visible photon block features or ,in, Indicates feature splicing, This represents a linear mapping layer. The reconstruction process does not force the selection of a specific infrared sub-block feature, but rather aggregates the reconstructed representation from all infrared sub-blocks based on soft matching probabilities to find the one that is closest in semantics to the current visible light sub-block feature, thereby completing cross-modal semantic compensation and alignment at the local sub-block level.

[0050] In practical applications, the weighted reconstruction process can also be as follows: For each infrared sub-block feature in the infrared sub-block feature set, weighted aggregation is performed based on the association probability corresponding to the cross-modal soft matching matrix on the basis of the visible light sub-block feature set to obtain the visible light reconstructed feature of the corresponding infrared sub-block feature. Then, the visible light reconstructed feature is residually fused or spliced ​​with the corresponding infrared sub-block feature to obtain the cross-modal aligned infrared sub-block feature set. For example, for the _ Infrared sub-block features The corresponding visible light reconstruction features can be expressed as: Then, the visible light reconstructed features and the corresponding infrared sub-block features are residually fused or stitched together to obtain the cross-modal aligned first... Infrared sub-block features or .

[0051] After obtaining the cross-modal aligned sub-block feature set, global aggregation is performed on the cross-modal aligned sub-block feature set using either Global Average Pooling (GAP) or sub-block feature concatenation to obtain global features. Then, global features The data is fed into a BNNeck or normalization layer to obtain the target embedding features. Then, obtain the candidate embedding features of each candidate image in the candidate image library. And based on target embedding features With candidate embedding features Cosine similarity between or Euclidean distance Finally, based on cosine similarity... From high to low or according to Euclidean distance The system sorts the candidate images in the candidate image library from smallest to largest and outputs Rank-1, Rank-5, Rank-10, etc. The vehicle target corresponding to the candidate image with the highest ranking is the same target identified by the system.

[0052] In its specific implementation, the cross-modal vehicle re-identification method in this embodiment further includes end-to-end training and optimization using a multi-objective joint loss function to train swarm log-polar coordinate space perceptual feature extraction, intramodal graph perception optimization, and fine-grained cross-modal semantic alignment. The multi-objective joint loss function is a weighted combination of global identity classification loss, global infrared-visible cross-modal triplet loss, fine-grained cross-modal optimal transmission alignment loss, intramodal structure preservation regularization loss, negative sample constraint loss, and entropy regularization constraint term.

[0053] Global identity classification loss ( This is used to constrain the model to learn a global vehicle representation with identity discrimination, that is, to constrain the network to learn a highly discriminative representation oriented towards the global identity of vehicles, ensuring the basic linear separability between different vehicles. For modalities... The classification probability is:

[0054] in, Represents the classification probability. , They represent the first The weights and biases of the classifier Visible light samples or infrared samples, The number of classifiers; If the ordinary cross-entropy loss is used, the global identity classification loss is:

[0055] in, For the number of categories, For tags.

[0056] Global infrared-visible cross-modal triplet loss ( This is used to bring the cross-modal feature distance of the same vehicle closer together in the global metric space, while simultaneously widening the cross-modal feature distance of different vehicles. The cross-modal feature distance is defined as:

[0057] in, For visible light samples, Infrared sample; For visible light samples Its cross-modal positive samples and negative samples They are respectively:

[0058]

[0059] The triplet loss from visible light to infrared direction The triplet loss from infrared to visible light is:

[0060]

[0061] in, The triplet interval hyperparameter, ; Ultimately, the global infrared-visible cross-modal triplet loss is .

[0062] Fine-grained cross-modal optimal transmission alignment loss ( The calculation process for ) is as follows: For local sub-block features in visible light and infrared, a cross-modal cost matrix is ​​first constructed. ,matrix elements for:

[0063] in, No. The characteristics of the first visible photon block are similar to those of the second... Cosine distance between features of infrared sub-blocks; Let the transport polyhedron be:

[0064] in, and These represent the edge distributions of the visible photon block and the infrared photon block, respectively. The optimal transmission soft matching matrix is ​​solved by Sinkhorn iteration. :

[0065] in, Here is the entropy regularization coefficient. Represents the inner product of matrices; Therefore, fine-grained cross-modal optimal transmission alignment loss By minimizing this alignment loss, the cooperative driving network automatically reduces the physical heterogeneity gap across modes at the local component level.

[0066] Intramodal structure preservation regularization loss ( This is used to constrain the consistency of sub-block relationships within the visible light and infrared modes, preventing cross-modal mapping from disrupting the structural relationships between local components within the vehicle. The calculation process is as follows: Construct the modal intra-adjacency matrix based on the cosine similarity between local sub-blocks, for each mode. Its adjacency matrix elements are:

[0067] in, For temperature coefficient, Indicates cosine similarity; The intramodal structure preservation regularization loss can be defined as follows: .

[0068] Negative sample constraint loss ( This is used to constrain the cross-modal local feature distance between different vehicle identities, preventing the visible light and infrared features of different identities from being incorrectly magnified. Let... Let represent the set of cross-modal negative sample pairs composed of different vehicle identities. For any negative sample pair ,in and To represent different vehicle identities, their visible light sub-block feature sets and infrared sub-block feature sets are as follows: , Negative sample constraint loss ( The calculation process for ) is as follows: First calculate the... The characteristics of the first visible photon block are similar to those of the second... Cross-modal distance between infrared sub-block features ; Negative sample pairs are obtained through optimal transmission matching. Soft matching matrix:

[0069] Then negative sample pairs The fine-grained cross-modal optimal transmission distance is defined as:

[0070] Therefore, the negative sample constraint loss is defined as:

[0071] in, This represents the negative sample safety interval threshold, which is the fine-grained cross-modal distance between different vehicle identities. When the distance is less than 0, it indicates that the two have been sufficiently separated, and the sample constraint loss is 0; when the distance is less than 0, it indicates that the two have been sufficiently separated. This indicates that the cross-modal local features of different vehicle identities are incorrectly magnified, and this loss term needs to be used to penalize them.

[0072] Entropy regularization constraint term ( This is used to constrain the distribution of the optimal transmission soft-matching matrix, ensuring the matching process remains differentiable and stable. If used as the standard entropy regularization term in the Sinkhorn optimal transmission solution, it can be written as:

[0073] in, It is a very small constant; If it is desired to explicitly constrain the matching matrix to avoid excessive uniformity and enhance the clarity of local matching, a low-entropy constraint form can also be used, i.e.:

[0074] In practical applications, fine-grained cross-modal optimal transmission alignment loss Intramodal structure preservation regularization loss Negative sample constraint loss Entropy regularization constraint term Constituting the total loss of fine-grained alignment The multi-objective joint loss function in this embodiment .in, , , These weights are used to control the contribution of negative sample constraint loss, intramodal structure preservation loss, and entropy regularization loss to the total loss, respectively. Before training, these weights can be set based on validation set performance or experience; once training begins, each weight remains constant throughout the training process. By fixing the weights, the uncertainty of the training process can be reduced, ensuring the stable convergence of the joint optimization objective.

[0075] It is worth noting that, although this embodiment Figure 1 The steps are shown sequentially as indicated by the arrows, but they are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are performed; they can be executed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0076] Additionally, it should be noted that in practical applications, the cross-modal vehicle re-identification method in this embodiment can employ convolutional neural networks, visual Transformer networks, or a hybrid convolutional-Transformer architecture as the basic feature extractor for modality extraction and modality sharing. Regarding robust modeling in group log-polar coordinates, the number of angle partitions, radial levels, number of multi-angle samples, and aggregation function form can be adjusted, and the corresponding dynamic weight generation can be implemented by attention modules, gating modules, or hypernetworks. For intra-modal graph perception optimization, graph propagation can be implemented by graph convolution, graph attention networks, message passing networks, or relational Transformers, and the node construction method can be replaced with irregular region partitioning or keypoint region partitioning. For cross-modal matching, in addition to optimal transmission, bidirectional consistency matching, graph matching, Hungarian matching, learning-based matching networks, or probabilistic matching methods can be used to complete sub-block correspondence. Regarding joint loss, in addition to classification loss and triplet loss, contrast loss, center loss, mutual information constraints, or distillation constraints can be introduced to achieve similar cross-modal discrimination enhancement effects.

[0077] Furthermore, although this embodiment mainly describes visible light-infrared remote sensing vehicle re-identification, it can also be extended to UAV-satellite cross-view matching, visible light-near infrared target retrieval, and other target matching tasks that require fine-grained alignment under heterogeneous imaging conditions in specific applications.

[0078] The above description is only a preferred embodiment of the present invention and does not limit the scope of protection of the present invention. All equivalent structural transformations made under the inventive concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the scope of protection of the present invention.

Claims

1. A cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception, characterized in that, Includes the following steps: Step 1: Acquire visible light and infrared images of the vehicle to be identified in the same scene, and preprocess the visible light and infrared images to obtain visible light and infrared input images with uniform pixel resolution. Step 2: Perform group logarithmic polar coordinate spatial perception feature extraction on the visible light input image and the infrared input image respectively to obtain visible light feature map and infrared feature map with scale invariance and rotation invariance; Step 3: Perform modal in-map sensing optimization on the visible light feature map and the infrared feature map respectively to obtain the visible light sub-block feature set and the infrared sub-block feature set; Step 4: Perform fine-grained cross-modal semantic alignment on the visible light sub-block feature set and the infrared sub-block feature set to obtain a cross-modal soft matching matrix; Step 5: Based on the cross-modal soft matching matrix, the visible light sub-block feature set or the infrared sub-block feature set is weighted and reconstructed to obtain the cross-modal aligned sub-block feature set, and then the vehicle is re-identified based on the cross-modal aligned sub-block feature set.

2. The cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception according to claim 1, characterized in that, In step 2, the group logarithmic polar coordinate space-aware feature extraction has the following characteristics: The visible light input image and the infrared input image are respectively input into a deep convolutional neural network with a group logarithmic polar coordinate space perception mechanism; In the deep convolutional neural network, the anchor pixels in the visible light input image and the infrared input image are sampled by multi-angle rotation in non-uniform space, and the multi-angle feature groups obtained by multi-angle rotation sampling are adaptively aggregated based on an attention mechanism to obtain the visible light feature map and the infrared feature map.

3. The cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception according to claim 2, characterized in that, The multi-angle rotation sampling of the non-uniform space specifically includes: Construct a structure with a radius centered on the anchor pixel. The local neighborhood is defined, and the local neighborhood is divided into... Radial levels and Each angle and level determines A log-polar coordinate subregion; Perform in each of the said logarithmic polar coordinate sub-regions Rotation sampling in each angular direction.

4. The cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception according to claim 3, characterized in that, The attention-based adaptive aggregation specifically includes: Obtain the same logarithmic polar coordinate sub-region Rotation sampling Aspect features Then, all angle sub-features are stacked according to the angle dimension to obtain stacked features. ; Stacked features Input a lightweight angle attention branch and obtain an unnormalized response. : in, , For learnable parameters, It is a non-linear activation function; The normalized weights are obtained using the Softmax function. : Feature aggregation based on normalized weights yields logarithmic polar coordinate sub-regions. Corresponding sampling features .

5. The cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception according to any one of claims 1 to 4, characterized in that, In step 3, the modal intra-graph sensing optimization process is as follows: The unimodal feature map is uniformly divided along the height dimension. Divide the single-modal feature map into horizontal feature sub-blocks, uniformly segmenting it along the width dimension. A vertical feature sub-block, wherein the single-modal feature map is the visible light feature map or the infrared feature map; The horizontal and vertical feature sub-blocks are used as nodes in the topology graph, i.e., the total number of nodes. ; Calculate the cosine similarity between the features of corresponding sub-blocks of any two nodes in the topology graph, and use the cosine similarity as a weight to construct a dimension. Single-mode inner adjacency matrix ; For each node in the topology graph, based on the single-modal intra-adjacency matrix Relation inference is performed by applying multi-layer graph convolution operations, i.e., each node is based on the single-modal intra-adjacency matrix. The weights in the graph adaptively aggregate the semantic features of other nodes to obtain the graph convolutional features corresponding to each node; By performing residual connection between the original sub-block features corresponding to each node and the graph convolution features, the graph-optimized sub-block features of each node are obtained. By combining the graph-optimized sub-block features of all nodes, the visible light sub-block feature set or the infrared sub-block feature set is obtained.

6. The cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception according to any one of claims 1 to 4, characterized in that, In step 4, the fine-grained cross-modal semantic alignment process is as follows: Calculate the distance between any two sub-block features between the visible light sub-block feature set and the infrared sub-block feature set, and construct a fully connected cross-modal alignment cost matrix; The cross-modal sub-block matching problem is modeled as an optimal transmission problem. Based on the Sinkhorn algorithm with entropy regularization, the cross-modal alignment cost matrix is ​​subjected to differentiable row and column normalization iteration. After convergence after a preset number of iterations, the cross-modal soft matching matrix is ​​output. The elements in the cross-modal soft matching matrix represent the association probability that the visible photo sub-block and the infrared sub-block belong to the same physical component.

7. The cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception according to any one of claims 1 to 4, characterized in that, In step 5, the weighted reconstruction process is as follows: For each visible photonic sub-block feature in the visible photonic sub-block feature set, the infrared sub-block feature set is weighted and aggregated according to the association probability corresponding to the cross-modal soft matching matrix to obtain the infrared reconstructed feature corresponding to the visible photonic sub-block feature. Then, the infrared reconstructed feature is residually fused or spliced ​​with the corresponding visible photonic sub-block feature to obtain the cross-modal aligned sub-block feature set. or For each infrared sub-block feature in the infrared sub-block feature set, the visible light sub-block feature set is weighted and aggregated according to the association probability corresponding to the cross-modal soft matching matrix to obtain the visible light reconstructed feature corresponding to the infrared sub-block feature. Then, the visible light reconstructed feature and the corresponding infrared sub-block feature are residually fused or spliced ​​to obtain the cross-modal aligned sub-block feature set.

8. The cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception according to any one of claims 1 to 4, characterized in that, In step 5, the vehicle re-identification process is as follows: Global aggregation is performed on the feature set of sub-blocks after cross-modal alignment to obtain global features. ; global features The data is fed into a BNNeck or normalization layer to obtain the target embedding features; The candidate embedding features of each candidate image in the candidate image library are obtained. Based on the similarity or distance between the target embedding features and the candidate embedding features, the candidate images are sorted from high to low similarity or from small to large distance. The candidate image ranked first is the same target. The target embedding features are mapped to each candidate image in the candidate image library.

9. The cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception according to any one of claims 1 to 4, characterized in that, It also includes end-to-end training and optimization of the group log-polar coordinate space-aware feature extraction, the modal intragraph-aware optimization, and the fine-grained cross-modal semantic alignment using a multi-objective joint loss function.

10. The cross-modal vehicle re-identification method guided by group logarithmic polar coordinate space perception according to claim 9, characterized in that, The multi-objective joint loss function is a weighted combination of global identity classification loss, global infrared-visible cross-modal triplet loss, fine-grained cross-modal optimal transmission alignment loss, intra-modal structure preservation regularization loss, negative sample constraint loss, and entropy regularization constraint term.

Images