Multi-camera target association method

Abstract

The invention provides a multi-camera target association method, which comprises the following steps of: acquiring a plurality of groups of images as training samples, and inputting the training samples into a target association model; a global identity feature extraction branch of the visual angle information adversarial learning module removes visual angle features irrelevant to pedestrian identity features in the global identity features to generate second global identity features; a global view angle feature extraction branch of the feature decomposition module decomposes view angle irrelevant features and view angle specific features from the global view angle features; fusing the view angle irrelevant feature with the second global identity feature to generate a third global identity feature; splicing the third global identity feature with the view angle irrelevant feature; inputting the third global identity feature and the splicing feature into a feature decoupling module; the feature decoupling module performs causal reasoning operation, continuously eliminates interference of view angle deviation on identity features in the training process, and improves robustness; and performing multi-camera target association based on the trained target association model.

Description

Technical Field

[0001] The invention belongs to the technical field of image recognition, and in particular relates to a multi-camera target association method. Background Art

[0002] Due to the significant differences in perspective, posture, etc. between aerial and ground cameras, air-ground pedestrian target association using heterogeneous cameras faces severe challenges in the field of computer vision. The core difficulty lies in how to effectively eliminate the interference of perspective differences on identity (ID) features while improving the robustness and generalization ability of feature representation.

[0003] Heterogeneous camera air-ground pedestrian object association focuses on identifying the same pedestrian from different viewpoints (or devices). Its core goal is to establish the identity association of the same pedestrian between images captured by heterogeneous camera devices (air-ground camera systems). However, the heterogeneous nature of aerial and ground cameras leads to significant differences in viewpoint, resolution, and pose, which exacerbates the difficulty of feature alignment. In addition, pedestrian pose variations, self-occlusion, and data distribution heterogeneity further increase the challenge of recognition. Due to these challenges, many object association algorithms designed for homogeneous viewpoints have significantly degraded in heterogeneous viewpoint tasks, struggling to effectively address the feature differences between air-ground viewpoints caused by viewpoint changes.

[0004] In recent years, researchers have proposed several innovative methods to improve the performance of object association under heterogeneous viewpoints. For example, H. Nguyen et al. proposed a two-branch network that significantly improved recognition performance under heterogeneous viewpoints by incorporating pedestrian attribute information. To address the appearance variation caused by height differences, they further designed a three-branch architecture based on the two-branch architecture. The newly added head feature localization branch effectively extracts local features and combines them with global features, further improving the model's recognition capabilities. Furthermore, Q. Zhang et al. proposed a view feature decoupling algorithm based on hierarchical subtraction and orthogonality loss, starting from the perspective of view decoupling. This method gradually eliminates view interference in global features through subtraction and orthogonality constraints, significantly improving recognition accuracy under heterogeneous viewpoints. Wang et al. proposed a cross-view object association method with self-calibration and adaptive cues. By dynamically adjusting view cues and combining local feature refinement, it improves cross-view recognition performance in air-ground person re-identification. These studies have proposed innovative solutions from different perspectives, effectively addressing the technical challenges posed by heterogeneous viewpoints. However, while attribute-guided two-branch and three-branch network approaches can improve object association performance under heterogeneous viewpoints, they also have limitations. These methods rely on prior attribute labels, which typically require extensive manual annotation. Furthermore, these networks are complex, especially when multiple branches and feature fusion modules are introduced, resulting in high computational resource consumption. On the other hand, while view feature decoupling algorithms based on layered subtraction and orthogonal loss optimize global feature representation by eliminating viewpoint interference, this approach can lead to the inadvertent deletion of some identity-related local features during the process of removing viewpoint-related information, resulting in information loss. This weakens the model's ability to capture certain details and compromises the complete representation of identity information. This information loss can be particularly pronounced in scenarios with significant pose variations or severe occlusion. Furthermore, current heterogeneous view object association algorithms inevitably overlook the identity discriminative information inherent in viewpoint features. This identity discriminative information should be fully mined and utilized, especially when dealing with these complex scenarios, to further enhance the richness of identity features. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention proposes a multi-camera target association method.

[0006] According to a first aspect of the present invention, a multi-camera target association method is provided, the method comprising the following steps:

[0007] Step S1: Acquire multiple sets of images as training samples, each set of images including multiple images taken at the same time from different camera perspectives, and input the training samples into a target association model. The target association model includes a feature extraction module, a view information adversarial learning module, a feature decomposition module, and a feature decoupling module; the feature extraction module extracts global identity features and global view features from the set of images;

[0008] Step S2: The global identity feature extraction branch of the view information adversarial learning module removes view features irrelevant to the pedestrian identity features from the global identity features to generate a second global identity feature; the global view feature extraction branch of the feature decomposition module decomposes the global view features into view-independent features and view-specific features; and the view-independent features are fused with the second global identity feature to generate a third global identity feature.

[0009] Step S3: splicing the third global identity feature and the view-independent feature to obtain a spliced ​​feature; inputting both the third global identity feature and the spliced ​​feature into a feature decoupling module;

[0010] The multi-layer perceptron in the feature decoupling module fuses the third global identity feature and the view-independent feature in the spliced ​​features to obtain a first fused feature. The first classifier in the feature decoupling module classifies and predicts the first fused feature to generate a first classification prediction result. The second classifier in the feature decoupling module, which shares parameters with the first classifier, classifies and predicts the third global identity feature to generate a second classification prediction result. The first classification prediction result and the second classification prediction result are differentially calculated to simulate TDE causal reasoning. The training process of the target association model is optimized based on the reasoning results.

[0011] Step S4: Perform multi-camera target association based on the trained target association model.

[0012] Preferably, in step S1, the feature extraction module includes a batch embedding layer, an attention layer, and a feedforward neural network layer connected in sequence. The batch embedding layer performs feature embedding on the group of images to obtain a view token and a CLS token respectively. The attention layer obtains token data based on the view token and the CLS token, and inputs the token data into the feedforward neural network layer to obtain a global view feature F. v and the global identity feature F id .

[0013] Preferably, in step S2, the global identity feature extraction branch of the view information adversarial learning module removes view features irrelevant to the pedestrian identity features from the global identity features to generate a second global identity feature, wherein:

[0014] The global identity feature extraction branch includes the gradient reversal layer and the view information adversarial discriminator D connected in sequence.va , global identity feature F v Input perspective information to fight against the discriminator D va Perform perspective classification. During the gradient backpropagation process, the gradient reversal layer uses the perspective information to counter the discriminator D va The gradient of is multiplied by the negative scalar -λ to achieve adversarial training;

[0015] During training, the adversarial discriminator D va Constrained by the first discriminant loss function:

[0016]

[0017] in, is the first discriminant loss function, N B is the total number of training samples in a batch, j is the training sample number, v j is the view label corresponding to the jth training sample, p(·) represents the predicted probability obtained by the softmax function, and D va The function represents the operation of the view classifier, and GRL(·) represents the operation of the gradient reversal layer.

[0018] Preferably, the loss function of the view information adversarial learning module is:

[0019]

[0020] in, F id The classification loss, is the loss function of the perspective information adversarial learning module, and α is the weight coefficient.

[0021] Preferably, in step S2, the global view feature extraction branch of the feature decomposition module decomposes the global view feature into view-independent features and view-specific features, wherein:

[0022] The global view feature extraction branch includes parallel average pooling layers and maximum pooling layers, convolutional layers connected to the parallel average pooling layers and maximum pooling layers, and a dynamic mask generation layer connected to the convolutional layers;

[0023] The average pooling layer and the maximum pooling layer perform average pooling and maximum pooling operations on the global view features respectively, and concatenate the two pooling results. The two pooling results are fused through the convolution layer and the result is expressed as the initial mask and recorded as in, d is the global perspective feature F v The feature dimension of ; initial mask Input dynamic mask generation layer, dynamic mask generation layer is activation function layer, activation function layer uses Gumbel-Sigmoid estimator to generate initial mask of input Processing is performed to generate the first dynamic mask and the second dynamic mask; the global viewing angle feature F v Performing element-wise dot multiplication operations with the first dynamic mask and the second dynamic mask, respectively, to obtain view-specific features and view-independent features, respectively; wherein the second dynamic mask is calculated based on the first dynamic mask, the first dynamic mask is denoted as M, and the second dynamic mask is denoted as 1-M;

[0024] The calculation of the Gumbel-Sigmoid estimator is implemented by the Gumbel noise sampling function and the Sigmoid function.

[0025] Initial mask For each dimension k∈{1,2,...,d}, the mask value corresponding to each dimension is recorded as θ k , then the k-th soft mask value M in the first mask M is generated by the Gumbel-Sigmoid estimator (k) The process is expressed as:

[0026]

[0027] Among them, u k ~Uniform(0,1), represents Gumbel noise sampled from a uniform distribution, τ is the temperature coefficient, and σ(·) represents the sigmoid activation function;

[0028] Total loss function of target association model The function value is passed back to θ k The gradient of is expressed as:

[0029]

[0030] in, To perform the derivative operation.

[0031] Preferably, the loss function of the feature decomposition module is:

[0032]

[0033] in, is the loss function of the feature decomposition module, is the classification loss of the global identity feature, is the classification loss of view-specific features, is the view-specific feature F vs View-independent feature F vi mutual information constraints between .

[0034] Preferably, the loss function of the feature decoupling module is:

[0035]

[0036] in, is the loss function of the feature decoupling module, y j is the identity label of the jth training sample; represents the prediction result of the target association model under the actual situation, represents the third global identity feature corresponding to the j-th training sample; represents the prediction result of the target-related target-related model under the counterfactual situation;

[0037] The total loss function of the target association model is:

[0038]

[0039] in, is the triplet loss;

[0040]

[0041] Among them, N B is the total number of samples in a Batch during training, is the feature of the jth training sample, and each training sample has no anchor sample; is the feature of the positive sample corresponding to the j-th training sample, is the feature of the negative sample corresponding to the jth training sample, and m is the minimum distance between the training sample and the positive sample.

[0042] According to a second aspect of the present invention, a multi-camera target association device is provided, the device comprising:

[0043] Feature acquisition module: configured to acquire multiple sets of images as training samples, each set of images including multiple images taken from different camera perspectives at the same time, and input the training samples into a target association model, which includes a feature extraction module, a view information adversarial learning module, a feature decomposition module, and a feature decoupling module; the feature extraction module extracts global identity features and global view features from the set of images;

[0044] Decomposition module: The global identity feature extraction branch of the view information adversarial learning module removes view features irrelevant to the pedestrian identity features from the global identity features to generate a second global identity feature. The global view feature extraction branch of the feature decomposition module decomposes the global view features into view-independent features and view-specific features. The view-independent features are then fused with the second global identity feature to generate a third global identity feature.

[0045] A decoupling module is configured to concatenate the third global identity feature with the view-independent feature to obtain a concatenated feature; and input both the third global identity feature and the concatenated feature into the feature decoupling module;

[0046] The multi-layer perceptron in the feature decoupling module fuses the third global identity feature and the view-independent feature in the spliced ​​features to obtain a first fused feature. The first classifier in the feature decoupling module classifies and predicts the first fused feature to generate a first classification prediction result. The second classifier in the feature decoupling module, which shares parameters with the first classifier, classifies and predicts the third global identity feature to generate a second classification prediction result. The first classification prediction result and the second classification prediction result are differentially calculated to simulate TDE causal reasoning. The training process of the target association model is optimized based on the reasoning results.

[0047] Association module: configured to perform multi-camera target association based on the trained target association model.

[0048] According to a third aspect of the present invention, there is provided an electronic device, comprising:

[0049] A processor, which is used to execute multiple instructions;

[0050] A memory for storing a plurality of instructions;

[0051] The plurality of instructions are used to be stored by the memory and loaded and executed by the processor to implement the method as described above.

[0052] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein a plurality of instructions are stored in the storage medium; the plurality of instructions are used for a processor to load and execute the aforementioned method.

[0053] The present invention introduces an identity-preserving view adversarial learning module (IP-VALM), which suppresses the expression of view information in the global identity feature extraction branch to improve the model's perception of identity features. The dynamic mask-based feature decomposition module (DM-FDM) decomposes view-independent features and view-specific features from the global view feature extraction branch through adaptive dynamic masking, and uses the view-independent features as a supplement to the global identity features to mine richer identity discrimination information. The causally-driven feature disentanglement module (CCFGD) constructs a causal structure model and incorporates view information as a deviation into it. Combined with the causal inference method, it continuously eliminates the interference of view deviation, thereby effectively improving the robustness of identity features.

[0054] The present invention has achieved significant performance improvements on the CARGO, AG-ReIDv1, and AG-ReIDv2 datasets, verifying its effectiveness in the heterogeneous view object association task.

[0055] The present invention has the following technical effects:

[0056] (1) The present invention introduces a GRL-based view information adversarial discriminator and utilizes an adversarial learning mechanism to suppress the interference of view information on identity feature extraction while retaining identity information, thereby enhancing the model's ability to perceive identity features under different view angles.

[0057] (2) This paper introduces the Gumbel-Sigmoid mechanism to generate a differentiable dynamic mask, which can refine and decompose viewpoint features, decouple viewpoint-specific features from viewpoint-independent features, and supplement the viewpoint-independent features into the global features to further mine identity discriminant information. At the same time, combined with the mutual information minimization loss, it further promotes feature decomposition, reduces information redundancy, and enhances the model's identity representation ability.

[0058] (3) The feature decoupling module of the present invention further eliminates the interference of viewpoint information and extracts more viewpoint-independent features. By constructing a causal structure model, viewpoint information is incorporated as a confounding factor, and combined with causal intervention and total direct effect (TDE) methods, the impact of viewpoint bias on model output is quantified from the perspective of causal inference. CCFGD can continuously eliminate the interference of viewpoint information during training, achieving effective decoupling of identity features from viewpoint information, thereby improving the model's ability to discriminate identity features and its robustness.

[0059] (4) The present invention suppresses perspective interference and retains identity discrimination information. By introducing an identity-preserving perspective information adversarial learning module and using the gradient reversal layer (GRL) and the perspective discriminator to construct an adversarial mechanism, the perspective-related interference information can be effectively suppressed while retaining identity-related features, so that the model has stronger cross-perspective identity discrimination and generalization capabilities.

[0060] (5) The present invention can finely decouple features and enhance feature representation capabilities. With the help of a dynamic mask mechanism, it can adaptively decompose view-specific features and view-independent features from view features, and use the latter as a supplement to enhance the global identity features. At the same time, the decomposition quality is improved by minimizing the loss of mutual information, effectively reducing information redundancy and enhancing the richness and expressiveness of the model's identity features.

[0061] (6) Introducing causal inference mechanism to improve robustness and interpretability (CCFGD module): By constructing a causal graph model and combining it with causal intervention (TDE) analysis, perspective information is modeled as a bias term, and its impact on identity characteristics is quantitatively stripped away, fundamentally eliminating the interference of perspective confounding factors, and improving the robustness, stability and interpretability of the model in complex environments.

[0062] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings, which constitute part of the present invention, are used to provide a further understanding of the present invention. The present invention is described with the following accompanying drawings. In the accompanying drawings:

[0064] Figure 1 The figure is a flowchart of a multi-camera target association method according to an embodiment of the present invention.

[0065] Figure 2 A schematic diagram of an architecture for a target association model according to an embodiment of the present invention.

[0066] Figure 3 This is a schematic diagram of a causal structure model according to one embodiment of the present invention.

[0067] Figure 4(A)-Figure 4(B) Schematic diagram of the recognition effect of the multi-camera target association method of the present invention. DETAILED DESCRIPTION

[0068] First combine Figure 1-Figure 2 The following describes a multi-camera target association method according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0069] Step S1: Acquire multiple sets of images as training samples, each set of images including multiple images taken at the same time from different camera perspectives, and input the training samples into a target association model. The target association model includes a feature extraction module, a view information adversarial learning module, a feature decomposition module, and a feature decoupling module; the feature extraction module extracts global identity features and global view features from the set of images;

[0070] Step S2: The global identity feature extraction branch of the view information adversarial learning module removes view features irrelevant to the pedestrian identity features from the global identity features to generate a second global identity feature; the global view feature extraction branch of the feature decomposition module decomposes the global view features into view-independent features and view-specific features; and the view-independent features are fused with the second global identity feature to generate a third global identity feature.

[0071] Step S3: splicing the third global identity feature and the view-independent feature to obtain a spliced ​​feature; inputting both the third global identity feature and the spliced ​​feature into a feature decoupling module;

[0072] The multi-layer perceptron in the feature decoupling module fuses the third global identity feature and the view-independent feature in the spliced ​​features to obtain a first fused feature. The first classifier in the feature decoupling module classifies and predicts the first fused feature to generate a first classification prediction result. The second classifier in the feature decoupling module, which shares parameters with the first classifier, classifies and predicts the third global identity feature to generate a second classification prediction result. The first classification prediction result and the second classification prediction result are differentially calculated to simulate TDE causal reasoning. The training process of the target association model is optimized based on the inference results, that is, the interference of viewpoint bias on the identity feature is continuously eliminated during the training process.

[0073] Step S4: Perform multi-camera target association based on the trained target association model.

[0074] The feature decoupling results of the present invention can improve the recognition performance of the target association model in a heterogeneous camera system.

[0075] Furthermore, in step S1, the feature extraction module includes a batch embedding layer, an attention layer, and a feedforward neural network layer connected in sequence. The batch embedding layer performs feature embedding on the group of images to obtain a view token and a CLS token respectively. The attention layer obtains token data based on the view token and the CLS token, and inputs the token data into the feedforward neural network layer to obtain a global view feature F. v and the global identity feature F id .

[0076] In this paper, the heterogeneous viewpoint object association AGP ReID dataset is represented as a triple: Among them, N D Represents the total number of samples in the dataset D; represents the i-th pedestrian image in the image space, H, W, C represent the height, width and number of channels of the image respectively; y i ∈Y={1,2,...,C id} is the pedestrian ID label, C id is the total number of categories in D; v i ∈V={v a ,v g} is the viewpoint label of the pedestrian image, where v a and v g Denote the aerial view label and the ground view label respectively. In order to facilitate model training and testing, D is further divided into training set D train and the test set D test , then: D=D train ∪D test ,and

[0077] During the training process, the input images sampled in a mini-batch and their corresponding labels are represented as where N B =P×K, P represents the number of different human categories sampled, K represents the number of samples for each category, N B Represents the total number of samples in a mini-batch. Then, X B Feed it into the ViT-base backbone network, and the present invention extracts the global view feature F through view token and CLS token respectively. v and the global identity feature F id :

[0078]

[0079] in, is the ViT feature extractor, which represents the mapping of the image space to the corresponding feature space. The present invention does not use internal layer subtraction and orthogonal loss. v and F id Constrained by classification loss:

[0080]

[0081] Among them, D gv The view classifier representing the global view feature branch, D gidrepresents the identity classifier of the global identity feature extraction branch. p(·) represents the predicted probability obtained by the softmax function.

[0082] In step S2, the global identity feature extraction branch of the view information adversarial learning module removes view features irrelevant to the pedestrian identity features from the global identity features to generate a second global identity feature, wherein:

[0083] The global identity feature extraction branch includes a gradient reversal layer (GLR) and a view information adversarial discriminator D connected in sequence. va , global identity feature F v Input perspective information to fight against the discriminator D va Perform perspective classification. During the gradient backpropagation process, the gradient reversal layer uses the perspective information to counter the discriminator D va The gradient of is multiplied by the negative scalar -λ to achieve adversarial training;

[0084] During training, the adversarial discriminator D va Constrained by the first discriminant loss function:

[0085]

[0086] in, is the first discriminant loss function, N B is the total number of training samples in a batch, j is the training sample number, v j is the view label corresponding to the jth training sample, p(·) represents the predicted probability obtained by the softmax function, and D va The function represents the operation of the view classifier, and GRL(·) represents the operation of the gradient reversal layer. In the present invention, the global identity feature extraction branch relies on the identity discrimination information extracted from the global features for prediction, but this information is susceptible to view bias, which may not only cause the model to be biased towards specific features, but also cause it to over-rely on information irrelevant to the identity, thereby weakening the recognition performance. In order to extract identity-specific features as much as possible and suppress the interference of view information, a view information adversarial learning module based on GRL is designed in this branch. In the view information adversarial learning module, the view information adversarial discriminator D va Constraints are performed through the first discriminant loss function.

[0087] Among them, D va and D in formula (3) gid It constitutes a collaborative optimization adversarial learning mechanism. During the training process, and Performing joint optimization, the loss function of the perspective information adversarial learning module is:

[0088]

[0089] in, F id The classification loss.

[0090] The core idea of ​​the total loss function is to make the discriminator D va It can identify perspective information that is irrelevant to identity features and enable the target association model to ignore these irrelevant features during learning. The key role of the gradient reversal layer (GRL) is to reverse the gradient by multiplying it by a negative scalar -λ during backpropagation, thereby forcing the model to optimize both the objective function and the adversarial loss function during training, namely:

[0091]

[0092] Among them, λ represents the gradient reversal coefficient, w represents D va The model parameters.

[0093] In the above formula, GRL reverses the direction of the gradient, allowing the target association model to learn how to ignore viewpoint information, making the learning of identity discriminant features more robust. Through this adversarial learning approach, the target association model enhances its ability to perceive identity features from different viewpoints to a certain extent, while at the same time reducing the interference of viewpoint information while retaining identity information.

[0094] Furthermore, in step S2, the global view feature extraction branch of the feature decomposition module decomposes the global view feature into view-independent features and view-specific features, wherein:

[0095] The global view feature extraction branch includes parallel average pooling layers and maximum pooling layers, convolutional layers connected to the parallel average pooling layers and maximum pooling layers, and a dynamic mask generation layer connected to the convolutional layers;

[0096] The average pooling layer and the maximum pooling layer perform average pooling and maximum pooling operations on the global view features respectively, and concatenate the two pooling results. The two pooling results are fused through the convolution layer and the result is expressed as the initial mask and recorded as in, d is The feature dimension of ; initial mask Input dynamic mask generation layer, dynamic mask generation layer is activation function layer, activation function layer uses Gumbel-Sigmoid estimator to generate initial mask of input Processing is performed to generate the first dynamic mask and the second dynamic mask; the global viewing angle feature F vPerforming element-wise dot multiplication operations with the first dynamic mask and the second dynamic mask, respectively, to obtain view-specific features and view-independent features, respectively; wherein the second dynamic mask is calculated based on the first dynamic mask, the first dynamic mask is denoted as M, and the second dynamic mask is denoted as 1-M;

[0097] The calculation of the Gumbel-Sigmoid estimator is implemented by the Gumbel noise sampling function and the Sigmoid function.

[0098] Initial mask For each dimension k∈{1,2,...,d}, the mask value corresponding to each dimension is recorded as θ k , then the k-th soft mask value M in the first mask M is generated by the Gumbel-Sigmoid estimator (k) The process is expressed as:

[0099]

[0100] Among them, u k ~Uniform(0,1), represents Gumbel noise sampled from a uniform distribution, τ is the temperature coefficient, and σ(·) represents the sigmoid activation function;

[0101] Total loss function of target association model The function value is passed back to θ k The gradient of is expressed as:

[0102]

[0103] in, To perform the derivative operation.

[0104] In the present invention, in addition to including the viewing angle related information, F v It also potentially retains some view-independent features related to identity discrimination. These residual identity clues can be regarded as F id Based on this, the feature decomposition module uses the Gumbel-Sigmoid estimator to generate a differentiable dynamic mask, and uses this to transform F v Decomposed into view-specific and view-independent parts. v Different dimensions of may carry different types of information, so the feature decomposition module achieves effective decoupling and separation through dimension-level feature screening. v Perform maximum pooling and average pooling separately: maximum pooling extracts the most significant local features, while average pooling retains the overall information. Subsequently, the two pooling results are concatenated and fused through a 1×1 convolutional layer, and finally the initial feature mask is generated using the Sigmoid activation function. The process can be expressed as:

[0105]

[0106] in, d is the feature dimension, σ(·) represents the Sigmoid activation function, MP represents the maximum pooling operation, AP represents the average pooling operation, and [·,·] represents the concatenation operation.

[0107] Traditional methods usually Binarization is performed, that is, each mask value is hard-quantized to 0 or 1. This approach will cause rigid screening in feature separation, making it difficult to capture the correlation between features, and thus leading to a certain amount of information loss. To solve this problem, the present invention introduces Gumbel noise in the feature decomposition module and generates a soft mask in combination with the Sigmoid function. Unlike fixed binary values, the soft mask dynamically represents the importance of the feature in the form of probability (ranging between 0 and 1). In addition, the Gumbel-Sigmoid estimator can realize derivable sampling of the mask during training, enabling it to be coordinated with downstream task objectives for optimization.

[0108] For each feature dimension k∈{1,2,…,d}, Gumbel noise g k The sampling process is expressed as:

[0109] g k =-log(-log(u k )) (10)

[0110] Among them, u k ~Uniform(0,1) represents Gumbel noise sampled from a uniform distribution.

[0111] Initial mask Each dimension k∈{1,2,…,d} (d is the feature dimension), the mask value corresponding to each dimension is recorded as θ k , then the process of generating the kth soft mask value in the first mask M through the Gumbel-Sigmoid estimator is expressed as:

[0112]

[0113] Here, τ is the temperature coefficient and σ(·) represents the sigmoid activation function.

[0114] The total loss of the target association model is propagated back to the element θ in the first dynamic mask k The gradient of is expressed as:

[0115]

[0116] in, To perform the derivative operation, is the total loss of the target association model.

[0117] From F v Decompose the view-specific feature F vs View-independent feature F vi The process can be expressed as:

[0118]

[0119] in, In order to make the separated features still retain specific semantic information, the present invention uses classification loss to calculate F vs To conduct supervision:

[0120]

[0121] Among them, D vs represents the view feature classifier, represents the j-th view-specific feature corresponding to the j-th training sample.

[0122] In step S2, the view-independent feature is fused with the second global identity feature to generate a third global identity feature, including:

[0123]

[0124] In addition, in order to promote F vs With F vi The present invention further introduces the mutual information minimization loss between the two. Given variables u and V, their mutual information can be defined as:

[0125]

[0126] In order to minimize the mutual information between u and V, an optional method is to use the CLUB method to calculate the upper bound of the mutual information between variables:

[0127]

[0128] However, in the above formula, the conditional distribution p(v|u) between u and V is difficult to express and calculate. For this, we can further use the variational distribution q v (v|u) to approximate p(v|u), the above formula can be expanded to vCLUB, which is defined as follows:

[0129]

[0130] Among them, q v (v|u) can be estimated through an encoder network.

[0131] Finally, F vs With F vi The mutual information constraint between is expressed as:

[0132]

[0133] The loss function of the feature decomposition module is:

[0134]

[0135] in, is the loss function of the feature decomposition module, is the classification loss of the global identity feature, is the classification loss of view-specific features, is the view-specific feature F vs View-independent feature F vi mutual information constraints between .

[0136] In the present invention, the feature decoupling module is based on causal drive. The following describes the principle of the feature decoupling module of the present invention based on causal drive. In order to promote the decoupling of identity features and view features and eliminate the influence of view bias, the following constructs Figure 3 The causal relationship between perspective information and identity information is described by the causal model shown in . Figure 3 In the figure, the left figure shows the causal structural model (SCM) under the factual situation, where V represents the perspective-specific factor, X represents the identity-specific factor, Z represents the intermediate feature generated by the target association model, and Y represents the observable variable of the identity label, which is jointly determined by the variables V and X through the intermediate variable Z.

[0137] like Figure 3 As shown in the SCM on the left, the first path is X→Z→Y, indicating that identity-specific factors influence the identity prediction result through the feature extraction process of the target association model. Ideally, these factors directly determine the predicted identity label, indicating that in the absence of interfering factors, identity information should be the primary determinant of identity prediction. However, in real-world scenarios, there is a complex feature coupling relationship between viewpoint and identity. The two may share the same expression space in terms of appearance features, and the distortion of image features caused by viewpoint changes exacerbates this confusion. Because conventional target association models cannot effectively decouple viewpoint bias, they often mistakenly use viewpoint-related features as the basis for identity discrimination, thus transforming the above path into V→X→Z→Y. This path demonstrates that in the real world, V and X are spuriously correlated, and the representation Z generated by the target association model is not only determined by X but also influenced by V.

[0138] To mitigate the interference caused by perspective bias, an effective method in causal inference is to use a backdoor adjustment strategy. This is achieved by directly intervening on the variable V (do(V=v)). This operation simulates an ideal intervention scenario by severing the connection between X and all its parent nodes, thereby explicitly controlling the impact of the perspective variable V on X. In this way, the causal effect of variable X on the observed outcome Y can be accurately estimated. The causal estimation process based on the backdoor adjustment formula can be formally expressed as:

[0139]

[0140] However, in practical applications, it is difficult to precisely control the perspective variable V. To this end, this patent draws on the Total Direct Effect (TDE) framework and proposes an approximate solution strategy to simulate the causal effect of identity. This method estimates the causal effect by comparing the results under two paths: the first path V→X→Z→Y corresponds to the observed result Y Z,X,V , reflecting the output under the joint conditions of perspective V and X; the second path represents the intervention of V, that is, fixing X = v, thus obtaining the result Y Z,V , this result only considers the causal effect of the view factor. Therefore, TDE can be approximately calculated as:

[0141] TDE≈Y Z,X,V -Y Z,V (twenty two)

[0142] Among them, Y Z,X,V = P(Y|X,V), which represents the observation result of the proposed model under the joint condition of view V and X, Y Z,V =P(Y|X=v)P(V=v), which represents the observation results of the proposed model under the perspective V as the dominant factor.

[0143] To calculate Y Z,V , the present invention explicitly introduces perspective causal factors, constructs a prediction branch dominated by perspective variable V, and uses counterfactual feature classifier The goal of this branch is to help the model understand what kind of misprediction may occur when the perspective bias is entangled with the identity feature and V becomes the main causal factor. Considering that there may be a complex nonlinear interaction between V and X, the present invention uses a multi-layer perceptron (MLP) to implement the perspective feature F v and identity characteristics The nonlinear fusion of , its calculation process can be expressed as:

[0144]

[0145] To complete the calculation of TDE, the present invention introduces an auxiliary identity classifier Its counterfactual classifier Shared parameters. On this basis, the identity prediction under counterfactual and factual conditions is defined as:

[0146]

[0147] Finally, the counterfactual discriminant loss function can be expressed as follows:

[0148]

[0149] in, is the loss function of the feature decoupling module, y j is the identity label of the jth training sample. Represents the prediction result of the target association model under the actual situation, where is the auxiliary classifier, Represents the enhanced features of the jth training sample (the third global identity feature) represents the prediction result of the target association model under the counterfactual situation, where is the counterfactual feature classifier, represents the counterfactual feature of the jth training sample.

[0150] With the help of the above formula, the feature decoupling module can continuously eliminate the influence of perspective bias during the training process, thereby improving the robustness of the ID discriminant features.

[0151] Will As the final output feature of the model, it is further optimized by triplet loss:

[0152]

[0153] Among them, N B is the total number of samples in a Batch during training, is the feature of the jth training sample, and each training sample has no anchor sample; is the feature of the positive sample corresponding to the j-th training sample, is the feature of the negative sample corresponding to the jth training sample, and m is the minimum distance between the training sample and the positive sample.

[0154] The total loss function of the target association model is:

[0155]

[0156] In the present invention, the feature decomposition module generates learnable masks and uses these learnable masks to further transform the global view feature F vDecomposed into view-specific features F vs View-Invariant Features vi At the same time, the mutual information minimization loss is introduced to F vs 、F vi Constraints are imposed to promote feature decomposition and reduce feature redundancy to enhance the diversity of information expression between different features. vi As a supplement to the global identity feature, it is further integrated into the global identity feature F id To improve the representational capabilities of global features. Although this reduces sensitivity to viewpoint features to a certain extent, it cannot completely eliminate the interference of viewpoint information. Viewpoint bias remains the primary factor affecting target association performance under heterogeneous viewpoints. To further decouple identity features from viewpoint features and eliminate the impact of viewpoint bias, a causal-driven feature decoupling module is further employed. In this module, a causal model is constructed and, based on TDE theory, the impact of viewpoint bias on identity features is gradually eliminated, thereby improving the robustness of identity features under different viewpoints.

[0157] In the present invention, it is believed that the perspective feature not only contains perspective information, but also implies identity discrimination features related to identity. These features can be used as a supplement to the global identity feature to enhance the identity representation ability of the model. However, the global feature may be mixed with a certain perspective information, which will interfere with the expression of the identity feature. Therefore, it is necessary to refine the perspective features extracted by the perspective branch, distinguish the identity-specific features from the perspective-specific features, and supplement the identity-specific features to the global features. In addition, the global feature extraction branch should extract perspective-invariant features by suppressing the interference of perspective information to ensure the robustness of the global feature. To this end, in order to eliminate the interference of perspective deviation on the identity (ID) feature and at the same time improve the robustness and generalization ability of feature representation, the present invention effectively removes the perspective interference in the global feature and fully mines the identity discrimination information in the perspective feature, thereby further improving the performance of target association under heterogeneous perspectives.

[0158] The datasets used in the experiments of this paper include CARGO, AG-ReID.v1, and AG-ReID.v2. A brief summary of each dataset is shown in Table 1.

[0159] (1) CARGO: The CARGO dataset is a simulation dataset that uses 13 virtual cameras to capture images from both aerial and ground perspectives. The dataset contains 5,000 identities and 108,563 images. The training set contains 51,451 images and 2,500 identities, and the test set contains 51,024 images and the remaining 2,500 identities. The CARGO dataset uses four protocols to evaluate the model, namely, ALL, and The protocol “ALL” is used to comprehensively evaluate the retrieval performance, while the other three protocols focus on specific retrieval modes.

[0160] (2) AG-ReID.v1: The image data of the AG-ReID.v1 dataset is collected by a combination of unmanned aerial vehicles (UAVs) and ground-based fixed surveillance cameras. All data are taken from actual locations, and the image capture height ranges from 14 to 45 meters. The dataset contains 21,893 images of 388 identities (IDs), of which 199 identities (IDs) are used for training and 189 identities (IDs) are used for testing. In addition, 15 attribute annotations are provided for each identity. The cross-view evaluation protocol of this dataset includes two evaluation methods: A→G and G→A.

[0161] (3) AG-ReID.v2: AG-ReID.v2 is an improved version of AG-ReID.v1. The dataset contains 1615 identities and a total of 100,502 images. The images are taken from three different shooting platforms: drones, wearable cameras, and CCTVs, with shooting heights ranging from 14 meters to 45 meters. The AG-ReID.v2 dataset is divided into training and test sets, containing 807 and 808 identities respectively, and 15 attributes are annotated at the identity level. In addition, the dataset also provides four cross-view evaluation protocols: A→C, A→W, C→A, and A→W.

[0162] We evaluate models using two key metrics: Rank-1 Cumulative Matching Characteristic (CMC) and Mean Average Precision (mAP). Rank-1 CMC measures the model's ability to correctly match queries to the top-ranked gallery examples, while mAP provides an overall performance assessment by averaging precision across all recall levels.

[0163] This paper uses the PyTorch framework for model training and deployment, and all experiments are performed on an Nvidia 3090 GPU. The backbone network of the model uses the ViT-Base model pre-trained on the ImageNet dataset. The input image is cropped to 256×128 pixels during the preprocessing stage, and data augmentation methods such as random cropping, color jittering, and random occlusion are used to improve the generalization and robustness of the model. The image is tokenized before being input into the ViT model, with the patch and stride size set to 16×16 and the token feature dimension set to 768.

[0164] Table 1

[0165]

[0166] This paper uses the PyTorch framework for model training and deployment, and all experiments are performed on an Nvidia 3090 GPU. The backbone network of the model uses the ViT-Base model pre-trained on the ImageNet dataset. The input image is cropped to 256×128 pixels during the preprocessing stage, and data augmentation methods such as random cropping, color jittering, and random occlusion are used to improve the generalization and robustness of the model. The image is tokenized before being input into the ViT model, with the patch and stride size set to 16×16 and the token feature dimension set to 768.

[0167] During the training process, the stochastic gradient descent (SGD) optimizer was used, the batch size was set to 128, and the initial learning rate was 0.008. Combined with the cosine learning rate decay strategy, the learning rate was increased from 8×10 -3 Gradually decayed to 1.6×10 -6 , ensuring the stability and convergence of the training process. The entire model training was performed for 120 epochs. No data augmentation or reordering operations were used during the inference phase.

[0168] The object association model (CAVAD) of the present invention and the most advanced algorithm were compared in experiments on the above-mentioned datasets.

[0169] CARGO: The comparison results of the proposed CAVAD and the most advanced algorithms on the CARGO dataset are shown in Table 2. Under the ALL setting (reflecting the overall retrieval performance under mixed air-ground perspectives), CAVAD achieved the best retrieval performance, obtaining a Rank1 of 73.72% and a mAP of 67.44%, significantly outperforming other cross-perspective methods. Under the setting (the retrieval task within the ground view), CAVAD Rank1 accuracy reached 84.82%, ranking second, and mAP reached 80.14%, an increase of 4.72% compared to SeCa4. In the most challenging setting (image retrieval inside the aerial view), CAVAD achieves the highest mAP of 76.27% and the highest Rank1 accuracy of 85.00%. In the setting (air-ground cross-view image retrieval), CAVAD achieved an mAP of 58.5% and a Rank1 accuracy of 63.13%. Although slightly lower than SeCap's performance, it is still significantly better than other methods, indicating that compared with most methods, CAVAD can better learn perspective-invariant features and promote the improvement of recognition performance. It is worth mentioning that VDT, as a strong baseline method, is used as the backbone network in SeCap, while CAVAD uses ViT-Base as the backbone. In this context, CAVAD still shows matching performance and generalization ability that are better than most methods, which further shows that its performance improvement is mainly due to the algorithm itself, rather than relying on a robust baseline model.

[0170] AG-ReID.v1: The comparison results of the proposed CAVAD and the most advanced algorithms on the AG-ReID.v1 dataset are shown in Table 3. In the A→G task, CAVAD's Rank1 reached 84.67%, leading the other methods; the mAP was 75.76%, second only to SeCap (whose mAP reached 76.16%). In the G→A task, CAVAD's Rank1 was 86.83%, second only to SeCap (whose Rank1 reached 87.01%); the mAP value was 78.88%, leading the other methods. The above results show that in the heterogeneous view pedestrian retrieval task, compared with most methods, CAVAD has better performance in terms of cross-view matching accuracy and robustness.

[0171] AG-ReID.v2: Table 4 compares the performance of the proposed CAVAD with five mainstream methods in four representative cross-view retrieval tasks on the AG-ReIDv2 dataset. These tasks cover challenging real-world scenarios such as long-distance cross-platform matching (air-ground) and strong perspective changes (wearable and ground / aerial). In the A→C task, CAVAD achieved a mAP of 81.07%, the highest among all methods, with a Rank1 of 88.16%, second only to AG-ReIDv2 (88.77%). In the C→A setting, CAVAD achieved a Rank1 of 87.24% and a mAP of 79.74%, second only to SeCap. In the A→W task, the retrieval spans the aerial and wearable perspectives, with significant appearance changes and local perspective restrictions. CAVAD achieved a Rank1 of 91.58% and a mAP of 85.12%, the best. In the W→A task, CAVAD achieved a Rank1 of 87.31%, with mAP reaching the best 80.95%, demonstrating good generalization ability.

[0172] Table 2

[0173]

[0174] Table 3

[0175]

[0176] Table 4

[0177]

[0178] Table 5

[0179]

[0180] In order to evaluate the contribution of each module, a comprehensive ablation experiment is conducted on the CARGO dataset. The components tested in the experiment include the ViT backbone network (as a baseline), IP-VALM, DM-FDM and CCFGD. As shown in Table 5, under four evaluation protocols ('ALL', ) shows the Rank-1 (%) and mean average precision (mAP%) results corresponding to each protocol.

[0181] When the IP-VALM module is introduced into the baseline, the performance is significantly improved. Under the "ALL" evaluation protocol, the Rank-1 is improved by 2.88% and the mAP is improved by 3.33%.

[0182] exist Under the evaluation protocol, mAP is improved by 2.29%.

[0183] exist Under the evaluation protocol, mAP is improved by 1.53%.

[0184] Especially in the most challenging The performance improvement under the evaluation protocol was particularly significant, with Rank-1 improving by 4.99% (from 43.13% to 48.12%) and mAP increasing by 3.33% (from 40.11% to 43.44%). This significant improvement is primarily due to IP-VALM's ability to suppress the interference of viewpoint information in the global feature extraction branch through adversarial learning of viewpoint information. This effectively enhances the ability to perceive identity features from different viewpoints while preserving identity information.

[0185] When DM-FDM is further introduced on the basis of "Baseline+IP-VALM" but not used In the case of , the model improves Rank-1 and mAP by 4.81% and 5.53% respectively under the “ALL” evaluation protocol; Under the protocol, mAP increased by 2.12%; Under the protocol, mAP increased by 2.94%; Under the protocol, Rank-1 and mAP are improved by 9.38% and 9.36% respectively. In the case of , under the “ALL” evaluation protocol, Rank-1 and mAP are further improved to 71.15% and 64.35% under the “ALL” protocol, an increase of 1.92% and 1.95% respectively; Under the protocol, mAP increased by 1.29%; Under the protocol, Rank-1 and mAP are further improved to 82.5% and 73.15% respectively; Under the protocol, Rank-1 and mAP are further improved to 59.38% and 54.36% respectively. The above results show that after the introduction of IP-VALM, the cross-view retrieval performance of the model is significantly improved. This is because under the action of DM-FDM, the features extracted by the view branch are effectively decomposed into view-specific features and view-independent features. Among them, the identity information contained in the view-independent features further strengthens the expression of the global identity features, thereby enhancing the feature discrimination ability. This enables the model to better maintain consistency and robustness when dealing with identity discrimination under different viewpoints. It also shows that The introduction of can further promote the decomposition of features, reduce the information redundancy between features, and thus improve the representation ability of features.

[0186] Finally, the model combined with the CCFGD module showed further performance improvement. Under the "ALL" evaluation protocol, Rank-1 and mAP further improved to 73.72% and 67.44%, respectively, an increase of 2.57% and 3.09%; Under the evaluation protocol, Rank-1 and mAP further improved to 84.82% and 80.14%, respectively, an increase of 1.78% and 3.1%; Under the evaluation protocol, Rank-1 and mAP further increased to 63.13% and 58.5%, respectively, an increase of 3.75% and 4.14%, respectively. The above results show that after the introduction of the IP-VALM module, although the global identity feature extraction branch has suppressed the interference of viewpoint information to a certain extent, it cannot completely eliminate the influence of viewpoint information. In addition, the enhanced information introduced from DM-FDM may contain certain viewpoint information interference. Although the introduction of DM-FDM improves performance, this improvement is relatively limited. Based on the first two modules introduced, CCFGD establishes an effective causal model for viewpoint interference. Finally, with the help of the causal inference method of total direct effect, the interference of viewpoint information is continuously eliminated during the training process. While promoting the decomposition of DM-FDM features, it further improves the robustness of the global identity feature expression, thereby enhancing the robustness of the model and further improving the cross-viewpoint recognition performance of the model.

[0187] In order to deeply analyze the feature distribution characteristics of CAVAD in cross-view object association, t-SNE is used to visualize the pedestrian image features extracted based on the CARGO dataset and compared with the Baseline method. The comparison results are shown in the figure below. Figure 4(A)-Figure 4(B) As shown. Figure 4(A)-Figure 4(B)As can be observed, in the baseline method (Figure 4(A)), most features exhibit significant intra-class dispersion, with a particularly pronounced feature offset between ground and aerial perspectives. This indicates that this method fails to effectively align identity features from different perspectives when modeling cross-view features, resulting in a loose intra-class structure. In contrast, the CAVAD method (Figure 4(B)) exhibits a more compact intra-class clustering structure. In this method, aerial and ground samples for most features are closer or intertwined in feature space, indicating that the learned features are more view-invariant. Furthermore, Figure 4(B) shows that both aerial and ground samples exhibit improved compactness within their corresponding specific perspectives. This significant improvement is primarily due to the view adversarial training mechanism and causal inference architecture introduced in the CAVAD model. Through the view adversarial module, the model learns view-independent identity representations, suppressing redundant information interference caused by viewpoint variations. The causal inference strategy further enables the model to explore the essential causal factors of identity, thereby removing non-causal viewpoint artifacts. DM-FDM extracts effective identity discriminant features from the view branch as a supplement to the global identity features, which improves the feature representation ability under different viewpoints to a certain extent. These mechanisms work together to make the identity representation learned by the model more consistent under different viewpoints.

[0188] The present invention provides a multi-camera target association device, the device comprising:

[0189] Feature acquisition module: configured to acquire multiple sets of images as training samples, each set of images including multiple images taken from different camera perspectives at the same time, and input the training samples into a target association model, which includes a feature extraction module, a view information adversarial learning module, a feature decomposition module, and a feature decoupling module; the feature extraction module extracts global identity features and global view features from the set of images;

[0190] Decomposition module: The global identity feature extraction branch of the view information adversarial learning module removes view features irrelevant to the pedestrian identity features from the global identity features to generate a second global identity feature. The global view feature extraction branch of the feature decomposition module decomposes the global view features into view-independent features and view-specific features. The view-independent features are then fused with the second global identity feature to generate a third global identity feature.

[0191] A decoupling module is configured to concatenate the third global identity feature with the view-independent feature to obtain a concatenated feature; and input both the third global identity feature and the concatenated feature into the feature decoupling module;

[0192] The multi-layer perceptron in the feature decoupling module fuses the third global identity feature and the view-independent feature in the spliced ​​features to obtain a first fused feature. The first classifier in the feature decoupling module classifies and predicts the first fused feature to generate a first classification prediction result. The second classifier in the feature decoupling module, which shares parameters with the first classifier, classifies and predicts the third global identity feature to generate a second classification prediction result. The first classification prediction result and the second classification prediction result are differentially calculated to simulate TDE causal reasoning. The training process of the target association model is optimized based on the inference results, that is, the interference of viewpoint bias on the identity feature is continuously eliminated during the training process.

[0193] Association module: configured to perform multi-camera target association based on the trained target association model.

[0194] An embodiment of the present invention further provides an electronic device, including:

[0195] A processor, which is used to execute multiple instructions;

[0196] A memory for storing a plurality of instructions;

[0197] The plurality of instructions are used to be stored by the memory and loaded and executed by the processor to implement the method as described above.

[0198] An embodiment of the present invention further provides a computer-readable storage medium, wherein a plurality of instructions are stored in the storage medium; the plurality of instructions are used for a processor to load and execute the method described above.

[0199] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0200] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, which may be electrical, mechanical or other forms.

[0201] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0202] In addition, the functional units in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional units.

[0203] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a physical server, or a network cloud server, etc., and requires the Ubuntu operating system to be installed) to perform some of the steps of the method described in various embodiments of the present invention. The aforementioned storage medium includes: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0204] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Any simple modifications, equivalent changes and modifications made to the above embodiment based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A multi-camera target association method, characterized in that: Methods include: Step S1: Acquire multiple sets of images as training samples, each set of images includes multiple images taken from different camera perspectives at the same time, and input the training samples into a target association model. The target association model includes a feature extraction module, a view information adversarial learning module, a feature decomposition module, and a feature decoupling module; The feature extraction module extracts global identity features and global view features from the set of images; Step S2: The global identity feature extraction branch of the view information adversarial learning module removes view features irrelevant to the pedestrian identity features from the global identity features to generate a second global identity feature; The global view feature extraction branch of the feature decomposition module decomposes the global view features into view-independent features and view-specific features; The view-independent feature is fused with the second global identity feature to generate a third global identity feature; Step S3: splicing the third global identity feature and the view-independent feature to obtain a spliced ​​feature; inputting both the third global identity feature and the spliced ​​feature into a feature decoupling module; The multi-layer perceptron of the feature decoupling module fuses the third global identity feature and the view-independent feature in the splicing feature to obtain the first fused feature; The first classifier of the decoupling module performs classification prediction on the first fusion feature to generate a first classification prediction result. The second classifier in the feature decoupling module, which shares parameters with the first classifier, performs classification prediction on the third global identity feature to generate a second classification prediction result. A differential operation is performed on the first classification prediction result and the second classification prediction result to simulate TDE causal reasoning. Optimize the training process of the target association model based on the inference results; Step S4: Perform multi-camera target association based on the trained target association model.

2. The method according to claim 1, wherein In step S1, the feature extraction module includes a batch embedding layer, an attention layer, and a feedforward neural network layer connected in sequence. The batch embedding layer embeds features of the group of images to obtain a view token and a CLS token respectively. The attention layer obtains token data based on the view token and the CLS token, and inputs the token data into the feedforward neural network layer to obtain a global view feature F. v and the global identity feature F id .

3. The method according to claim 2, wherein In step S2, the global identity feature extraction branch of the view information adversarial learning module removes view features irrelevant to the pedestrian identity features from the global identity features to generate a second global identity feature, wherein: The global identity feature extraction branch includes the gradient reversal layer and the view information adversarial discriminator D connected in sequence. va , global identity feature F v Input perspective information to the adversarial discriminator D va Perform perspective classification. During the gradient backpropagation process, the gradient reversal layer uses the perspective information to counter the discriminator D va The gradient of is multiplied by the negative scalar -λ to achieve adversarial training; During training, the adversarial discriminator D va Constrained by the first discriminant loss function: in, is the first discriminant loss function, N B is the total number of training samples in a batch, j is the training sample number, v j is the view label corresponding to the jth training sample, p(·) represents the predicted probability obtained by the softmax function, and D va The function represents the operation of the view classifier, and GRL(·) represents the operation of the gradient reversal layer.

4. The method according to claim 3, wherein The loss function of the perspective information adversarial learning module is: in, F id The classification loss, is the loss function of the perspective information adversarial learning module, and α is the weight coefficient.

5. The method according to claim 4, wherein In step S2, the global view feature extraction branch of the feature decomposition module decomposes the global view feature into view-independent features and view-specific features, wherein: The global view feature extraction branch includes parallel average pooling layers and maximum pooling layers, convolutional layers connected to the parallel average pooling layers and maximum pooling layers, and a dynamic mask generation layer connected to the convolutional layers; The average pooling layer and the maximum pooling layer perform average pooling and maximum pooling operations on the global view features respectively, and concatenate the two pooling results. The two pooling results are fused through the convolution layer and the result is expressed as the initial mask and recorded as in, d is the global perspective feature F v The feature dimension of ; initial mask Input dynamic mask generation layer, dynamic mask generation layer is activation function layer, activation function layer uses Gumbel-Sigmoid estimator to generate initial mask of input Processing is performed to generate the first dynamic mask and the second dynamic mask; the global viewing angle feature F v Performing element-wise dot multiplication operations with the first dynamic mask and the second dynamic mask, respectively, to obtain view-specific features and view-independent features, respectively; wherein the second dynamic mask is calculated based on the first dynamic mask, the first dynamic mask is denoted as M, and the second dynamic mask is denoted as 1-M; The calculation of the Gumbel-Sigmoid estimator is implemented by the Gumbel noise sampling function and the Sigmoid function. For each dimension k∈{1,2,...,d}, the mask value corresponding to each dimension is recorded as θ k , then the k-th soft mask value M in the first mask M is generated by the Gumbel-Sigmoid estimator (k) The process is expressed as: Among them, u k ~Uniform(0,1), represents Gumbel noise sampled from a uniform distribution, τ is the temperature coefficient, and σ(·) represents the sigmoid activation function; Total loss function of target association model The function value is passed back to θ k The gradient of is expressed as: in, To perform the derivative operation.

6. The method according to claim 5, wherein The loss function of the feature decomposition module is: in, is the loss function of the feature decomposition module, is the classification loss of the global identity feature, is the classification loss of view-specific features, is the view-specific feature F vs View-independent feature F vi mutual information constraints between .

7. The method according to claim 6, wherein The loss function of the feature decoupling module is: in, is the loss function of the feature decoupling module, y j is the identity label of the jth training sample; represents the prediction result of the target association model under the actual situation, represents the third global identity feature corresponding to the j-th training sample; represents the prediction result of the target-related target-related model under the counterfactual situation; The total loss function of the target association model is: in, is the triplet loss; Among them, N B is the total number of samples in a Batch during training, is the feature of the jth training sample, and each training sample has no anchor sample; is the feature of the positive sample corresponding to the j-th training sample, is the feature of the negative sample corresponding to the jth training sample, and m is the minimum distance between the training sample and the positive sample.

8. A multi-camera target association device, characterized in that: The device includes: Feature acquisition module: configured to acquire multiple sets of images as training samples, each set of images including multiple images taken from different camera perspectives at the same time, and input the training samples into a target association model, which includes a feature extraction module, a view information adversarial learning module, a feature decomposition module, and a feature decoupling module; the feature extraction module extracts global identity features and global view features from the set of images; Decomposition module: The global identity feature extraction branch of the view information adversarial learning module removes view features irrelevant to the pedestrian identity features from the global identity features to generate a second global identity feature. The global view feature extraction branch of the feature decomposition module decomposes the global view features into view-independent features and view-specific features. The view-independent features are then fused with the second global identity feature to generate a third global identity feature. A decoupling module is configured to concatenate the third global identity feature with the view-independent feature to obtain a concatenated feature; and input both the third global identity feature and the concatenated feature into the feature decoupling module; The multi-layer perceptron in the feature decoupling module fuses the third global identity feature and the view-independent feature in the spliced ​​features to obtain a first fused feature. The first classifier in the feature decoupling module classifies and predicts the first fused feature to generate a first classification prediction result. The second classifier in the feature decoupling module, which shares parameters with the first classifier, classifies and predicts the third global identity feature to generate a second classification prediction result. The first classification prediction result and the second classification prediction result are differentially calculated to simulate TDE causal reasoning. The training process of the target association model is optimized based on the reasoning results. Association module: configured to perform multi-camera target association based on the trained target association model.

9. An electronic device comprising: A processor, which is used to execute multiple instructions; A memory for storing a plurality of instructions; The plurality of instructions are used to be stored in the memory and loaded and executed by the processor according to any one of claims 1 to 7.

10. A computer-readable storage medium, wherein a plurality of instructions are stored in the storage medium; the plurality of instructions are used for a processor to load and execute the method according to any one of claims 1 to 7.

Images