An adaptive fusion pedestrian re-identification method, device, equipment and medium

Abstract

The present application relates to the field of image recognition, and discloses a self-adaptive fusion pedestrian re-identification method, device, equipment and medium, the method comprises adopting a heterogeneous double-branch backbone network to extract a local facial visual feature and a full-body discriminative feature irrelevant to clothes respectively; the semantic confidence score of the local facial visual feature is calculated in the form of posterior probability after pre-training the text semantic anchor point; the entropy regularization strength of the optimal transmission model is dynamically adjusted according to the semantic confidence score, the marginal constraint and the entropy regularization constraint are introduced in the feature fusion process, and the self-adaptive fusion and consistency enhancement of the local facial feature and the full-body feature in the distribution level are realized. The present application can realize more stable and robust identity feature expression in the dressing change scene, and improve the accuracy of pedestrian re-identification and the generalization ability across scenes.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and specifically to an adaptive fusion method, apparatus, device, and medium for pedestrian re-identification. Background Technology

[0002] Pedestrian re-identification is a crucial research area in computer vision, aiming to retrieve and match individuals with the same identity from massive amounts of pedestrian images across different cameras and scenes. This technology has wide-ranging applications in public safety, intelligent security, smart retail, and human-computer interaction.

[0003] Traditional pedestrian re-identification methods primarily rely on pedestrian appearance information, especially clothing features, for identification, and typically assume that the same pedestrian maintains the same clothing under different cameras. However, in real-world applications, pedestrians frequently change clothing, causing significant changes in appearance-based identification cues, leading to a substantial performance degradation of traditional methods. To address this issue, researchers have proposed a clothing-changing pedestrian re-identification task. Its core objective is to mine stable identity features unrelated to clothing, such as facial information or human anatomy information, to improve the robustness of the model in clothing-changing scenarios.

[0004] In CC-ReID tasks, existing methods typically employ multi-branch structures (e.g., face and body branches) to extract identity-discriminating features from multiple sources. However, these methods still face the following key challenges in practical applications: First, due to factors such as occlusion, low resolution, and pose variations, face regions often suffer from blurring, missing features, or invisibility, leading to strong instability in their feature representation and making it difficult to consistently provide reliable identity-discriminating information. Second, low-quality face features may introduce noise during the fusion process, interfering with the originally relatively stable body feature representation and reducing the overall discriminative power. Third, because face and body regions differ significantly in spatial scale, structural composition, and semantic hierarchy, features from different sources often exhibit structural inconsistencies, making it difficult for simple feature fusion methods to effectively model their intrinsic relationships.

[0005] To address the aforementioned issues, two main types of feature fusion methods integrate facial and body information: The first type directly uses the feature quality scores of each part of the query image to guide the feature weighting of the library images. This approach over-relies on the information in the query image, ignoring the reliability differences in the features of the library images themselves, potentially leading to the library features failing to effectively represent identity. The second type independently evaluates the feature quality of the query image and the library images, and then performs weighted fusion accordingly. However, this type of method typically calculates similarity directly at the final feature vector level, failing to fully consider the inconsistencies in the sources of discrimination clues between different images. When the query image primarily relies on facial features while the library images primarily rely on body features, direct feature matching can easily produce significant matching biases, thus affecting the accuracy of the recognition results.

[0006] Furthermore, most existing feature quality assessment methods rely on low-level visual cues (such as image sharpness and occlusion ratio) or confidence scores obtained through supervised learning. These methods have two limitations: firstly, there is a lack of direct semantic correspondence between low-level visual statistical information and identity discrimination ability, making it difficult to accurately reflect the actual discriminative value of features; secondly, quality assessment models based on data-driven learning typically depend on specific data distributions, have limited generalization ability, and their performance tends to degrade when applied across scenarios or datasets. In contrast, large-scale pre-trained visual language models (such as CLIP) can provide a semantically consistent representation space, offering new possibilities for characterizing feature discriminability.

[0007] In summary, existing technologies still struggle to simultaneously achieve effective assessment of feature reliability, suppression of low-quality information, and reasonable modeling of the correlation between features from different sources in pedestrian re-identification tasks involving changes in clothing. Consequently, the recognition accuracy and robustness of the model in complex real-world scenarios still need improvement. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention provides an adaptive fusion pedestrian re-identification method, apparatus, device, and medium to solve the problem that existing pedestrian re-identification technologies for people in different outfits have difficulty in simultaneously achieving effective evaluation of feature reliability, suppression of low-quality information, and reasonable modeling of the correlation between features from different sources, resulting in the need to improve the recognition accuracy and robustness of the model in complex real-world scenarios.

[0009] To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides an adaptive fusion method for pedestrian re-identification, comprising the following steps: S1. Input a pedestrian image, construct a heterogeneous dual-branch backbone network including a face branch and a global branch, and use the face branch to extract the set of local visual features of the pedestrian image. Global branching is used to extract a set of full-body discriminative features from pedestrian images that are independent of clothing. ,in, Represents the real number field. and These represent the number of local visual features of the face and the number of discriminative features of the whole body, respectively. Indicates feature dimension; S2. Construct a positive semantic description set and a negative semantic description set. Utilize the text encoder of a pre-trained visual language model to encode the positive and negative semantic description sets into semantic embedding representations, which serve as positive and negative sample semantic anchors, respectively. Calculate the similarity between the projected local visual features of the face and the positive and negative sample semantic anchors, and then employ a method based on... The semantic confidence score of local visual features of the face is obtained from the probability form. ; S3, based on a set of local visual features of the face and whole-body discriminant feature set The cosine distance between each local feature is used as the transmission cost matrix. And construct an entropy-regularized optimal transmission model, and determine the entropy regularization strength. Based on semantic confidence score Perform adaptive modulation to meet ,in, Using preset parameters, the optimal transmission plan is obtained by solving the optimal transmission model. To introduce marginal constraints and entropy regularization constraints during feature fusion; S4. According to the optimal transmission plan The local visual features of the face are weighted and aggregated, and then fused into the whole body feature space by residual injection, thereby generating a fused re-identification feature representation.

[0010] Secondly, the present invention provides an adaptive fusion pedestrian re-identification device, comprising: The feature extraction module is used to input pedestrian images and construct a heterogeneous dual-branch backbone network including a face branch and a global branch. The face branch is used to extract a set of local visual features of the pedestrian image. Global branching is used to extract a set of full-body discriminative features from pedestrian images that are independent of clothing. ,in, Represents the real number field. and These represent the number of local visual features of the face and the number of discriminative features of the whole body, respectively. Indicates feature dimension; The confidence evaluation module constructs positive and negative semantic description sets. It uses the text encoder of a pre-trained visual language model to encode these sets into semantic embeddings, which serve as semantic anchors for positive and negative samples, respectively. The module calculates the similarity between the projected local visual features of the face and the positive and negative semantic anchors, and then employs a confidence-based approach. The semantic confidence score of local visual features of the face is obtained from the probability form. ; The optimal transmission fusion module is used to extract local visual features of the face. and whole-body discriminant feature set The cosine distance between each local feature is used as the transmission cost matrix. And construct an entropy-regularized optimal transmission model, and determine the entropy regularization strength. Based on semantic confidence score Perform adaptive modulation to meet ,in, Using preset parameters, the optimal transmission plan is obtained by solving the optimal transmission model. To introduce marginal constraints and entropy regularization constraints during feature fusion; The re-identification output module is used to determine the optimal transmission plan. The local visual features of the face are weighted and aggregated, and then fused into the whole body feature space by residual injection, thereby generating a fused re-identification feature representation.

[0011] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more computer programs, which, when executed by the one or more processors, cause the one or more processors to implement the method described in the first aspect.

[0012] Fourthly, a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the method described in the first aspect.

[0013] One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages: calculating the semantic confidence score of local visual features of a face by utilizing a visual language prior semantic confidence evaluation module. and with The entropy regularization intensity ε (ε=λ / q) of the optimal transmission model is directly modulated, making the fusion tend to be uniform at low confidence and precise alignment at high confidence. Finally, the optimal transmission fusion module with confidence modulation achieves adaptive distribution-level fusion of features from multiple sources, thereby enabling more stable and robust identity feature representation in the dressing scene, improving the accuracy of pedestrian re-identification and the generalization ability across scenes. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the execution process of an adaptive fusion pedestrian re-identification method according to the present invention. Figure 2 This is a schematic diagram of the structure of an adaptive fusion pedestrian re-identification device according to the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device according to the present invention; Figure 4 This is a schematic diagram of the structure of a computer-readable storage medium according to the present invention; Figure 5 This is a schematic diagram of the structure of the Visual Language Prior Semantic Confidence Evaluation Module (VL-SCE) involved in this invention. The diagram shows the specific process of encoding positive and negative semantic prompts to form semantic anchors by freezing the visual language model text encoder, projecting local visual features of the face onto the shared semantic space, and calculating the similarity with the semantic anchors to obtain the semantic confidence score. This reflects the facial feature reliability evaluation mechanism based on semantic space probability modeling. Figure 6 This is a schematic diagram of the confidence-modulated optimal transmission fusion module (QM-OTF) involved in this invention. The diagram illustrates the mechanism of introducing semantic confidence scores to modulate the entropy regularization term during the optimal transmission modeling process, and compares the changes in the distribution of transmission plans under different confidence levels: under low confidence, the transmission tends to be uniformly distributed to suppress noise interference; under high confidence, the transmission tends to be sparsely aligned to enhance effective feature matching, thus illustrating the adaptive regulation effect of confidence on feature fusion behavior. Detailed Implementation

[0015] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] This invention comprises three core functional modules: a heterogeneous dual-branch feature extraction backbone network, a vision-language semantic confidence estimator (VL-SCE), and a confidence-modulated optimal transport fusion module (QM-OTF). Figure 5 and Figure 6 As shown, the core functional modules work together to achieve adaptive modeling and fusion of features from multiple sources. The core technical path of this invention is as follows: first, the semantic confidence score of local visual features of the face is calculated through the visual language prior semantic confidence evaluation module. ; then, with The entropy regularization intensity ε (ε=λ / q) of the optimal transmission model is directly modulated to make the fusion tend to be uniform at low confidence and to be precisely aligned at high confidence. Finally, the optimal transmission fusion module modulates the confidence to achieve adaptive distribution-level fusion of features from multiple sources, thereby enabling more stable and robust identity feature representation in the dressing scene, improving the accuracy of pedestrian re-identification and the generalization ability across scenes.

[0017] Example 1 like Figure 1 , Figure 5 and Figure 6 As shown, this embodiment provides an adaptive fusion method for pedestrian re-identification, which includes the following steps: Step S1: In the pedestrian re-identification task after changing clothes, input the pedestrian image. A heterogeneous dual-branch backbone network, including a face branch and a global branch, is constructed, and the face branch is used to extract pedestrian images. Set of local visual features of the face Pedestrian images are extracted using a global branch and a masked body generator. The set of full-body discriminative features unrelated to clothing ,in, Represents the real number field. and These represent the number of local visual features of the face and the number of discriminative features of the whole body, respectively. Indicates the feature dimension.

[0018] In specific implementation, step S1 can be performed in the task of re-identifying pedestrians in disguise, where the training dataset is set to... ,in, This represents the total number of training samples. Indicates the input number of the first... Zhang pedestrian images (i.e., training samples). This represents the identity label corresponding to the training sample. Meanwhile, for any input pedestrian image... Both methods extract local visual features of the face and whole-body discrimination features through a heterogeneous dual-branch backbone network to construct multi-source feature representations.

[0019] In some embodiments of the present invention, the face branch may employ a high-resolution, shallow receptive field design to enhance the ability to depict key local details; the global branch focuses on global structural modeling and combines a masked body generation mechanism to suppress clothing areas, thereby improving robustness to cross-clothing changes.

[0020] This invention employs a heterogeneous dual-branch backbone network to jointly represent pedestrian identities from both fine-grained local information and global structural information levels, thereby providing a multi-source information foundation for subsequent fusion.

[0021] In the training process of the heterogeneous dual-branch backbone network in step S1, this embodiment employs a multi-task joint optimization objective function to perform end-to-end training on the heterogeneous dual-branch backbone network. Multi-task joint optimization objective function It is a weighted combination of identity recognition loss, semantic confidence-based constraint term, and optimal transmission regularization term, and its expression is: ; in, and These are the weighting parameters used to balance the various loss terms; the specific loss terms include: 1. Loss of identity recognition ( ): Used to constrain the discriminative power of fused features, its form is as follows ,in, For cross-entropy loss based on category labels, This is the triplet loss based on the distance constraint between samples.

[0022] 2. Constraints based on semantic confidence ( To enhance the consistency between local facial visual features and the semantic embedding space, an auxiliary loss is introduced. ,in, This is the semantic confidence score output by the Visual Language Prior Semantic Confidence Evaluation (VL-SCE) module. This item is used to constrain the rationality of the semantic confidence score, so that it can reflect the effectiveness of local visual features of the face and improve the stability of the system.

[0023] 3. Optimal transmission regularization term ( ): To constrain the optimal transmission plan generated during the optimal transmission process. To ensure consistency with the characteristic distribution and stabilize the transmission process, a transmission cost matrix-based approach was introduced. constraint terms .

[0024] The above loss terms are used to update the network parameters in a unified manner through the backpropagation mechanism, thereby achieving collaborative optimization of multiple modules.

[0025] In step S1, to address potential face detection failures in the input data, this embodiment employs an automatic suppression mechanism: for the input pedestrian image (i.e., sample), if a face detection failure occurs, the corresponding local facial visual features are suppressed. Set to zero, and simultaneously set the semantic confidence score to zero. Set to the preset minimum positive value ( According to optimal transport theory, the entropy regularization intensity at this point is... It will tend to be maximized, thus making the optimal transmission plan The distribution approaches uniformity. By setting this automatic suppression mechanism, the system can automatically suppress the contribution of the face branch during feature fusion and simultaneously reduce the weight of the semantic alignment loss term, thereby effectively improving the robustness of the system in complex scenarios (such as back view shooting and occlusion).

[0026] In the inference phase of the heterogeneous dual-branch backbone network, the process is consistent with that in the training phase: (1) extract local visual features of the face and full-body discriminative features respectively; (2) input the features into the confidence modulation optimal transmission fusion module (QM-OTF) to obtain the fused feature representation, wherein the confidence modulation mechanism is automatically adjusted according to the actual situation of the input image; (3) construct the fused features into a retrieval descriptor for pedestrian matching tasks in cross-camera scenarios. If the confidence modulation optimal transmission fusion module is unavailable in a specific implementation scenario, the output features of the face branch and the full-body branch are dimensionally concatenated as an alternative retrieval representation.

[0027] Step S2: Construct a set of positive semantic descriptions for representing high-quality faces. With a set of negative semantic descriptions used to represent low-quality faces Text encoders utilizing visual language pre-trained models (such as the CLIP model) The positive and negative semantic description sets are encoded into semantic embeddings, which are then used as semantic anchors for positive and negative samples, respectively. This constructs two types of semantic cues as reference anchors for confidence evaluation. The text encoder... The semantic embedding representation obtained by encoding is as follows: ; Among them, the superscript " " indicates a positive guiding meaning, indicated by the superscript " "Indicates negative interference semantics; The similarity between the projected local visual features of the face and the semantic anchors of positive and negative samples is calculated, and a method based on... The semantic confidence score of local visual features of the face is obtained from the probability form. Specifically, it includes: Local visual feature vectors of the face Mapping to a shared embedding space of visual language via a learnable linear projection layer: ; The mapping dimension is consistent with the embedding space of the visual language pre-trained model. The dimension of the local visual feature vector of a face. These are learnable mapping parameters; Calculate the mapped features semantic anchors of positive samples and negative sample semantic anchors Cosine similarity between them; based on The function calculates the semantic confidence score of local visual features of a face:

[0028] in, For temperature parameters, Represents the cosine similarity function; Semantic confidence score It is used to characterize the reliability of local visual features of the face and as a modulating factor in the subsequent feature fusion process to achieve adaptive control of the contribution of features from different sources.

[0029] To address the issue of unstable reliability of facial features in complex scenarios, this invention introduces a semantic confidence assessment mechanism based on a visual language pre-trained model. By leveraging the semantic expressive capabilities of this model, the confidence assessment of facial features is modeled as a probabilistic inference problem in the semantic space, thus avoiding reliance on low-level visual statistics or specific supervision signals from the dataset. Through this approach, this invention achieves semantic-level confidence assessment without the need for additional manual quality labels, enabling the obtained confidence scores to more accurately reflect the feature's identity discrimination ability and possessing good cross-scenario generalization capabilities.

[0030] Step S3: First, model the features as a distribution and construct the transmission cost matrix. : Set of local visual features of the face With the whole body discriminant feature set Each feature is modeled as a distribution defined in the feature space, and the fusion process is modeled as an information transfer process between different feature distributions; specifically, the above two types of features are represented as empirical distributions: ; in, and Both represent the marginal weight distribution; For feature index; and They represent the first Individual facial local visual features and the first A whole-body discriminative feature, For Dirac measurement; Set of local visual features of the face and whole-body discriminant feature set The cosine distance between each local feature is used to construct a transmission cost matrix based on feature similarity. :

[0031] in, This is the transpose symbol (i.e., the transpose symbol for a matrix / vector). It is the L2 norm (Euclidean norm) of the vector. and They represent the first Individual facial local visual features and the first A whole-body discriminative feature.

[0032] Secondly, an entropy-regularized optimal transmission model is constructed, and adaptive modulation is performed: Under the condition of satisfying marginal constraints To enable the fusion process to adapt to feature reliability, this invention constructs an entropy-regularized optimal transmission model, with boundary constraints satisfying the following equation: ; in, It is a vector of all 1s. Used to describe how information is distributed among different features. For the transmission plan matrix; Entropy regularization intensity in the optimal transmission model Based on semantic confidence score Perform adaptive modulation to meet ,in, Given preset parameters, this adaptive modulation mechanism is introduced into the optimal transmission model, and the following optimization problem is constructed:

[0033] Here, arg min represents the parameters that minimize the objective function. Denotes the Frobenius inner product of matrices; in:

[0034] The entropy represents the transmission plan. For the transmission plan matrix The Line number Column elements.

[0035] Through the aforementioned adaptive modulation mechanism, the fusion process can dynamically adjust the information allocation method according to the quality of local visual features of the face, thereby reducing the impact of unreliable features on the fusion result to a certain extent and enhancing the contribution of high-quality features.

[0036] Finally, the optimal transmission model is solved to obtain the optimal transmission plan. : The above-mentioned optimal transport plan incorporates marginal constraints and entropy regularization constraints. pass Iterative algorithms are used to solve the problem, specifically including: remember The uniform marginal weight distribution of the set of local visual features of a face. The uniform marginal weight distribution of the whole-body discriminative feature set, i.e. , An index representing local visual features of a human face. An index representing the whole-body discriminative features; Define kernel matrix ,in, The cost matrix uses cosine distance as the transmission cost; the scaling vector is initialized. and Iterative execution: ; in, This represents the element-wise division operator; Until convergence, we obtain:

[0037] in, This is a diagonal matrix function used to convert a vector into a diagonal matrix.

[0038] Through the above mechanism, the present invention can adaptively model the correlation between features from multiple sources without relying on explicit structural correspondences.

[0039] Step S4: Obtain the optimal transmission plan Then, according to the optimal transmission plan Weighted aggregation of local facial visual features is performed and then fused into the whole-body feature space using residual injection, thereby generating a fused re-identification feature representation. This method maps local facial information to the whole-body feature space, thus achieving information supplementation. Specifically, the weighted aggregation process is represented as follows: ; in, For the transmission plan matrix, A collection of local visual features of a face. For the transmission plan matrix The Line number Column elements, Indicates the first Local visual features of an individual's face.

[0040] In step S4, the fused re-identification features are specifically represented as follows:

[0041] in, These are learnable parameters used to control the injection intensity of local visual features of the face; It is a set of features for whole-body discrimination. A collection of local visual features of a face. For the transmission plan matrix, It is the transpose symbol; This invention enhances identity recognition capabilities while maintaining stability by introducing local facial visual features as supplementary information, while preserving the whole-body discrimination features as the primary representation.

[0042] Furthermore, in specific implementations of this invention, if either the local visual features of the face or the whole-body discrimination features are missing, the other feature that is not missing will be used as the global feature.

[0043] Based on the same inventive concept, this application also provides an apparatus corresponding to the method in Embodiment 1, as detailed in Embodiment 2.

[0044] Example 2 like Figure 2 , Figure 5 and Figure 6 As shown, this embodiment provides an adaptive fusion pedestrian re-identification device, the device comprising: The feature extraction module is used to input pedestrian images in the pedestrian re-identification task involving people in different outfits. A heterogeneous dual-branch backbone network, including a face branch and a global branch, is constructed, and the face branch is used to extract pedestrian images. Set of local visual features of the face Pedestrian images are extracted using a global branch and a masked body generator. The set of full-body discriminative features unrelated to clothing ,in, Represents the real number field. and These represent the number of local visual features of the face and the number of discriminative features of the whole body, respectively. Indicates the feature dimension.

[0045] In practical implementation, the feature extraction module can be used in the task of re-identifying pedestrians in disguise, where the training dataset is... ,in, This represents the total number of training samples. Indicates the input number of the first... Zhang pedestrian images (i.e., training samples). This represents the identity label corresponding to the training sample. Meanwhile, for any input pedestrian image... Both methods extract local visual features of the face and whole-body discrimination features through a heterogeneous dual-branch backbone network to construct multi-source feature representations.

[0046] In some embodiments of the present invention, the face branch may employ a high-resolution, shallow receptive field design to enhance the ability to depict key local details; the global branch focuses on global structural modeling and combines a masked body generation mechanism to suppress clothing areas, thereby improving robustness to cross-clothing changes.

[0047] This invention employs a heterogeneous dual-branch backbone network to jointly represent pedestrian identities from both fine-grained local information and global structural information levels, thereby providing a multi-source information foundation for subsequent fusion.

[0048] In the training process of the heterogeneous dual-branch backbone network of the feature extraction module, this embodiment employs a multi-task joint optimization objective function to perform end-to-end training of the heterogeneous dual-branch backbone network. Multi-task joint optimization objective function It is a weighted combination of identity recognition loss, semantic confidence-based constraint term, and optimal transmission regularization term, and its expression is: ; in, and These are the weighting parameters used to balance the various loss terms; the specific loss terms include: 1. Loss of identity recognition ( ): Used to constrain the discriminative power of fused features, its form is as follows ,in, For cross-entropy loss based on category labels, This is the triplet loss based on the distance constraint between samples.

[0049] 2. Constraints based on semantic confidence ( To enhance the consistency between local facial visual features and the semantic embedding space, an auxiliary loss is introduced. ,in, This is the semantic confidence score output by the Visual Language Prior Semantic Confidence Evaluation (VL-SCE) module. This item is used to constrain the rationality of the semantic confidence score, so that it can reflect the effectiveness of local visual features of the face and improve the stability of the system.

[0050] 3. Optimal transmission regularization term ( ): To constrain the optimal transmission plan generated during the optimal transmission process. To ensure consistency with the characteristic distribution and stabilize the transmission process, a transmission cost matrix-based approach was introduced. constraint terms .

[0051] The above loss terms are used to update the network parameters in a unified manner through the backpropagation mechanism, thereby achieving collaborative optimization of multiple modules.

[0052] In the feature extraction module, to address potential face detection failures in the input data, this embodiment designs an automatic suppression mechanism: for the input pedestrian image (i.e., sample), if a face detection failure occurs, the corresponding local visual features of the face are suppressed. Set to zero, and simultaneously set the semantic confidence score to zero. Set to the preset minimum positive value ( According to optimal transport theory, the entropy regularization intensity at this point is... It will tend to be maximized, thus making the optimal transmission plan The distribution approaches uniformity. By setting this automatic suppression mechanism, the system can automatically suppress the contribution of the face branch during feature fusion and simultaneously reduce the weight of the semantic alignment loss term, thereby effectively improving the robustness of the system in complex scenarios (such as back view shooting and occlusion).

[0053] In the inference phase of the heterogeneous dual-branch backbone network, the process is consistent with that in the training phase: (1) extract local visual features of the face and full-body discriminative features respectively; (2) input the features into the confidence modulation optimal transmission fusion module (QM-OTF) to obtain the fused feature representation, wherein the confidence modulation mechanism is automatically adjusted according to the actual situation of the input image; (3) construct the fused features into a retrieval descriptor for pedestrian matching tasks in cross-camera scenarios. If the confidence modulation optimal transmission fusion module is unavailable in a specific implementation scenario, the output features of the face branch and the full-body branch are dimensionally concatenated as an alternative retrieval representation.

[0054] The confidence evaluation module is used to construct a set of positive semantic descriptions for representing high-quality faces. With a set of negative semantic descriptions used to represent low-quality faces Text encoders utilizing visual language pre-trained models (such as the CLIP model) The positive and negative semantic description sets are encoded into semantic embeddings, which are then used as semantic anchors for positive and negative samples, respectively. This constructs two types of semantic cues as reference anchors for confidence evaluation. The text encoder... The semantic embedding representation obtained by encoding is as follows: ; Among them, the superscript " " indicates a positive guiding meaning, indicated by the superscript " "Indicates negative interference semantics; The similarity between the projected local visual features of the face and the semantic anchors of positive and negative samples is calculated, and a method based on... The semantic confidence score of local visual features of the face is obtained from the probability form. Specifically, it includes: Local visual feature vectors of the face Mapping to a shared embedding space of visual language via a learnable linear projection layer: ; The mapping dimension is consistent with the embedding space of the visual language pre-trained model. The dimension of the local visual feature vector of a face. These are learnable mapping parameters; Calculate the mapped features semantic anchors of positive samples and negative sample semantic anchors Cosine similarity between them; based on The function calculates the semantic confidence score of local visual features of a face:

[0055] in, For temperature parameters, Represents the cosine similarity function; Semantic confidence score It is used to characterize the reliability of local visual features of the face and as a modulating factor in the subsequent feature fusion process to achieve adaptive control of the contribution of features from different sources.

[0056] To address the issue of unstable reliability of facial features in complex scenarios, this invention introduces a semantic confidence assessment mechanism based on a visual language pre-trained model. By leveraging the semantic expressive capabilities of this model, the confidence assessment of facial features is modeled as a probabilistic inference problem in the semantic space, thus avoiding reliance on low-level visual statistics or specific supervision signals from the dataset. Through this approach, this invention achieves semantic-level confidence assessment without the need for additional manual quality labels, enabling the obtained confidence scores to more accurately reflect the feature's identity discrimination ability and possessing good cross-scenario generalization capabilities.

[0057] The optimal transmission fusion module is used to first model the features as a distribution and construct the transmission cost matrix. : Set of local visual features of the face With the whole body discriminant feature set Each feature is modeled as a distribution defined in the feature space, and the fusion process is modeled as an information transfer process between different feature distributions; specifically, the above two types of features are represented as empirical distributions: ; in, and Both represent the marginal weight distribution; For feature index; and They represent the first Individual facial local visual features and the first A whole-body discriminative feature, For Dirac measurement; Set of local visual features of the face and whole-body discriminant feature set The cosine distance between each local feature is used to construct a transmission cost matrix based on feature similarity. :

[0058] in, This is the transpose symbol (i.e., the transpose symbol for a matrix / vector). It is the L2 norm (Euclidean norm) of the vector. and They represent the first Individual facial local visual features and the first A whole-body discriminative feature.

[0059] Secondly, an entropy-regularized optimal transmission model is constructed, and adaptive modulation is performed: Under the condition of satisfying marginal constraints To enable the fusion process to adapt to feature reliability, this invention constructs an entropy-regularized optimal transmission model, with boundary constraints satisfying the following equation: ; in, It is a vector of all 1s. Used to describe how information is distributed among different features. For the transmission plan matrix; Entropy regularization intensity in the optimal transmission model Based on semantic confidence score Perform adaptive modulation to meet ,in, Given preset parameters, this adaptive modulation mechanism is introduced into the optimal transmission model, and the following optimization problem is constructed:

[0060] Here, arg min represents the parameters that minimize the objective function. Denotes the Frobenius inner product of matrices; in:

[0061] The entropy represents the transmission plan. For the transmission plan matrix The Line number Column elements.

[0062] Through the aforementioned adaptive modulation mechanism, the fusion process can dynamically adjust the information allocation method according to the quality of local visual features of the face, thereby reducing the impact of unreliable features on the fusion result to a certain extent and enhancing the contribution of high-quality features.

[0063] Finally, the optimal transmission model is solved to obtain the optimal transmission plan. : The above-mentioned optimal transport plan incorporates marginal constraints and entropy regularization constraints. pass Iterative algorithms are used to solve the problem, specifically including: remember The uniform marginal weight distribution of the set of local visual features of a face. The uniform marginal weight distribution of the whole-body discriminative feature set, i.e. , An index representing local visual features of a human face. An index representing the whole-body discriminative features; Define kernel matrix ,in, The cost matrix uses cosine distance as the transmission cost; the scaling vector is initialized. and Iterative execution: ; in, This represents the element-wise division operator; Until convergence, we obtain:

[0064] in, This is a diagonal matrix function used to convert a vector into a diagonal matrix.

[0065] Through the above mechanism, the present invention can adaptively model the correlation between features from multiple sources without relying on explicit structural correspondences.

[0066] The re-identification output module is used to obtain the optimal transmission plan. Then, according to the optimal transmission plan Weighted aggregation of local facial visual features is performed and then fused into the whole-body feature space using residual injection, thereby generating a fused re-identification feature representation. This method maps local facial information to the whole-body feature space, thus achieving information supplementation. Specifically, the weighted aggregation process is represented as follows: ; in, For the transmission plan matrix, A collection of local visual features of a face. For the transmission plan matrix The Line number Column elements, Indicates the first Local visual features of an individual's face.

[0067] In the re-identification output module, the fused re-identification features are specifically represented as follows:

[0068] in, These are learnable parameters used to control the injection intensity of local visual features of the face; It is a set of features for whole-body discrimination. A collection of local visual features of a face. For the transmission plan matrix, It is the transpose symbol; This invention enhances identity recognition capabilities while maintaining stability by introducing local facial visual features as supplementary information, while preserving the whole-body discrimination features as the primary representation.

[0069] Furthermore, in specific implementations of this invention, if either the local visual features of the face or the whole-body discrimination features are missing, the other feature that is not missing will be used as the global feature.

[0070] Based on the same inventive concept, this application provides an electronic device embodiment corresponding to Embodiment 1, as detailed in Embodiment 3.

[0071] Example 3 This embodiment provides an electronic device, such as... Figure 3 As shown, it includes one or more processors; and a storage device for storing one or more computer programs, which, when executed by the one or more processors, enable the one or more processors to implement any of the embodiments in Example 1.

[0072] Since the electronic device described in this embodiment is the device used to implement the method in Embodiment 1 of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in Embodiment 1 of this application. Therefore, how the electronic device implements the method in the embodiment of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiment of this application falls within the scope of protection of this application.

[0073] Based on the same inventive concept, this application provides a storage medium corresponding to Embodiment 1, as detailed in Embodiment 4.

[0074] Example 4 This embodiment provides a computer-readable storage medium, such as... Figure 4 As shown, a computer program is stored thereon, which, when executed by a processor, can implement any of the embodiments in Example 1.

[0075] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An adaptive fusion method for pedestrian re-identification, characterized in that, Includes the following steps: S1. Input a pedestrian image, construct a heterogeneous dual-branch backbone network including a face branch and a global branch, and use the face branch to extract the set of local visual features of the pedestrian image. Global branching is used to extract a set of full-body discriminative features from pedestrian images that are independent of clothing. ,in, Represents the real number field. and These represent the number of local visual features of the face and the number of discriminative features of the whole body, respectively. Indicates feature dimension; S2. Construct a positive semantic description set and a negative semantic description set. Utilize the text encoder of a pre-trained visual language model to encode the positive and negative semantic description sets into semantic embedding representations, which serve as positive and negative sample semantic anchors, respectively. Calculate the similarity between the projected local visual features of the face and the positive and negative sample semantic anchors, and then employ a method based on... The semantic confidence score of local visual features of the face is obtained from the probability form. ; S3, based on a set of local visual features of the face and whole-body discriminant feature set The cosine distance between each local feature is used as the transmission cost matrix. And construct an entropy-regularized optimal transmission model, and determine the entropy regularization strength. Based on semantic confidence score Perform adaptive modulation to meet ,in, Using preset parameters, the optimal transmission plan is obtained by solving the optimal transmission model. To introduce marginal constraints and entropy regularization constraints during feature fusion; S4. According to the optimal transmission plan The local visual features of the face are weighted and aggregated, and then fused into the whole body feature space by residual injection, thereby generating a fused re-identification feature representation.

2. The adaptive fusion pedestrian re-identification method according to claim 1, characterized in that, In step S2, the similarity between the projected local visual features of the face and the semantic anchors of positive and negative samples is calculated, and a method based on... The semantic confidence score of local visual features of the face is obtained from the probability form. Specifically: Local visual feature vectors of the face Mapping to a shared embedding space of visual language via a learnable linear projection layer: ； in, The dimension of the local visual feature vector of a face. These are learnable mapping parameters; Calculate the mapped features semantic anchors of positive samples and negative sample semantic anchors Cosine similarity between them; based on The function calculates the semantic confidence score of local visual features of a face: in, For temperature parameters, This represents the cosine similarity function.

3. The adaptive fusion pedestrian re-identification method according to claim 1, characterized in that, In step S3, the set of local visual features of the face and whole-body discriminant feature set The cosine distance between each local feature is used as the transmission cost matrix. Specifically: Set of local visual features of the face With the whole body discriminant feature set Each feature is modeled as a distribution defined in the feature space, and the fusion process is modeled as an information transfer process between different feature distributions. Specifically, the two types of features are represented as empirical distributions: ； in, and Both represent the marginal weight distribution; For feature index; and They represent the first Individual facial local visual features and the first A whole-body discriminative feature, For Dirac measure; Set of local visual features of the face and whole-body discriminant feature set The cosine distance between each local feature is used to construct a transmission cost matrix based on feature similarity. : in, It is the transpose symbol. Let L2 be the norm of the vector.

4. The adaptive fusion pedestrian re-identification method according to claim 1, characterized in that, In step S3, the optimal transmission plan pass Iterative algorithms are used to solve the problem, specifically including: remember The uniform marginal weight distribution of the set of local visual features of a face. The uniform marginal weight distribution of the whole-body discriminative feature set is, i.e. , An index representing local visual features of a human face. An index representing the whole-body discriminative features; Define kernel matrix ,in, The cost matrix uses cosine distance as the transmission cost; the scaling vector is initialized. and Iterative execution: ； in, This represents the element-wise division operator; Until convergence, we obtain: in, This is a diagonal matrix function used to convert a vector into a diagonal matrix.

5. The adaptive fusion pedestrian re-identification method according to claim 1, characterized in that, In step S1, the heterogeneous dual-branch backbone network is trained using a multi-task joint optimization objective function, which is a weighted combination of identity recognition loss, semantic confidence-based constraint terms, and optimal transmission regularization terms.

6. The adaptive fusion pedestrian re-identification method according to claim 1, characterized in that, In step S1, if face detection fails in the input pedestrian image, the corresponding local visual features of the face are set to zero, and the semantic confidence score is set to a preset minimum positive value. , making the entropy regularization intensity It tends to be extremely large.

7. The adaptive fusion pedestrian re-identification method according to claim 1, characterized in that, In step S4, the fused re-identification features are specifically represented as follows: in, These are learnable parameters used to control the injection intensity of local visual features of the face; It is a set of features for whole-body discrimination. A collection of local visual features of a face. For the transmission plan matrix, It is the transpose symbol; Furthermore, if either the local visual features of the face or the whole-body discriminative features are missing, the other feature that is not missing will be used as the global feature.

8. An adaptive fusion pedestrian re-identification device, characterized in that, include: The feature extraction module is used to input pedestrian images and construct a heterogeneous dual-branch backbone network including a face branch and a global branch. The face branch is used to extract a set of local visual features of the pedestrian image. Global branching is used to extract a set of full-body discriminative features from pedestrian images that are independent of clothing. ,in, Represents the real number field. and These represent the number of local visual features of the face and the number of discriminative features of the whole body, respectively. Indicates feature dimension; The confidence evaluation module constructs positive and negative semantic description sets. It uses the text encoder of a pre-trained visual language model to encode these sets into semantic embeddings, which serve as semantic anchors for positive and negative samples, respectively. The module calculates the similarity between the projected local visual features of the face and the positive and negative semantic anchors, and then employs a confidence-based approach. The semantic confidence score of local visual features of the face is obtained from the probability form. ; The optimal transmission fusion module is used to extract local visual features of the face. and whole-body discriminant feature set The cosine distance between each local feature is used as the transmission cost matrix. And construct an entropy-regularized optimal transmission model, and determine the entropy regularization strength. Based on semantic confidence score Perform adaptive modulation to meet ,in, Using preset parameters, the optimal transmission plan is obtained by solving the optimal transmission model. To introduce marginal constraints and entropy regularization constraints during feature fusion; The re-identification output module is used to determine the optimal transmission plan. The local visual features of the face are weighted and aggregated, and then fused into the whole body feature space by residual injection, thereby generating a fused re-identification feature representation.

9. An electronic device comprising one or more processors; and a storage device for storing one or more computer programs that, when executed by the one or more processors, cause the one or more processors to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images