A Multimodal, Multi-Platform Person Re-identification Method Based on Canonical Domain Guidance

Abstract

This invention discloses a multimodal, multi-platform person re-identification method based on canonical domain guidance. The method includes: acquiring a target image and determining whether the target image originates from a canonical domain or a restricted domain; using the target image as an input image, obtaining corresponding feature vectors using a trained unified backbone and differential routing network; and obtaining a person re-identification result based on the feature vectors. The unified backbone and differential routing network is constructed based on a Transformer model and contains multiple Transformer blocks. Each Transformer block has a convolutional adapter, which is configured to propagate canonical domain features directly along the backbone for the canonical domain and to superimpose calibration increments on restricted domain features beyond the backbone output for the restricted domain. This invention improves the accuracy and robustness of person re-identification in complex multimodal, multi-platform environments.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a multimodal, multi-platform pedestrian re-identification method based on canonical domain guidance. Background Technology

[0002] Pedestrian re-identification technology, as an important research direction in the field of intelligent security, initially focused on application scenarios with a single platform and single imaging modality, such as cross-camera pedestrian identity matching based on ground visible light cameras. With the continuous expansion of practical application needs, monitoring systems have gradually incorporated drone platforms and various imaging devices such as infrared and thermal imaging, leading to complex situations where pedestrian re-identification tasks simultaneously span imaging modalities and acquisition platforms. This has given rise to the new research direction of multimodal, multi-platform pedestrian re-identification. Multimodal, multi-platform pedestrian re-identification tasks involve spectral differences and platform perspective differences, exhibiting significant complexity. Most existing solutions still employ a unified, symmetrical feature learning strategy, assuming that data from different modalities and platforms have equal value in terms of information quality, without effectively modeling the differences in imaging information completeness and discriminative capabilities between different data sources. When images acquired by some modalities or platforms have problems such as low resolution, missing textures, or perspective distortion, existing solutions have limitations in feature alignment and identity discrimination.

[0003] In multimodal, multi-platform pedestrian re-identification scenarios, pedestrian images may originate from ground-based visible light cameras, drone cameras, and infrared or thermal imaging devices. These different data sources exhibit significant differences in imaging conditions, resolution, viewpoint structure, and texture details, greatly increasing the difficulty of pedestrian identification. Furthermore, in these complex scenarios, the data acquired from different modalities and platforms exhibits a clear imbalance in information quality. Typically, ground-based visible light images provide relatively complete and clear appearance texture and structural information, while drone viewpoints or infrared / thermal imaging images, due to limitations in imaging principles, shooting distance, and environmental conditions, often suffer from texture loss, blurred details, and severe viewpoint distortion. This information asymmetry caused by differences in imaging modalities and acquisition platforms leads to inherent gaps in the pedestrian identification capabilities of different data sources.

[0004] Existing multimodal, multi-platform person re-identification methods typically employ a symmetric modeling strategy, treating pedestrian images from different modalities and platforms as equivalent inputs and mapping them to the same feature space through a unified feature extraction and alignment mechanism. However, these methods fail to adequately consider the differences in information richness among various data sources, easily introducing noise interference during feature learning. This causes information-limited data to adversely affect the overall feature space, resulting in discrete feature distributions for the same identity across different modalities and platforms, reducing the accuracy and stability of cross-modal and cross-platform matching.

[0005] Furthermore, in existing technologies, the semantic representation or supervisory information of pedestrian identity is usually constructed by the joint participation of multimodal data. When the proportion of information-restricted modalities is high, it is easy to cause semantic representation deviation, affecting the reliability of identity identification. At the same time, relying on a single supervisory method is difficult to simultaneously ensure the consistency of identity semantics and visual structure, resulting in the difficulty of effectively aligning pedestrian features in complex cross-modal and cross-platform application scenarios.

[0006] Analysis shows that existing multimodal, multi-platform pedestrian re-identification technologies are still in the exploratory stage and generally have the following shortcomings: (1) Existing technologies generally adopt a unified or symmetrical feature learning approach, treating pedestrian images from different imaging modalities and acquisition platforms as equivalent information sources for processing. This fails to effectively distinguish the differences in information integrity and discrimination capabilities between different data sources. In practical applications, when images acquired by some modalities or platforms have problems such as low resolution, missing texture information, or severe viewpoint distortion, this unified modeling approach is easily affected by data with limited information, resulting in scattered feature representations and affecting the accuracy of cross-modal and cross-platform pedestrian identity matching.

[0007] (2) In the process of feature alignment, existing technologies usually lack stable and reliable reference benchmarks. Features of different modalities and platforms often exhibit large distribution shifts in the feature space, making it difficult to form a unified and highly discriminative identity representation. In multimodal and multi-platform scenarios, relying solely on a single form of supervision information makes it difficult to simultaneously ensure the semantic consistency and visual structural consistency of pedestrian identity, which can easily lead to problems such as non-compact intra-class features and insufficient inter-class discriminability.

[0008] (3) Existing solutions for pedestrian re-identification typically involve multimodal data in the construction of semantic representations or supervisory signals when using semantic information or pre-trained models. When the proportion of information-restricted modalities is high, noise interference is easily introduced, causing the semantic representation to deviate from the true identity features, thereby reducing the overall recognition performance. This type of problem is particularly prominent in multimodal and multi-platform mixed scenarios, which restricts the application effect of pedestrian re-identification technology in complex real-world environments.

[0009] In summary, existing technologies lack effective methods for modeling information asymmetry issues in multimodal and multi-platform pedestrian re-identification scenarios. In particular, there is a lack of pedestrian re-identification technology solutions that use information-rich data sources as references to provide targeted guidance and correction for information-limited data, while simultaneously taking into account semantic consistency and visual structural consistency. Summary of the Invention

[0010] The purpose of this invention is to overcome the shortcomings of the prior art and provide a multimodal, multi-platform person re-identification method based on canonical domain guidance. This method includes the following steps: Acquire a target image and determine whether the target image comes from a canonical domain or a restricted domain; Using the target image as the input image, the corresponding feature vector is obtained by utilizing the trained unified backbone and differentiated routing network; Based on the feature vector, the pedestrian re-identification result is obtained; The unified backbone and differentiated routing network is constructed based on the Transformer model and contains multiple Transformer blocks. Each Transformer block is equipped with a convolutional adapter, which is configured to propagate the features of the normalized domain directly along the backbone for the normalized domain and to superimpose the calibration increment on the features of the restricted domain outside the backbone output for the restricted domain.

[0011] Compared with existing technologies, the advantages of this invention lie in its provision of a multimodal, multi-platform pedestrian re-identification method guided by the canonical domain. This method distinguishes between information-rich and information-limited data in multimodal, multi-platform pedestrian re-identification scenarios, introduces information-rich data sources as stable references, and provides targeted guidance and correction for information-limited data. This alleviates the information asymmetry problem caused by differences in imaging modality and acquisition platform. This invention can construct a more unified, compact, and discriminative pedestrian feature representation, improving the accuracy and robustness of pedestrian re-identification in complex multimodal, multi-platform environments.

[0012] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with their description, serve to explain the principles of the invention.

[0014] Figure 1 This is a flowchart of a multimodal, multi-platform pedestrian re-identification method based on canonical domain guidance according to an embodiment of the present invention. Detailed Implementation

[0015] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0016] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0017] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0018] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0019] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0020] In summary, the multimodal, multi-platform pedestrian re-identification method based on canonical domain guidance provided by this invention first distinguishes between information-rich canonical domain data and information-limited data from other modalities or platforms based on the data sources of different pedestrian images. Under a unified feature extraction framework, the canonical domain data retains its original feature expression capabilities, while the information-limited data undergoes targeted feature calibration and alignment. Using the canonical domain data as a reference, a guidance mechanism constrains the distribution of pedestrian features from different modalities and platforms in a unified feature space, making the features of the same identity more consistent across different modalities and platforms. This mitigates the impact of information asymmetry and improves the accuracy and stability of multimodal, multi-platform pedestrian re-identification.

[0021] Specifically, see Figure 1 As shown, the provided multimodal, multi-platform person re-identification method based on canonical domain guidance includes the following steps: Step S1: Obtain multimodal and multi-platform pedestrian image data to construct a training set, and determine the normalized domain data and the restricted domain data.

[0022] The training set reflects the correspondence between pedestrian images, domain identifiers, and identity labels, and includes both canonical and restricted domain data. In this paper, canonical domain data refers to data that is information-rich or of high quality. In contrast, restricted domain data refers to data that is information-limited or of relatively low quality.

[0023] The training set can be formally represented as: (1) in, For pedestrian images, For identity tags, For domain identification, This refers to the number of samples. Domain identifiers can consist of "platform + modality," such as terrestrial RGB, terrestrial IR, UAV RGB, UAV Thermal (thermal imaging), etc.

[0024] normal domain The selection principle is as follows: among the multiple domains involved in the current task, the domain with richer information, more complete texture details, higher resolution, more stable viewpoint, and stronger identity discrimination is selected as the canonical domain, and the remaining domains are selected as the restricted domains. 。 When the task is "paired retrieval / paired evaluation," dynamic canonical domain selection can be employed. For example, for each retrieval pair, the domain with richer information is selected as the reference canonical domain for that alignment, ensuring that the "guiding signal" comes from a higher-quality data source. By distinguishing between information-rich canonical domains and other information-constrained domains (i.e., restricted domains) in multi-source pedestrian data, and using the canonical domain as a feature alignment reference, pedestrian features from different modalities and platforms can be guided to converge towards a unified distribution, effectively alleviating the information asymmetry problem in multimodal and multi-platform scenarios.

[0025] Step S2: Construct a unified backbone and differentiated routing network, using a direct path for inputs in the normalized domain and an adaptive path for inputs in the restricted domain to perform feature calibration.

[0026] In one embodiment, the Unified Backbone and Differentiated Routing (AAUB) network can be constructed based on a Transformer model. For example, the unified visual coding backbone preferably uses a pre-trained visual Transformer (such as an image encoder of a vision-language pre-trained model) as the basic feature extraction network, containing multiple Transformer blocks. To achieve "same backbone, different routes," compared to existing technologies, the Unified Backbone and Differentiated Routing (AAUB) network constructed in this invention adds a restricted domain convolutional adapter (ConvAdapter) to each Transformer block to implement domain-dependent adaptations. When the input sample comes from the normal domain season This allows the features in the normalized domain to propagate directly along the main trunk, preserving as much as possible the discriminative ability and compact distribution of the pre-trained model on high-quality data. When the input samples come from a restricted domain... season This design allows a "calibration increment" to be superimposed on the confined domain features outside the backbone output, thereby compensating for feature shifts in the confined domain caused by spectral degradation, low resolution, or viewpoint distortion. This structural design forms an asymmetric adaptive unified backbone of "unified base + parallel adaptation of confined domain".

[0027] Asymmetric adaptive unified backbone is used to extract visual feature vectors, such as input images. and domain identifier To obtain the output visual feature vector It can be used for subsequent classification, distillation, and metric learning.

[0028] By designing an asymmetric network structure under a unified feature extraction backbone, a direct feature path is used to maintain high discriminative power for inputs in the normal domain, while an adaptive path is introduced for inputs in the restricted domain for targeted calibration. This achieves differentiated processing of data with different information quality while ensuring the overall consistency of the network.

[0029] Specifically, within the constructed unified backbone and differentiated routing network architecture, convolutional adapters are introduced to compensate and calibrate the restricted domain features within the unified backbone, making their representation closer to the distribution of canonical domain features. The restricted domain convolutional adapter (ConvAdapter) is used to introduce local spatial priors to the restricted domain input. For example, the restricted domain convolutional adapter can be constructed using bottleneck convolutional modules, including feature reshaping modules, convolutional layers for dimensionality reduction, ReLU activation layers, depthwise separable convolutional layers, and convolutional layers for dimensionality increase. The execution process of the convolutional adapter includes: reshaping the Transformer's patchtoken sequence into a two-dimensional feature map; and reducing dimensionality through 1×1 convolutions (…). Down ); through 3×3 depth-separable convolution ( DWConv Extracting local details; increasing dimensionality through 1×1 convolution ( Up Multiply by the scaling factor As a calibration increment, it is fused with the backbone output. The computation process of the restricted-domain convolutional adapter can be expressed as: (2) in, These are learnable parameters, initialized to 0. This "zero initialization" ensures that the adapter output is approximately zero in the early stages of training, equivalent to not changing the pre-trained backbone output, thus guaranteeing training stability; as training progresses, By gradually learning the limited domain compensation capability from the adapter parameters, "progressive calibration" is achieved.

[0030] The input to the restricted-domain convolutional adapter is an intermediate feature of a Transformer block, and the output is a calibration increment. This increment is added to the backbone output to obtain the restricted-domain calibrated feature representation. By introducing a restricted-domain feature calibration module into the unified backbone network, a parallel feature compensation method is implemented to calibrate the features of the restricted-domain image, enhancing its local structure and discriminative ability. The restricted-domain convolutional adapter is fused with the backbone features incrementally, achieving progressive compensation and stable training of the restricted-domain features.

[0031] Step S3: In the pre-training phase, text prompts are trained, which are used to obtain canonical semantic anchors in the main training phase.

[0032] During the pre-training phase, text encoders and image encoders are used to train text prompts. The input to the text encoder is the text prompt, and the input to the image encoder is an image sample from the canonical domain. The loss function can be the image-text contrast loss from the CLIP (Contrastive Language-Image Pre-training) model. Only canonical domain samples are used to train the text-side identity semantic prototype during the pre-training phase, which is then used for cross-domain identity consistency constraints in the subsequent main training phase.

[0033] Specifically, a text prompt template is created as input to the text encoder, such as "a photo of [X]1[X]2…[X]". M "person", where [X] is a learnable context token.

[0034] The training of semantic anchors follows an "Anti-Pollution" strategy: only canonical domain samples are used to optimize learnable tokens, ensuring that text embeddings align only with high-quality visual distributions in the canonical domain, thus avoiding confined domain noise pulling the text prototype towards a compromise distribution. After completing this stage, a set of semantic anchor vectors corresponding to each identity is obtained. The values ​​are frozen during the main training phase and used as fixed classifier weights to apply identity supervision to all domain samples.

[0035] In one embodiment, the pre-training process for generating identity-to-semantic anchors uses cross-entropy loss, denoted as: (3) in, As a visual feature, For the semantic anchor point corresponding to the real identity, The temperature coefficient is used. This constraint causes visual features of the same identity in different domains to converge towards the same semantic center, improving cross-domain identity consistency.

[0036] Step S4: In the main training phase, knowledge distillation training is performed using the set total loss function to generate canonical domain visual anchors and perform distillation alignment. The teacher network outputs canonical domain visual anchors, and the student network is constrained to align its restricted domain features with the canonical domain structure.

[0037] The main training phase is primarily used to train the AAUB and convolutional adapter, where the parameters of the pre-trained text encoder are frozen. During the main training phase, knowledge distillation can be used to generate canonical domain visual anchor points (CVAs).

[0038] For example, construct a teacher network T and a student network S. The student network is the AAUB (with restricted domain adapter) of step S2, and the teacher network is a standard Transformer encoder without adapters, used to provide a stable canonical domain visual reference. The teacher network parameters are updated using the exponential moving average (EMA) of the student network backbone parameters: (4) in, This represents the parameters of the backbone component (excluding adapters) that is isomorphic to the teacher's network in the student network. The momentum coefficient, This represents the parameters of the teacher network.

[0039] During distillation alignment, for each restricted domain sample Student network output Simultaneously, select reference samples of the same type in the normative domain. Output of the teacher network As visual anchors, the two are constrained to be close by distillation loss. For example, the visual anchor distillation loss is expressed as: (5) in, To stop gradient operations and ensure that the teacher's network output is only used as the target and does not participate in backpropagation updates, this mechanism pulls the features of the restricted domain back to the normalized manifold at the "structure / topology" level, correcting intra-class structural drift within the restricted domain.

[0040] During the main training phase, end-to-end training is performed using a joint loss function and a training sampling strategy. For example, the total loss function measures the learning loss. (Used for intra-class aggregation / inter-class separation), semantic anchor supervision loss and visual anchor point distillation loss The weighted sum form is expressed as: (6) in, , and These are the weighting coefficients for the corresponding terms.

[0041] To ensure that distillation alignment can construct same-identity pairings of "restricted domain sample - canonical domain reference sample", a P×K sampling strategy with identity balance is adopted for training batches: P identities are randomly selected in each batch, and K instances of each identity are sampled to form the batch; within each batch, canonical domain samples of the same identity are selected as the reference samples for each restricted domain sample. During training, the teacher network only performs EMA updates; both the student network backbone and adapter participate in gradient updates (the backbone learning rate is usually less than the adapter learning rate to preserve pre-training capabilities and accelerate restricted domain compensation learning).

[0042] In summary, this invention designs a semantic and visual dual-anchor guidance mechanism based on the canonical domain, including a canonical domain semantic anchor for identity consistency constraints and a canonical domain visual anchor for structural alignment. By jointly guiding semantics and visual distillation alignment, the constrained domain features simultaneously converge towards the canonical domain features at both the identity semantic and visual structural levels, improving the stability and accuracy of cross-modal and cross-platform person re-identification.

[0043] Step S5: For the acquired target image, the corresponding feature vector is extracted using the trained unified backbone and differentiated routing network to achieve pedestrian re-identification.

[0044] During the model application phase, only the unified backbone after training is retained. Features are extracted based on the corresponding path selected according to the domain, and cross-modal and cross-platform retrieval matching is completed. For example, to reduce computational and storage overhead, the teacher network and text encoder are removed during model application, retaining only the trained AAUB as a single feature extractor. For the input image, a direct path (canonical domain) or an adapted path (restricted domain) is selected based on the domain identifier to output feature vectors. The similarity (e.g., cosine similarity) between the query features and the search database features is calculated and sorted, outputting the re-identification results. Since the restricted domain features have been aligned to the canonical domain feature manifold during training, cross-modal and cross-platform retrieval matching can be directly completed in the unified feature space during the model application phase.

[0045] It should be noted that this invention can be compliantly applied in fields such as intelligent video surveillance and intelligent security, and can also be applied to traffic flow monitoring and congestion prediction. The training process of the model or network can be carried out offline on a server or in the cloud. Embedding the trained model into an electronic device enables real-time pedestrian re-identification. This electronic device can be a terminal device or a server. Terminal devices include any terminal device such as mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, and smart wearable devices (smartwatches, virtual reality glasses, virtual reality headsets, etc.). Servers include, but are not limited to, application servers or web servers, and can be independent servers, cluster servers, or cloud servers.

[0046] In summary, compared with the prior art, the present invention has the following main advantages: (1) This invention can effectively solve the information asymmetry problem in multimodal and multi-platform scenarios. Compared with the existing technology, which usually adopts symmetrical unified modeling and assumes that data from different modalities and platforms are equivalent, this invention introduces the concept of a normalized domain and uses information-rich data as a guiding benchmark to perform directional calibration on information-limited data, thereby fundamentally alleviating the information asymmetry problem caused by differences in imaging modalities and acquisition platforms.

[0047] (2) The feature alignment method designed in this invention is more stable and has stronger discriminative power. Existing technologies are prone to the problem of scattered distribution of the same identity features in multimodal and multi-platform scenarios. However, this invention uses a canonical domain guidance mechanism to make pedestrian features under different modalities and platforms converge to a unified reference distribution, thereby obtaining a more compact feature representation with strong discriminative power and improving the accuracy of pedestrian re-identification.

[0048] (3) This invention can avoid the negative impact of restricted domain noise on overall feature learning. Unlike existing methods where multimodal data participate in feature and semantic modeling, this invention uses the canonical domain as the main reference in the key guidance stage, which reduces the interference of information-restricted modalities on model training, reduces the risk of noise accumulation, and makes the model more robust in complex environments.

[0049] (4) This invention simultaneously considers both semantic consistency of identity and visual structural consistency. Existing technologies typically rely on a single form of supervision or alignment strategy, which makes it difficult to guarantee the unity of semantic and structural levels at the same time. This invention introduces a dual guidance mechanism of canonical domain semantic anchors and canonical domain visual anchors, which simultaneously constrains the restricted domain features at both the semantic and visual structural levels, making the cross-modal and cross-platform re-identification effect more reliable.

[0050] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0051] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0052] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0053] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0054] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0055] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0056] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0057] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0058] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A multimodal, multi-platform pedestrian re-identification method based on canonical domain guidance, comprising the following steps: Acquire a target image and determine whether the target image comes from a canonical domain or a restricted domain; Using the target image as the input image, the corresponding feature vector is extracted using a trained unified backbone and differentiated routing network; Based on the feature vector, the pedestrian re-identification result is obtained; The unified backbone and differentiated routing network is constructed based on the Transformer model and contains multiple Transformer blocks. Each Transformer block is equipped with a convolutional adapter, which is configured to propagate the features of the normalized domain directly along the backbone for the normalized domain and to superimpose the calibration increment on the features of the restricted domain outside the backbone output for the restricted domain.

2. The method according to claim 1, characterized in that, The unified backbone and differentiated routing network are trained according to the following steps: A training set is constructed by acquiring multimodal, multi-platform pedestrian image data, represented as follows: in, For pedestrian images, For identity tags, For domain identification, It is a sample index. It is the sample size; Pre-trained text prompts are used to generate canonical semantic anchors, where only canonical domain samples are used to optimize learnable tokens and obtain a set of semantic anchor vectors corresponding to each identity. Knowledge distillation training is performed using a set total loss function to generate canonical domain visual anchors and perform distillation alignment. The teacher network uses a standard Transformer encoder that does not include the convolutional adapter, and the student network uses the unified backbone and differentiated routing network.

3. The method according to claim 2, characterized in that, During the knowledge distillation training process, the training batch adopts an identity-balanced P×K sampling strategy: each batch randomly selects P identities, and each identity is composed of K instances to form a batch; within each batch, a canonical domain sample of the same identity is selected for each restricted domain sample.

4. The method according to claim 2, characterized in that, During the knowledge distillation training process, the parameters of the teacher network are updated through the exponential moving average of the backbone parameters of the student network, expressed as: in, This represents the backbone parameters in the student network that are isomorphic to the teacher network. The momentum coefficient, This represents the parameters of the teacher network.

5. The method according to claim 2, characterized in that, The total loss function is set as follows: in, , and These are the weighting coefficients of the corresponding terms. It measures the learning loss. It is the semantic anchor supervision loss. It is visual anchor point distillation loss.

6. The method according to claim 5, characterized in that, The semantic anchor point supervision loss and the distillation loss of the visual anchor point Set them to: in, It is a restricted field sample. It is student network output. It is a canonical domain reference sample of the same identity. It serves as the visual anchor point for the teacher's online output. It stops the gradient operation. It is a visual feature. It is the semantic anchor point corresponding to the real identity. This is the temperature coefficient.

7. The method according to claim 2, characterized in that, Introduce domain-dependent adapters in each Transformer block. When the input sample comes from the normal domain season This allows the features in the normalized domain to propagate directly along the main trunk when the input sample comes from the restricted domain. season This allows a calibration increment to be superimposed on the restricted domain features outside the main output.

8. The method according to claim 7, characterized in that, Will Set to: in, These are learnable parameters. Indicates dimensional ascension. This indicates dimensionality reduction. This represents depthwise separable convolution.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer device comprising a memory and a processor, wherein a computer program capable of running on the processor is stored in the memory, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images