A cross-view pedestrian re-identification method based on visual language prompt learning

Abstract

The present application relates to the technical field of computer vision and cross-view pedestrian recognition, and particularly relates to a cross-view pedestrian re-identification method based on visual language prompt learning, comprising: acquiring a target image; inputting the target image into a preset pedestrian recognition model to output a pedestrian re-identification result, wherein the pedestrian recognition model is obtained by training a visual language pre-training model CLIP through a prompt learning mechanism and a two-stage training strategy, wherein the prompt learning mechanism is used for modeling view bias, and the two-stage training strategy is used for realizing cross-modal semantic alignment. The present application can significantly improve the accuracy and robustness of cross-view recognition.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and cross-view pedestrian recognition technology, and in particular to a cross-view pedestrian re-recognition method based on visual language cue learning. Background Technology

[0002] Since the beginning of the 21st century, breakthroughs in deep learning, computer vision, and multimodal representation learning technologies have driven the widespread application of intelligent sensing systems in fields such as public safety, smart cities, and unmanned systems. Pedestrian re-identification, a core task in visual perception, aims to identify the same individual pedestrian across camera viewpoints and plays a crucial role in cross-camera tracking, behavior analysis, and security systems.

[0003] While traditional pedestrian re-identification tasks have achieved significant results in ground-to-ground and sky-to-sky perspectives, performance suffers greatly in heterogeneous aerial and ground-view scenarios due to substantial differences in viewpoint, resolution, and background. This makes model generalization difficult and leads to a significant drop in recognition performance. This task requires accurate matching of the same pedestrian between aerial and ground perspectives, presenting the following key challenges:

[0004] First, the difference between aerial and ground perspectives causes serious inconsistencies in the appearance, posture, and structural expression of pedestrians, limiting the effectiveness of traditional texture or local feature comparison strategies. Second, aerial images have a wide field of view but low resolution, while ground images have clear textures but limited field of view, and the difference in data distribution between the two exacerbates modal shift. Third, although existing models based on CNN[9] or ViT

[10] can extract image features, they lack the ability to explicitly model the semantic bias induced by perspective.

[0005] To address the aforementioned issues, existing methods such as VDT employ feature subtraction strategies to remove viewpoint components, achieving some success. However, this strategy may lead to a decline in identity representation capabilities and fails to achieve fine-grained structural alignment within and across modalities. Meanwhile, emerging visual-language pre-trained models have demonstrated excellent cross-modal semantic alignment capabilities, but their native cue designs are primarily used for classification tasks, lacking the ability to model structural differences and viewpoint biases. Therefore, a novel cross-viewpoint recognition method is needed that integrates visual-language modeling capabilities, introduces a learnable viewpoint cue mechanism, adapts to spatial and ground modal differences, and improves cross-viewpoint semantic consistency and identity discrimination capabilities. Summary of the Invention

[0006] The purpose of this invention is to provide a cross-view pedestrian re-identification method based on visual language cue learning. Based on a visual-language pre-trained model, a cue template containing viewpoint semantics and a viewpoint-free general template are designed. A two-stage training strategy is adopted, and an air-ground similarity consistency loss is introduced to build a fine-grained alignment relationship between image and text modalities, which significantly improves the accuracy and robustness of cross-view recognition.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] A cross-view pedestrian re-identification method based on visual language cue learning includes:

[0009] Acquire the target image;

[0010] The target image is input into a preset pedestrian recognition model, and the pedestrian re-recognition result is output. The pedestrian recognition model is obtained by training a visual language pre-trained model CLIP using a cue learning mechanism and a two-stage training strategy. The cue learning mechanism is used to model viewpoint bias, and the two-stage training strategy is used to achieve cross-modal semantic alignment.

[0011] Optionally, the cue learning mechanism models perspective biases including:

[0012] In the first training phase, a viewpoint-aware text template is used, which includes semantic information about character categories or identities and injected viewpoint information to guide the model to perceive the structural offset between the viewpoints of the ground.

[0013] In the second training phase, a view-aware text template is used, which contains identity semantic information to guide the model to learn a view-aware identity consistency representation.

[0014] Optionally, the first training phase models the differences in structure, attitude, and scale between aerial and ground-view images, guiding image features to align with corresponding semantic prompts in the embedding space, thereby achieving structural correspondence between modalities.

[0015] Optionally, the first training phase employs a first loss function, which includes image-text contrast loss, text-image contrast loss, and air-ground similarity consistency loss.

[0016] Optionally, the first loss function is:

[0017]

[0018]

[0019]

[0020]

[0021]

[0022]

[0023] Among them, L stage1L represents the total loss in the first training phase. AGSCL The loss is for the consistency of similarity between the ground and the virtual space, where N represents the total number of pedestrian identities in the dataset, and <, ·> denote the inner product operation. Let the aerial image features and ground image features of the j-th sample be represented respectively. Let the aerial text features and ground text features of the j-th sample be represented respectively. Let the aerial image features and ground image features of the k-th sample be represented respectively. Let these represent the aerial text features and ground text features of the k-th sample, respectively. represents the image-text contrast loss under the sky view and the ground view, respectively; F(·) represents the image coding function. This represents the text encoding function. and Let represent the image input samples from the i-th aerial viewpoint and the ground viewpoint, respectively. and Let j be the input of the j-th image sample at the corresponding viewpoint. and This represents the prompt text input for the i-th sample from an open-air perspective. Input the prompt text for the j-th sample; log(·) represents the natural logarithm function, exp(·) represents the exponential function; parameter B represents the number of samples in the current training batch. P(y) represents the text image contrast loss from the sky view and the ground view, respectively; i ) indicates the relationship with category y i The set of matching image sample indices, |P(y i | indicates the number of samples in the set; Let these represent the inputs of the p-th image sample in the set from aerial and ground perspectives, respectively. and This represents the prompt text input for the p-th sample from an open-air perspective.

[0024] Optionally, the second training phase, based on the viewpoint bias modeling completed in the first training phase, further enhances the image encoder's ability to distinguish identities and improves its robustness to changes in air and ground viewpoints, ultimately learning a stable identity representation vector with viewpoint invariance.

[0025] Optionally, the second training phase employs a second loss function, which includes classification loss, triplet loss, and cross-modal alignment loss.

[0026] Optionally, the second loss function is:

[0027]

[0028]

[0029] L tri =max(d p -d n +α,0);

[0030]

[0031]

[0032] Among them, L stage2 L represents the total loss in the second training phase. id Let q represent the classification loss. k p represents the probability that category k is the true category y. k L represents the probability that a sample is predicted to be of class k. tri Denotes the triplet loss, d p d represents the features of positive samples. n The features representing negative samples, where α is the boundary value. represent the cross-modal alignment loss under the sky view and the ground view, respectively, and F(·) represents the image coding function. This represents the text encoding function. and Let represent the image input samples from the i-th aerial viewpoint and the ground viewpoint, respectively. and For category y j The corresponding representative image samples are input from the corresponding viewpoint. This represents the text input features of the i-th sample under the condition of no viewpoint cue template. Represents category y j The corresponding text input features are log(·) representing the natural logarithm function and exp(·) representing the exponential function.

[0033] The beneficial effects of this invention are as follows:

[0034] This invention proposes a cross-view pedestrian re-identification method based on visual language cue learning, applicable to individual identity matching tasks in scenarios involving drones and ground cameras. Addressing issues such as appearance changes, scale inconsistencies, and background interference caused by differences in aerial and ground perspectives, it introduces a cue word mechanism to model perspective bias and designs a two-stage training strategy to achieve cross-modal semantic alignment. In the first stage, cue text containing perspective semantics guides image-text pairing learning, enabling image features to possess clear perspective structural information. To further improve alignment, an aerial-ground similarity consistency loss function is constructed to constrain the structural similarity between image and text features at a fine-grained level, achieving stable cross-modal alignment. In the second stage, a cue-free approach is used to eliminate perspective-specific information, optimizing the identity representation learned by the image encoder and achieving more discriminative and perspective-invariant recognition results. This invention constructs fine-grained alignment relationships between image and text modalities, significantly improving the accuracy and robustness of cross-view recognition. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart of a cross-view pedestrian re-identification method based on visual language cue learning according to an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram illustrating the mechanism of the air-ground similarity consistency loss function in the two-stage training process according to an embodiment of the present invention.

[0038] Figure 3 This is a schematic diagram of the two-stage training strategy process according to an embodiment of the present invention;

[0039] Figure 4 The graph shows the baseline method and the test results of this method on the CARGO test set in this embodiment of the invention.

[0040] Figure 5 This is a comparison of the attention heatmap results of the baseline method and the method described in this embodiment of the invention on the AG-ReID dataset. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] This embodiment provides a cross-view pedestrian re-identification method based on visual language cue learning, including:

[0044] Acquire the target image;

[0045] The target image is input into a preset pedestrian recognition model, and the pedestrian re-recognition result is output. The pedestrian recognition model is obtained by training a visual language pre-trained model CLIP using a cue learning mechanism and a two-stage training strategy. The cue learning mechanism is used to model viewpoint bias, and the two-stage training strategy is used to achieve cross-modal semantic alignment.

[0046] Specifically, this embodiment proposes a novel cue learning mechanism and a two-stage training framework based on the visual language pre-trained model CLIP. It combines learnable cue templates, a spatial-ground similarity consistency loss function, and a semantic separation training strategy to achieve fine-grained consistency alignment between image and text modalities. The overall structure is as follows: Figure 1 As shown, the model consists of an image encoder, a text encoder, a cueing module, and a cross-modal alignment module, and learns view perception and view suppression features step by step through two stages.

[0047] Furthermore, the perspective bias in the modeling of the cue learning mechanism includes:

[0048] In the first training phase, a viewpoint-aware text template is used, which includes semantic information about character categories or identities and injected viewpoint information to guide the model to perceive the structural offset between the viewpoints of the ground.

[0049] In the second training phase, a view-aware text template is used, which contains identity semantic information to guide the model to learn a view-aware identity consistency representation.

[0050] Specifically, perspective modeling mechanisms based on cue-based learning include:

[0051] This embodiment constructs viewpoint-aware, learnable cue templates to guide image features to adaptively align semantic differences between empty spaces and ground. Two types of text cue templates are constructed during the training phase:

[0052] Perspective-aware text template: The structure is "a [Y1]...[Y]". m [X1][X2]...[X] perspective n The text prompt "[X1][X2]...[X]" indicates a photo of a person. n [] is used to indicate a person's category or identity semantics, [Y1]...[Y m The [] template is used to inject viewpoint information. This type of template explicitly contains viewpoint-related vocabulary to guide the model to perceive structural offsets between air and ground viewpoints in the early stages of training.

[0053] No-viewpoint text template: Uses the structure "a [X1][X2]...[X]" n The text prompt for "a person's photo" contains only identity semantic information and no viewpoint description words. It aims to extract viewpoint features and guide the model to learn identity consistency representations that are independent of viewpoint. It is suitable for the later stage of model training and can effectively guide image features to explicitly model viewpoint bias. In this way, the viewpoint information that is originally implicit in the image representation is clearly expressed and aligned in the semantic space through text prompts.

[0054] Furthermore, the first training phase models the differences in structure, attitude, and scale between aerial and ground-view images, guiding image features to align with corresponding semantic prompts in the embedding space, thereby achieving structural correspondence between modalities.

[0055] The first training phase employs a first loss function, which includes image-text contrast loss, text-image contrast loss, and air-ground similarity consistency loss.

[0056] Furthermore, based on the viewpoint bias modeling completed in the first training phase, the second training phase further enhances the image encoder's ability to distinguish identities and improves its robustness to changes in aerial and ground viewpoints, ultimately learning a stable identity representation vector with viewpoint invariance.

[0057] The second training phase employs a second loss function, which includes classification loss, triplet loss, and cross-modal alignment loss.

[0058] Specifically, Figure 2 This demonstrates the role of the Air-to-Ground Similarity Consistency (AGSC) loss function in the two-stage training process, divided into left and right sub-modules corresponding to the two training stages respectively. The Air-to-Ground Similarity Consistency (AGSC) loss function is as follows:

[0059]

[0060] Where N represents the total number of pedestrian identities in the dataset, and <·,·> represent the inner product operation, which measures the similarity between two features. Let the aerial image features and ground image features of the j-th sample be represented respectively. Let the aerial text features and ground text features of the j-th sample be represented respectively. Let the aerial image features and ground image features of the k-th sample be represented respectively. Let represent the aerial text features and ground text features of the k-th sample, respectively.

[0061] In Phase One ( Figure 2 On the left), the image encoder and text encoder extract image and text features from different perspectives and project them into the same semantic space. By applying the air-ground similarity consistency (AGSCloss), the consistency between the intramodal structure and the cross-modal similarity matrix is ​​constrained.

[0062] In Phase Two ( Figure 2 (On the right), the model uses text cues that do not contain viewpoint information (i.e., unbiased cues) T. N This guides image features to converge towards a unified identity space. During this stage, the text encoder weights are frozen, and only the image encoder is optimized to improve its viewpoint invariance and discriminative ability.

[0063] To improve training stability and feature generalization ability, a two-stage optimization strategy is proposed to gradually realize the identity representation transfer from "perceiving perspective differences" to "suppressing perspective differences." The process is as follows: Figure 3 As shown:

[0064] Phase 1: Bias Transfer

[0065] The primary objective of this stage is to explicitly model the differences between aerial and ground-based viewpoints in terms of structure, pose, and scale. This guides image features to align with corresponding semantic cues in the embedding space, achieving structural correspondence between modalities. The system receives pedestrian image samples from aerial and ground-based cameras and constructs two types of text cues templates (viewpoint-aware templates and viewpoint-free templates) for each pair. Each image-text pair is input into the image encoder and text encoder, respectively, to extract the corresponding image and text feature vectors. These vectors are then projected into a shared semantic space, allowing the image features to be supervised and guided by viewpoint semantics, thereby injecting distinguishable viewpoint structural bias information.

[0066] By introducing image-text alignment loss, intramodal consistency loss, and air-to-ground similarity consistency (AGSC) loss, this stage performs structural adjustment on image features at the semantic level, enabling aerial and ground images to establish fine-grained correspondences under a unified semantic framework. AGSC loss is used to enhance fine-grained alignment between images and text, preventing the model from relying solely on coarse-grained features and ignoring deep structural differences.

[0067] After this training phase, the image encoder possesses a certain viewpoint discrimination capability, capable of characterizing the shift trends between different viewpoints in the semantic space, thus laying the foundation for viewpoint suppression and identity abstraction modeling in the next phase. The loss function designed in this phase consists of three main parts: image-text (i2t) contrast loss, text-image (t2i) contrast loss, and spatial-ground similarity consistency (AGSC) loss.

[0068]

[0069]

[0070] The image-text contrast loss is shown in the formula above, where represents the image-text contrast loss under the sky view and the ground view, respectively; F(·) represents the image encoding function, which is used to encode the input image sample into a visual feature vector; This represents a text encoding function used to encode viewpoint cues into corresponding language feature vectors. and Let represent the image input samples from the i-th aerial viewpoint and the ground viewpoint, respectively. and Let j be the input of the j-th image sample at the corresponding viewpoint. and This represents the prompt text input for the i-th sample from an open-air perspective. Input the prompt text for the j-th sample; log(·) represents the natural logarithm function, exp(·) represents the exponential function; the symbols <·,·> represent the inner product similarity function between image and text features; parameter B represents the number of samples in the current training batch.

[0071]

[0072]

[0073] The text-image contrast loss is shown in the formula above, where P(y) represents the text image contrast loss from the sky view and the ground view, respectively; i ) indicates the relationship with category y i The set of matching image sample indices, |P(y i| indicates the number of samples in the set; and Let j be the input of the j-th image sample at the corresponding viewpoint. and This represents the prompt text input for the i-th sample from an open-air perspective. Input the prompt text for the j-th sample; Let these represent the inputs of the p-th image sample in the set from aerial and ground perspectives, respectively. and Let p represent the prompt text input for the p-th sample from an empty field of view. Then the total loss function for this stage can be expressed as:

[0074]

[0075] Phase Two: Unbiased Guidance

[0076] Phase Two, building upon the completed viewpoint bias modeling, further enhances the image encoder's ability to discriminate identities and improves its robustness to changes in aerial and ground viewpoints, ultimately learning a stable identity representation vector with viewpoint invariance. This phase employs a freezing strategy, fixing the text encoder parameters and cue template parameters, and ceasing updates to the language modality. This frees the image encoder training process from direct interference from viewpoint semantic cues, allowing it to focus on learning the identity information itself. In this phase, the model input remains image samples from aerial and ground viewpoints, but the corresponding text cue is changed to a unified "viewpoint-free template" ("a [X1][X2]...[X...]"). n [Photos of people] do not contain perspective descriptions and only provide neutral identity category semantics.

[0077] During training, the image encoder needs to independently extract image features and learn the ability to discriminate identity features without semantic difference cues, in order to improve its adaptability to viewpoint changes. To enhance this viewpoint-independent representation ability, the following loss functions are introduced for joint optimization in this stage: classification (id) loss, triplet (tri) loss, and cross-modal alignment loss.

[0078]

[0079] The classification (id) loss is shown in the formula above, where q k =(1-∈)δ k,y +∈ / N,q k p represents the probability that category k is the true category y. k This represents the probability that a sample is predicted to be of class k.

[0080] L tri =max(dp -d n +α,0);

[0081] The triplet (tri) loss is shown in the formula above, d p d represents the features of positive samples. n The features representing negative samples are α, which is the boundary value.

[0082]

[0083]

[0084] The cross-modal alignment loss is shown in the formula above. These represent the cross-modal alignment loss from the sky view and the ground view, respectively. and For category y j The corresponding representative image samples are input from the corresponding viewpoint, where Let represent the text input features of the i-th sample under the condition of "no viewpoint cue template". Represents category y j The corresponding text input features are both used to construct cross-modal alignment in the neutral semantic space. Finally, the total loss function for this stage is expressed as follows:

[0085]

[0086] Experimental verification:

[0087] The proposed model was trained on Nvidia GPUs using the PyTorch framework. During training, the model used a CLIP network with a ViT-B / 16 backbone as the joint feature encoder for images and text. Initially, all input images were uniformly adjusted to a resolution of 256×128. The first training phase used the Adam optimizer with a cosine annealing strategy to dynamically adjust the learning rate, initially set at 1e-4 and eventually decaying to 2e-6. To further validate the effectiveness of the proposed model, a quantitative comparison was performed with state-of-the-art (SOTA) methods for cross-view pedestrian re-identification. The results are shown in Tables 1 and 2. The tables list two commonly used metrics: Rank-1 accuracy and average accuracy (mAP) across different datasets. Rank-1 represents the Top-1 hit rate of the test image, used to measure recognition accuracy; mAP represents the average retrieval accuracy of all matching samples, comprehensively reflecting the model's overall discriminative ability and ranking performance. In addition, ALL, AA, GG and AG in the table represent different matching types. ALL represents the overall average performance under all view combinations; AA represents the matching performance between aerial view images; GG represents the matching performance between ground view images; and AG represents the cross-view matching performance between air and ground.

[0088] Table 1 compares the network with the current state-of-the-art (SOTA) method for cross-view pedestrian re-identification on the CARGO dataset, and Table 2 compares the network with the current state-of-the-art (SOTA) method for cross-view pedestrian re-identification on the AG-ReID dataset.

[0089] Table 1

[0090]

[0091]

[0092] Table 2

[0093]

[0094] As shown in Tables 1 and 2, through comparative experiments, this model demonstrates superior performance in the field of cross-view pedestrian re-identification. In the two key recognition metrics, Rank-1 and mAP, this method significantly surpasses current mainstream methods, fully demonstrating its technical advantages in identity feature modeling and cross-view matching alignment. To more intuitively demonstrate the recognition effect and representation capability of the model from the perspectives of feature space structure and model attention mechanism, further visualization evaluation analysis is introduced, such as... Figure 4 and Figure 5 As shown.

[0095] Figure 4The distribution of image features extracted by the baseline method and our proposed method (MVP) on the CARGO test set is shown, visualized using t-SNE dimensionality reduction. It can be observed that under the baseline model, features of the same identity still show significant shifts under different viewpoints, and there is overlap in the feature distribution between classes; while our proposed method achieves more compact intra-class aggregation and more explicit inter-class separation in the feature space, demonstrating stronger discriminative ability and cross-viewpoint alignment capability.

[0096] Figure 5 This section compares the attention heatmap results of the baseline method and our proposed method (MVP) on the AG-ReID dataset. Each image set includes the original input, the attention map generated by the baseline method, and the attention map generated by our proposed method. It can be seen that the baseline method often focuses on image edges or background areas, exhibiting issues of attention dispersion and viewpoint shift. In contrast, our proposed method significantly focuses on key human body parts (such as the head, torso, and clothing texture), maintaining consistency even under open-field perspectives, reflecting its superior cross-modal alignment and identity perception capabilities.

[0097] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A cross-view pedestrian re-identification method based on visual language prompt learning, characterized in that, The method comprises the following steps: acquiring a target image; inputting the target image into a preset pedestrian recognition model to output a pedestrian re-identification result, wherein the pedestrian recognition model is obtained by training a visual language pre-trained model CLIP using a prompt learning mechanism and a two-stage training strategy, wherein the prompt learning mechanism is used to model perspective bias, and the two-stage training strategy is used to realize cross-modal semantic alignment; the prompt learning mechanism modeling perspective bias comprises: in a first training stage, a perspective-aware text template is used, the perspective-aware text template comprising person category or identity semantic information and injected perspective information, which is used to guide the model to perceive the structural offset between aerial and ground perspectives, the first training stage modeling the differences in structure, pose and scale between aerial and ground perspective images, guiding the image features to align with the corresponding semantic prompt information in the embedding space, realizing the structural correspondence between modalities, the first training stage using a first loss function, the first loss function comprising image-text contrast loss, text-image contrast loss and aerial-ground similarity consistency loss; the first loss function is: ； ； ； ； ； ； wherein, represents the total loss of the first training stage, represents the aerial-ground similarity consistency loss, N represents the total number of pedestrian identities in the data set, represents an inner product operation, respectively represent the aerial image feature and the ground image feature of the jth sample, respectively represent the aerial text feature and the ground text feature of the jth sample, respectively represent the aerial image feature and the ground image feature of the kth sample, respectively represent the aerial text feature and the ground text feature of the kth sample, respectively represent the image-text contrast loss under the aerial view and the ground view; represents an image encoding function, represents a text encoding function, and respectively represent the image input sample under the ith aerial view and the ground view, and is the input of the jth image sample under the corresponding view, and represents the prompt text input of the ith sample under the aerial-ground view, , is the prompt text input of the jth sample; represents a natural logarithm function, represents an exponential function; parameter B represents the number of samples in the current training batch, , respectively represent the text-image contrast loss under the aerial view and the ground view; represents a set of image sample indexes matched with the class , represents the number of samples in the set; respectively represent the input of the pth image sample in the set under the aerial view and the ground view, and represents the prompt text input of the pth sample under the aerial-ground view; in a second training stage, a non-perspective text template is used, the perspective-aware text template containing identity semantic information, which is used to guide the model to learn identity consistency representation independent of perspective, the second training stage further enhancing the discrimination ability of the image encoder for identity and improving the robustness to aerial-ground perspective changes on the basis of the first training stage completing perspective bias modeling, ultimately learning a stable and perspective-invariant identity representation vector, the second training stage using a second loss function, the second loss function comprising classification loss, triplet loss and cross-modal alignment loss.

2. The cross-view pedestrian re-identification method based on visual language cue learning according to claim 1, characterized in that, the second loss function is: ； ； ； ； ； wherein, denotes the total loss of the second training stage, denotes the classification loss, denotes the class is the true class is the probability, denotes the probability that the sample is predicted as class denotes the probability that the sample is predicted as class denotes the triplet loss, denotes the feature representing the positive sample, denotes the feature representing the negative sample, is the boundary value, denote the cross-modal alignment loss under the sky view and the ground view, respectively, denotes the image encoding function, denotes the text encoding function, and denote the image input samples of the i-th under the sky view and the ground view, respectively, and is the input of the representative image sample corresponding to the class under the corresponding view, denotes the text input feature of the i-th sample under the no-view-prompt template condition, denotes the text input feature corresponding to the class , denotes the natural logarithm function, denotes the exponential function.

Images