An image classification method based on a joint embedding prediction architecture of local perception and global alignment

Abstract

This invention discloses a learning method based on a joint embedding prediction architecture of local perception and global alignment. First, image samples are read from a data loader to generate an original view and an enhanced view. The original view is input into a target encoder. The original view and the enhanced view are input into a context encoder and a predictor, respectively. An embedding prediction consistency loss is calculated based on the predicted token and the real target token. The predicted representations of the two views are aggregated and processed by a discriminative projection head to obtain the global embedding. A local-global semantic consistency regularization loss is calculated based on the local predicted features and the global embedding. An uncertain prediction head is introduced to adaptively weight the target prediction error. Finally, the above losses are combined according to weights into a total loss and backpropagation is performed to update the trainable parameters. This method optimizes local embedding prediction and global discriminative alignment, improving the robustness of feature representation and downstream recognition performance.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, specifically an image classification method based on a joint embedding prediction architecture of local perception and global alignment. It is mainly applicable to self-supervised visual representation learning tasks and can be used for feature pre-training for image classification. Background Technology

[0002] In recent years, self-supervised learning methods based on deep neural networks have made significant progress in the field of computer vision. Their goal is to learn transferable, general visual representations from large-scale image data without manual annotation. Existing mainstream self-supervised learning methods mainly follow two technical routes: predictive methods and contrastive methods.

[0003] Predictive self-supervised methods (such as I-JEPA, or Joint Image Embedding Prediction Architecture) construct context prediction tasks, enabling the model to predict the semantic representation of occluded regions (targets) from partially visible image regions (context). These methods perform predictions in the embedding space, avoiding the high-frequency noise introduced by direct pixel-level reconstruction, and effectively learn the local semantic structure and content relevance of images. However, these methods typically focus on modeling and semantic recovery of local context; their learning objective does not explicitly encourage the model to learn highly discriminative global feature representations. This results in room for performance improvement when the learned representations are directly applied to downstream tasks such as image classification and retrieval, which rely on the discriminative power of global features.

[0004] On the other hand, contrastive self-supervised methods construct different enhanced views of the image as positive sample pairs and increase the distance from other image samples, guiding the model to learn a robust and highly discriminative global embedding representation. These methods have achieved excellent performance in downstream classification tasks. However, standard contrastive learning methods typically operate directly on the global feature vector output by the encoder, easily neglecting the local semantic structure information within the image. More importantly, when the contrastive learning objective overemphasizes discriminability between instances, it may drive the global embedding representation to gradually deviate from the original intrinsic structure of the image, composed of local semantic content, resulting in a "semantic drift" phenomenon. That is, while the global representation is discriminative, it loses consistency with the local content of the original image, impairing the interpretability of the representation and its generalization ability in fine-grained understanding tasks.

[0005] Therefore, existing technologies present a prominent contradiction: predictive methods excel at learning local semantic realism but lack global discriminativeness; contrastive methods can obtain highly discriminative global representations but easily sacrifice local semantic consistency. How to synergistically leverage the advantages of prediction and contrastive tasks within a self-supervised learning framework, enabling the model to learn accurate local semantic representations through prediction mechanisms and obtain highly discriminative global embeddings through contrastive mechanisms, while ensuring that global discriminative features are strictly based on local predictive semantics to avoid semantic drift, is a pressing technical problem that needs to be solved. Summary of the Invention

[0006] The purpose of this invention is to improve the local semantic prediction ability and global embedding discrimination ability of a model in unsupervised / self-supervised visual representation learning scenarios, relying solely on image samples and without manual annotation. It also aims to alleviate the problem of semantic drift caused by the global embedding deviating from the local predicted semantic structure after the joint introduction of contrastive learning. This invention proposes an image classification method based on a joint embedding prediction architecture of local perception and global alignment. Based on the I-JEPA embedding space prediction mechanism, it introduces a view contrast enhancement method based on predicted features and adds a local-global semantic consistency regularization constraint. This ensures that the global embedding faithfully reflects the semantic information learned from the local prediction task, thereby improving the stability and generalization ability of the learned visual representation.

[0007] To achieve the above objectives, the technical solution specifically adopted by the present invention is as follows:

[0008] A learning method based on a joint embedding prediction architecture of local perception and global alignment includes the following steps:

[0009] S1. Acquire a batch of image samples and preprocess the image samples to generate an original view X and its corresponding enhanced view X'; generate a context index set indicating the visible area and a target index set indicating the occluded area for the original view X and the enhanced view X', respectively.

[0010] S2. Input the original view X and the enhanced view X' into the target encoder to obtain the true target representation; input the original view X and the enhanced view X' into the context encoder to obtain context features, and input the context features and target location information into the predictor to obtain the predicted representation of the target region;

[0011] S3. Perform dynamic feature fusion of uncertainty perception on the predicted representation and the real target representation to generate an optimized target representation;

[0012] S4. Based on the optimized target representation and the real target representation, calculate the prediction consistency loss; at the same time, based on the global embedding representation obtained by aggregating the optimized prediction target representation, calculate the contrastive learning loss, and calculate the local-global semantic consistency loss used to constrain the consistency between the global embedding representation and the local semantic features.

[0013] S5. Construct a total loss function based on the weighted sum of the prediction consistency loss, the contrastive learning loss, and the local-global semantic consistency loss, and update the model parameters based on the total loss function;

[0014] S6. Repeat steps S1 to S5 for iterative training to obtain the trained joint embedding prediction model, and apply the joint embedding prediction model to the classification task.

[0015] Preferably, in step S1, the enhanced view X' is obtained by performing at least one data augmentation operation on the original view X, including color dithering, rotation, random cropping, random horizontal flipping, or Gaussian blur.

[0016] Preferably, in step S1, the original view X and the enhanced view X' generate a context index set indicating the visible area and a target index set indicating the occluded area, respectively, through a random masking or block masking strategy.

[0017] Preferably, in step S2, the original view X and the enhanced view X' are respectively input to the target encoder, and the real target representation is extracted according to the target index set;

[0018] The image regions specified by the context index set in the original view X and the enhanced view X' are input to the context encoder to obtain context features; the context features and target location information are input to the predictor to obtain the predicted representation of the target region.

[0019] Preferably, in step S2, the parameters of the target encoder are kept frozen during training, or updated according to the parameters of the context encoder using an exponential moving average method.

[0020] Preferably, the dynamic feature fusion of uncertainty perception in step S3 specifically includes:

[0021] The prediction uncertainty corresponding to the prediction representation and the measurement uncertainty corresponding to the real target representation are estimated using a trainable uncertainty prediction head.

[0022] Based on the prediction uncertainty and the measurement uncertainty, a fusion gain is calculated, wherein the fusion gain is a function of the prediction uncertainty, the measurement uncertainty, and the contrastive learning temperature parameter;

[0023] Based on the fusion gain, the predicted representation and the real target representation are weighted and summed to generate the optimized target representation.

[0024] Preferably, step S4 includes the following sub-steps:

[0025] S4-1. Aggregate the optimized target representation to obtain a sample-level semantic feature vector; input the sample-level semantic feature vector into the discriminant projection head to obtain a normalized global embedding representation.

[0026] S4-2. Based on the global embedding representations of the original view X and the enhanced view X', calculate the contrastive learning loss to enhance the discriminative power between different samples;

[0027] S4-3. Calculate the prediction consistency loss based on the difference between the optimized target representation and the true target representation;

[0028] S4-4. Based on the sample-level semantic feature vector and the intermediate vector output by the discriminant projection head, calculate the local-global semantic consistency loss to constrain the global embedding representation to be faithful to the local predicted semantics.

[0029] Preferably, in step S5, a total loss function is constructed based on the weighted sum of the prediction consistency loss, the contrastive learning loss, and the local-global semantic consistency loss, and the parameters of the context encoder, the predictor, and the discriminant projection head are updated based on the total loss function.

[0030] Preferably, in step S4-3, the prediction consistency loss is calculated using the Smooth L1 loss function, and its value is the average of the Smooth L1 norm of the difference between the optimized target representation and the true target representation across all targets in the batch.

[0031] Preferably, in step S4-1, the aggregation is an average pooling operation along the target dimension; the discriminative projection head includes at least one affine transformation layer and a nonlinear activation function, used to map the sample-level semantic feature vector to the contrastive learning space and perform L2 norm normalization to obtain the global embedding representation of unit length.

[0032] Preferably, in step S4-2, the contrastive learning loss is calculated using the InfoNCE loss function, which is constructed as follows: the global embedding representations of different views of the same image are taken as positive sample pairs, the global embedding representations of different images within the batch are taken as negative samples, and the negative logarithm of the ratio of the cosine similarity between the positive sample pairs to the exponentially weighted sum of the cosine similarities of all negative sample pairs is taken as the loss value.

[0033] Preferably, in step S4-4, the local-global semantic consistency loss is obtained by calculating the mean square error between the vector of the sample-level semantic feature vector after linear mapping and the intermediate vector output by the discriminant projection head before normalization.

[0034] Preferably, in step S5, the total loss function is the sum of the prediction consistency loss, the contrastive learning loss, and the local-global semantic consistency loss, weighted by a preset coefficient.

[0035] This invention has the following characteristics and beneficial effects:

[0036] Without requiring manual annotation, the predictive self-supervised learning mechanism enables the model to learn the ability to recover high-level semantics of occluded regions, thereby improving the quality of local semantic representation.

[0037] By introducing a contrastive learning mechanism into the predictive feature space, the discriminativeness of the global embedding is enhanced, making the learned representations more suitable for downstream tasks such as classification.

[0038] By using local-global semantic consistency regularization constraints, we prevent global embeddings from deviating from local prediction semantics under contrastive learning, thereby improving the stability and generalization performance of model training.

[0039] By employing a dynamic feature fusion mechanism based on uncertainty awareness and using weighted updates equivalent to the Kalman filter approach, the impact of prediction noise on the learning process is effectively reduced, thereby enhancing the robustness of the target token representation. Attached Figure Description

[0040] Figure 1 This is a system architecture diagram of the image classification method based on the joint embedding prediction architecture of local perception and global alignment proposed in this invention.

[0041] Figure 2 This is a schematic diagram of the joint embedding prediction network structure of the present invention.

[0042] Figure 3 This is a flowchart illustrating the training process of the method of the present invention. Detailed Implementation

[0043] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0044] The following examples are provided for reference. Figure 1 , Figure 1This invention presents the system architecture of the image classification method based on a joint embedding prediction architecture of local perception and global alignment. In this embodiment, training is performed on the public image dataset tinyImageNet. The system reads a batch of samples from the data loader to obtain the original view image tensor. With Enhanced View And generate target index sets respectively. With context index set The target encoder extracts the complete token features from the input view and then... Extract the real target token representation The target encoder parameters are frozen or updated using an exponential moving average. The context encoder only updates parameters to... The corresponding visible token is encoded to obtain a context representation. The predictor receives the context representation and the mask token concatenated into a sequence, and outputs the predicted target token representation at the target position. and with Calculate the prediction consistency loss. The discriminant projection head aggregates the predicted target token representation and maps it to a unit-norm global embedding z. Positive and negative samples are constructed using the embeddings of the original view and the enhanced view, and the InfoNCE contrastive loss is calculated. To suppress the deviation of the global embedding from the local prediction semantics, a local-global consistency constraint is introduced to ensure that the output of the discriminant projection head and the locally aggregated features remain consistent in the same space.

[0045] See Figure 2 , Figure 2 This is a schematic diagram of the network structure of the present invention. The present invention includes a context encoder, a target encoder, a predictor, a discriminant projection head, an uncertainty prediction head, and a consistency mapping module. The target encoder and the context encoder can use the same backbone network; the target encoder outputs the true target token representation as a supervision signal, and the context encoder outputs the visible context representation for the predictor to reconstruct the target representation. The discriminant projection head is used to aggregate the predictor output into a global embedding and perform L2 normalization; the consistency mapping module maps the locally aggregated features to the discriminant space to calculate the consistency loss. Preferably, an uncertainty-aware dynamic fusion module can be introduced to... and Adaptive weighted fusion is performed to obtain Its concept can be equivalently understood as Kalman weighted update.

[0046] See Figure 3 , Figure 3This is a schematic diagram of the training process of the present invention. The present invention simultaneously optimizes the prediction consistency loss, contrastive loss, and consistency loss in the same training iteration: target encoding, context encoding, and prediction are performed on the two views respectively; the prediction output is aggregated into a global embedding computation InfoNCE; then the local-global consistency constraint is calculated; finally, the total loss is obtained by weighting and backpropagation is used to update the parameters of the context encoder, predictor, and each projection module, while the target encoder parameters are updated by exponential moving average.

[0047] The specific pseudocode implementation steps of this invention are as follows:

[0048]

[0049] The present invention will be further described below, with additional examples provided.

[0050] Example 1

[0051] Step 1. Initialize the model. Initialize the context encoder. Target encoder Predictor P, discriminant projection head Q, and uncertainty prediction head Among them, the target encoder The parameters are either frozen during training or updated by the context encoder using an exponential moving average (EMA).

[0052] Step 2. Data Preparation. Divide the training data into a training set and a validation set according to a preset ratio. During training, read a batch of image samples from the data loader to obtain the original view X and generate an enhanced view X'. The enhancement method includes one or both of color jitter and rotation. At the same time, generate a set of context indexes for the original view according to a preset patch masking strategy. With the target index set and the context index set of the enhanced view. With the target index set .

[0053] Step 3. Extract the true target representation and prior predicted representation. Input the original view X into the target encoder. and Extract the real target token representation The visible patch input context encoder specified by I_c will be used. Its output, concatenated with the mask token, is input into the predictor P, which outputs the predicted target token representation at the target location. The enhanced view X' is obtained similarly. and .

[0054] Step 4. Uncertainty-aware fusion. To reduce the impact of differences in occlusion difficulty and prediction noise on training stability, this embodiment introduces an uncertainty prediction head. .

[0055] The uncertainty prediction head represents the prediction for each target token. Output the corresponding prediction uncertainty This characterizes the confidence level of the token prediction result. The representation of the corresponding real target token output by the target encoder is denoted as... And set a measurement uncertainty term R for it.

[0056] Based on this, this embodiment introduces a temperature coefficient modulated fusion gain. The calculation method is as follows:

[0057]

[0058] in, The temperature coefficient in the comparative learning process is used to adjust the impact of the discriminant learning intensity on the fusion process. When the size is smaller and the contrastive learning discrimination constraint is stronger, the fusion gain is more sensitive to uncertainty, thus more actively suppressing the perturbation of the global embedding by highly uncertain predicted tokens.

[0059] Subsequently, the predicted representation and the true target representation are dynamically weighted and fused based on the fusion gain to obtain the fused target token representation. :

[0060]

[0061] Step 5. Generate global embedding. The aggregation is an average pooling operation along the target dimension; the discriminative projection head includes at least one affine transformation layer and a nonlinear activation function, used to map the sample-level semantic feature vector to the contrastive learning space and perform L2 norm normalization to obtain the global embedding representation of unit length.

[0062] The fused target token representation is averaged along the patch dimension to obtain a sample-level aggregated vector, and a unit norm global embedding representation is obtained through a discriminant projection head.

[0063]

[0064] Similarly, z' is obtained from the enhanced view.

[0065] Step 6. Calculate the contrastive loss. The contrastive learning loss is calculated using the InfoNCE loss function, which is constructed as follows: global embedding representations of different views of the same image are used as positive sample pairs, and global embedding representations of different images within the batch are used as negative samples. The negative logarithm of the ratio of the cosine similarity between positive sample pairs to the exponentially weighted sum of the cosine similarities of all negative sample pairs is used as the loss value.

[0066] Specifically, the original view and the enhanced view within the batch are embedded and stitched together to construct a set of positive and negative samples, and the InfoNCE contrast loss is calculated:

[0067]

[0068] in, Let τ be the cosine similarity, and τ be the temperature parameter.

[0069] Calculate prediction consistency loss: Based on the difference between the optimized target representation and the true target representation, calculate the prediction consistency loss. The prediction consistency loss is calculated using the Smooth L1 loss function, and its value is the average of the Smooth L1 norm of the difference between the optimized target representation and the true target representation across all targets in the batch.

[0070] Calculate the embedding prediction consistency loss based on the fused target token representation. Representation of the real target token Calculate the prediction consistency loss, which uses the Smooth L1 loss:

[0071]

[0072] Where B is the batch size. The target number of tokens for each sample. The predicted features of the i-th target token output by the predictor. This represents the corresponding real target token output by the target encoder.

[0073] Calculate the local-global semantic consistency loss: Based on the sample-level semantic feature vector and the intermediate vector output by the discriminant projection head, calculate the local-global semantic consistency loss to constrain the global embedding representation to be faithful to the local predicted semantics. The local-global semantic consistency loss is obtained by calculating the mean square error between the vector of the sample-level semantic feature vector after linear mapping and the intermediate vector output by the discriminant projection head before normalization.

[0074] Specifically, the local-to-global semantic consistency loss includes:

[0075] The optimized target token representation is aggregated to obtain local semantic aggregate features; the local semantic aggregate features are then mapped to the same feature space as the global embedding representation through a linear mapping.

[0076] The mean squared error between the local semantic aggregation features and the global embedding representation is calculated to constrain the ability of the global embedding representation to preserve local predicted semantics.

[0077]

[0078] In formula (6), To determine the unnormalized global semantic vector output by the projector, These are local semantic features obtained by aggregating the predicted features of the target token. is a linear mapping matrix used to map local semantic features to the global semantic space.

[0079] Step 7. Parameter Update. A total loss function is constructed based on the weighted sum of the prediction consistency loss, the contrastive learning loss, and the local-global semantic consistency loss. The parameters of the context encoder, the predictor, and the discriminant projection head are then updated based on this total loss function. The total loss function is the sum of the prediction consistency loss, the contrastive learning loss, and the local-global semantic consistency loss according to preset weighting coefficients.

[0080] The total loss function is a weighted average of prediction consistency loss, InfoNCE contrast loss, and local-global semantic consistency loss, and its form is as follows:

[0081]

[0082] In formula (7) and Preset weighting coefficients

[0083] The prediction consistency loss and contrastive loss are combined into a total loss according to their weights, and the context encoder is updated using backpropagation. Predictor P, discriminant projection head Q, and uncertainty prediction head Parameters; target encoder Update using EMA.

[0084] Step 8. This embodiment is one implementation of the training process. A training task includes one or more iteration rounds; steps 2 to 7 constitute a complete training iteration cycle, resulting in a trained joint embedding prediction model.

[0085] Step 9. Obtain the image data to be classified and apply the trained joint embedding prediction model to perform the classification task.

[0086] Experimental setup

[0087] The dataset used in the simulation experiments of this invention is Tiny ImageNet, a color image dataset that more closely approximates general objects. Each image is 64x64 pixels in size, with approximately 500 images per category, for a total of 200 categories, comprising 100,000 training images and 10,000 test images.

[0088] Results Analysis

[0089] The simulation experiment of this invention uses the linear probe top-1 accuracy as the evaluation index. That is, the encoder parameters are frozen in the pre-training stage, only one linear classification layer is trained to predict Tiny-ImageNet categories, and the top-1 accuracy is calculated on the validation set / test set to measure the separability and downstream transferability of the unsupervised pre-trained representation.

[0090] To verify the effectiveness of the improved scheme of this invention, this embodiment compares the performance of different training strategies under the same dataset, the same model backbone size, and the same evaluation method. The results are shown in the table below:

[0091]

[0092] As can be seen from the table above, the original tinyIJEPA baseline method relies solely on predictive consistency target learning representations, achieving a Top-1 accuracy of 71% for the linear probe. This indicates that its representations possess some semantic capabilities but lack global discriminative power. When contrastive learning is introduced, the model further enhances global alignment and instance differentiation capabilities across views while maintaining predictive representation learning, significantly improving the linear probe accuracy to 95%. This demonstrates that joint training can more effectively project semantic information from the prediction space into the separable discriminative space, thereby significantly improving downstream classification separability.

[0093] After adding the regularization term, the accuracy was further improved to 97%. This result shows that in the joint optimization process of "prediction consistency + contrastive learning", the regularization term can constrain the fidelity of the global embedding to the local semantic prediction, suppress the semantic drift caused by the excessive contrastive target, and improve the global discriminativeness while maintaining the stability of the local semantic structure, thereby bringing additional performance gains.

[0094] Furthermore, after introducing the uncertainty prediction head, the accuracy reached 98%. This is because the uncertainty prediction head can explicitly model the prediction confidence of the target token and dynamically weight and fuse the predicted representation and the target representation. This reduces the influence of noisy gradients caused by high occlusion regions, complex texture regions, or low-confidence predictions in the early stages of training, improves the numerical stability and effective gradient quality of the training process, and enables the model to obtain better representation quality and higher linear probe accuracy under the same training budget.

[0095] In summary, the method of this invention achieves a 71% to 98% improvement in Top-1 accuracy of linear probes on Tiny-ImageNet, demonstrating that this invention significantly improves the downstream usability and final classification performance of self-supervised representation learning under the I-JEPA framework by introducing contrastive learning to enhance global discriminativity, suppressing semantic drift through regularization terms, and improving training stability and robustness through uncertain prediction heads.

[0096] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. An image classification method based on a joint embedding prediction architecture of local perception and global alignment, characterized in that, Includes the following steps: S1. Acquire a batch of image samples and preprocess the image samples to generate an original view X and its corresponding enhanced view X'; generate a context index set indicating the visible area and a target index set indicating the occluded area for the original view X and the enhanced view X', respectively. S2. Input the original view X and the enhanced view X' into the target encoder to obtain the true target representation; input the image region specified by the context index set in the original view X and the enhanced view X' into the context encoder to obtain context features, and input the context features and target location information into the predictor to obtain the predicted representation of the target region; S3. Perform uncertainty-aware dynamic feature fusion on the predicted representation and the real target representation to generate an optimized target representation. The dynamic feature fusion of uncertainty perception specifically includes: The prediction uncertainty corresponding to the prediction representation and the measurement uncertainty corresponding to the real target representation are estimated using a trainable uncertainty prediction head. Based on the prediction uncertainty and the measurement uncertainty, a fusion gain is calculated, wherein the fusion gain is a function of the prediction uncertainty, the measurement uncertainty, and the contrastive learning temperature parameter; Based on the fusion gain, the predicted representation and the real target representation are weighted and summed to generate the optimized target representation; S4. Calculate the prediction consistency loss based on the optimized target representation and the true target representation; S4-1. Aggregate the optimized target representation to obtain a sample-level semantic feature vector; input the sample-level semantic feature vector into the discriminant projection head to obtain a normalized global embedding representation; The aggregation is an average pooling operation along the target dimension; the discriminative projection head includes at least one affine transformation layer and a nonlinear activation function, used to map the sample-level semantic feature vector to the contrastive learning space and perform L2 norm normalization to obtain the global embedding representation of unit length; S4-2. Based on the global embedding representations of the original view X and the enhanced view X', calculate the contrastive learning loss to enhance the discriminative power between different samples; S4-3. Calculate the prediction consistency loss based on the difference between the optimized target representation and the true target representation; S4-4. Based on the sample-level semantic feature vector and the unnormalized global semantic vector output by the discriminant projection head, calculate the local-global semantic consistency loss to constrain the global embedding representation to be faithful to the local predicted semantics. S5. Construct a total loss function based on the weighted sum of the prediction consistency loss, the contrastive learning loss, and the local-global semantic consistency loss, and update the model parameters based on the total loss function; S6. Repeat steps S1 to S5 for iterative training to obtain the trained joint embedding prediction model, and apply the joint embedding prediction model to the classification task.

2. The method according to claim 1, characterized in that, In step S1, the enhanced view X' is obtained by performing at least one data augmentation operation on the original view X, including color dithering, rotation, random cropping, random horizontal flipping, or Gaussian blur.

3. The method according to claim 1, characterized in that, In step S1, the original view X and the enhanced view X' generate a context index set indicating the visible area and a target index set indicating the occluded area, respectively, through random masking or block masking strategies.

4. The method according to claim 1, characterized in that, In step S2, the original view X and the enhanced view X' are respectively input to the target encoder, and the real target representation is extracted according to the target index set; The image regions specified by the context index set in the original view X and the enhanced view X' are input into the context encoder to obtain context features; The contextual features and target location information are input into the predictor to obtain a predicted representation of the target region.

5. The method according to claim 1, characterized in that, In step S2, the parameters of the target encoder are kept frozen during training, or updated based on the parameters of the context encoder using an exponential moving average method.

6. The method according to claim 1, characterized in that, In step S5, a total loss function is constructed based on the weighted sum of the prediction consistency loss, the contrastive learning loss, and the local-global semantic consistency loss, and the parameters of the context encoder, the predictor, and the discriminant projection head are updated based on the total loss function.

7. The method according to claim 1, characterized in that, In step S4-3, the prediction consistency loss is calculated using the Smooth L1 loss function, and its value is the average of the Smooth L1 norm of the difference between the optimized target representation and the true target representation across all targets in the batch.

8. The method according to claim 1, characterized in that, In step S4-1, the aggregation is an average pooling operation along the target dimension; the discriminative projection head includes at least one affine transformation layer and a nonlinear activation function, used to map the sample-level semantic feature vector to the contrastive learning space and perform L2 norm normalization to obtain the global embedding representation of unit length.

9. The method according to claim 1, characterized in that, In step S4-2, the contrastive learning loss is calculated using the InfoNCE loss function, which is constructed as follows: the global embedding representations of different views of the same image are taken as positive sample pairs, the global embedding representations of different images within the batch are taken as negative samples, and the negative logarithm of the ratio of the cosine similarity between the positive sample pairs to the exponentially weighted sum of the cosine similarities of all negative sample pairs is taken as the loss value.

10. The method according to claim 1, characterized in that, In step S4-4, the local-global semantic consistency loss is obtained by calculating the mean square error between the sample-level semantic feature vector after linear mapping and the intermediate vector output by the discriminant projection head before normalization.

11. The method according to claim 1, characterized in that, In step S5, the total loss function is the sum of the prediction consistency loss, the contrastive learning loss, and the local-global semantic consistency loss, weighted by a preset coefficient.

Images