Remote sensing image distribution outside detection method and device, electronic equipment and storage medium

Abstract

This invention provides a method, apparatus, electronic device, and storage medium for detecting out-of-distribution features in remote sensing images, belonging to the field of artificial intelligence technology. The method includes: determining global similarity based on the similarity between feature points of a global scene text prompt and a spatial feature map of the remote sensing image to be detected; determining category similarity based on the similarity between feature points of a specific category text prompt and a spatial feature map; determining background similarity based on the similarity between feature points of a negative background text prompt and a spatial feature map; and determining an in-distribution consistency score of the feature points based on the global similarity, category similarity, and background similarity to determine the out-of-distribution detection result of the remote sensing image to be detected. This invention improves the reliability of out-of-distribution detection by introducing a dual-prompt spatial semantic alignment mechanism containing two sets of learnable text prompts for consistency verification, and further introducing a negative background prompt mechanism to suppress common interference.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting the distribution of remote sensing images. Background Technology

[0002] The emergence of Vision-Language Models (VLMs) has brought a new technical approach to the detection of out-of-distribution (OOD) remote sensing images.

[0003] Taking the RemoteCLIP model in remote sensing VLMs as an example, its core technical solution is a dual encoder architecture and a specific training paradigm. The dual encoder architecture includes an image encoder for processing the remote sensing image in the input image-text pair (also known as image-text pair) and a text encoder for processing the remote sensing image description text in the input image-text pair. The image encoder generates a visual overall feature vector for the entire remote sensing image, and the text encoder generates a semantic feature for the remote sensing image description text. A bidirectional contrastive learning loss function is used to enable the model to learn to align visual features and semantic concepts in the same high-dimensional space and obtain zero-shot classification capability.

[0004] However, OOD patterns in remote sensing scenes often do not occupy the entire image, but rather appear as local anomalies. Global strategies for visual feature processing of remote sensing images are more suitable for natural images with clearly defined themes, leading to the neglect of crucial fine-grained spatial details and local information. Furthermore, existing visual language models typically employ a "forward matching" strategy, judging only whether image content semantically matches a known category. However, the background environment of remote sensing images is extremely complex, containing numerous non-target general interference items (such as thick clouds, shadows, water reflections, seasonal surface changes, etc.). Existing models often lack explicit suppression mechanisms for this background noise, making them easily misled by complex background textures. This leads to false positives (false positives) of background clutter or false positives of cluttered out-of-distribution samples, severely impacting detection robustness. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for detecting out-of-distribution samples in remote sensing images, in order to solve the defects in the prior art that ignore fine-grained spatial details and local information in remote sensing images, and that easily misdetect background clutter as out-of-distribution samples or misjudge out-of-distribution samples containing interference as background.

[0006] This invention provides a method for detecting out-of-distribution features in remote sensing images, comprising: Acquire spatial feature maps of the remote sensing image to be detected; The global similarity of the feature points is determined based on the similarity between the pre-trained global scene text prompts and the feature points of the spatial feature map. The category similarity of the feature points is determined based on the similarity between the pre-trained specific category text prompts and the feature points of the spatial feature map; The background similarity of the feature points is determined based on the similarity between the negative background text prompt and the feature points of the spatial feature map; the negative background text prompt is determined based on a general background interference description. The distributional consistency score of the feature points is determined based on the global similarity, the category similarity, and the background similarity of the feature points. Based on the in-distribution consistency score of the feature points, the out-of-distribution detection result of the remote sensing image to be detected is determined.

[0007] According to the remote sensing image distribution out-of-distribution detection method provided by the present invention, the global scene text prompt and the specific category text prompt are obtained by jointly optimizing the initial global scene text prompt feature map and the initial specific category text prompt feature map based on a composite loss function; the composite loss function is determined based on a global loss function, a category loss function and a diversity loss function.

[0008] According to the remote sensing image out-of-distribution detection method provided by the present invention, the initial global scene text cue feature map is determined based on the mean of global scene text features and a learnable perturbation term; the mean of global scene text features is determined based on the text description samples of all in-distribution image-text pair annotation samples; the initial specific category text cue feature map is determined based on the specific category text centroid features and optimized vectors; the specific category text centroid features are determined based on the text description samples of specific categories of in-distribution image-text pair annotation samples among all in-distribution image-text pair annotation samples.

[0009] According to a remote sensing image out-of-distribution detection method provided by the present invention, the step of obtaining the spatial feature map of the remote sensing image to be detected includes: inputting the remote sensing image to be detected into an image encoder of a remote sensing image out-of-distribution detection model to obtain an initial spatial feature map output by the image encoder; processing the initial spatial feature map based on a dual-branch attention mechanism to obtain a spatial attention weight matrix of the initial spatial feature map; and performing neighborhood information aggregation processing on the feature points of the initial spatial feature map based on Gaussian weighting and the spatial attention weight matrix to obtain the spatial feature map.

[0010] According to the remote sensing image out-of-distribution detection method provided by the present invention, the image encoder is obtained by jointly training the text encoder and the image encoder with the goal of maximizing a first similarity and minimizing a second similarity; the first similarity is the similarity between the predicted spatial feature map and the predicted text feature map of the matching image-text pair sample; the second similarity is the similarity between the predicted spatial feature map and the predicted text feature map of the unmatched image-text pair sample; wherein, the joint training loss function is determined based on the image-to-text loss and the text-to-image loss.

[0011] According to the present invention, a remote sensing image out-of-distribution detection method is provided, wherein the remote sensing image out-of-distribution detection model is trained using in-distribution image-text pair labeled samples and automatically labeled image-text pair samples; the automatically labeled image-text pair samples are determined based on unlabeled image-text pair samples whose prediction entropy is less than a preset entropy threshold, prediction consistency score is higher than a preset consistency score threshold, and prediction confidence is greater than an adaptive confidence threshold.

[0012] The present invention also provides a remote sensing image distribution detection device, comprising: The spatial feature map acquisition module is used to acquire the spatial feature map of the remote sensing image to be detected. The global similarity determination module is used to determine the global similarity of the feature points based on the similarity between the pre-trained global scene text prompts and the feature points of the spatial feature map. The category similarity determination module is used to determine the category similarity of the feature points based on the similarity between the pre-trained specific category text prompts and the feature points of the spatial feature map; The background similarity determination module is used to determine the background similarity of the feature points based on the similarity between the negative background text prompt and the feature points of the spatial feature map; the negative background text prompt is determined based on a general background interference description; The consistency score determination module is used to determine the distribution consistency score of the feature points based on the global similarity of the feature points, the category similarity of the feature points, and the background similarity of the feature points. The detection result determination module is used to determine the out-of-distribution detection result of the remote sensing image to be detected based on the in-distribution consistency score of the feature points.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the remote sensing image out-of-distribution detection method as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the remote sensing image out-of-distribution detection method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the remote sensing image distribution out-of-distribution detection method as described above.

[0016] The remote sensing image out-of-distribution detection method, device, electronic device, and storage medium provided by this invention introduce a dual-hint spatial semantic alignment mechanism containing two sets of learnable text hints for consistency verification. It designs parallel global scene text hints and specific category text hints, taking into account fine-grained spatial details and local information in the remote sensing image through specific category text hint alignment. Image content is cross-validated by comparing the global context of the scene and the fine-grained category semantics. Furthermore, the invention requires that the spatial features of the remote sensing image to be detected must be aligned with both the global scene text hints and the specific category text hints to be considered within the distribution. Based on this, the invention further introduces a learnable negative background hint mechanism, constructing a positive and negative bidirectional feature verification system. Negative hints are used to explicitly suppress common interference such as clouds and shadows, effectively filtering out abnormal predictions that conflict between fine-grained semantics and the global background. This helps improve the reliability of out-of-distribution detection, avoiding misdetection of background clutter as out-of-distribution samples or misjudging out-of-distribution samples containing interference as background. This enhances the ability of methods such as remote sensing image out-of-distribution detection models to distinguish between "different but similar" within-distribution samples and "truly unknown" out-of-distribution samples. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the remote sensing image distribution detection method provided by the present invention.

[0019] Figure 2 This is a schematic diagram of the structure of the remote sensing image distribution external detection model provided by the present invention.

[0020] Figure 3 This is a diagram showing the separation effect provided by the present invention.

[0021] Figure 4 This is a comparison chart of different sample numbers provided by the present invention.

[0022] Figure 5 This is a schematic diagram of the structure of the remote sensing image distribution detection device provided by the present invention.

[0023] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] The following is combined Figures 1 to 6 The present invention describes a remote sensing image distribution detection method, apparatus, electronic device, and storage medium.

[0026] In remote sensing applications, deep learning models are typically trained under the assumption of a "closed world," meaning all test samples are assumed to belong to categories known during training. However, in real-world "open-world" deployment scenarios, these systems inevitably encounter novel features, anomalous patterns, or unseen scene types never seen in the training data. To address the challenge of OOD detection, several mainstream methods have emerged: classification-based methods, reconstruction-based methods, and feature-based methods. Classification-based methods achieve OOD detection by modifying the classifier architecture, but struggle to handle anomalies with features similar to known categories. Reconstruction-based methods utilize generative models to learn the distribution of known data and identify OOD samples through reconstruction errors, but are computationally expensive. Feature-based methods detect OOD by analyzing differences in deep feature representations, but typically require large calibration datasets.

[0027] OOD (Out-of-Depth) detection in remote sensing faces a dual challenge. First, data scarcity: acquiring large-scale, precisely labeled imagery covering all land cover categories or geographic regions is extremely difficult in practice, requiring expensive field surveys or expert interpretation. Second, the inherent high complexity of remote sensing imagery: a single image often contains multi-scale land cover (from individual buildings to vast farmlands) embedded in a highly variable environmental context, and also exhibits significant distributional shifts caused by different sensors, geographical locations, or imaging conditions. A key flaw in existing models when faced with such OOD inputs is their tendency to incorrectly classify them with unreasonably high confidence into the most similar known category—a phenomenon known as "overconfidence misclassification." This unreliable prediction severely undermines the credibility of automated geospatial analysis systems and poses serious security risks in critical applications such as disaster response and urban planning.

[0028] The emergence of Vision-Language Models (VLMs) has brought new technical pathways to OOD detection, with the RemoteCLIP model being a representative advanced solution in the remote sensing field. The RemoteCLIP model is a VLM specifically trained for the remote sensing domain, built upon CLIP. By pre-training on a large number of remote sensing image-text pairs, it overcomes the shortcomings of general-purpose VLMs in understanding top-down view images. The core of RemoteCLIP's technical solution lies in its dual-encoder architecture and a specific training paradigm. This architecture includes an image encoder (typically a Vision Transformer, ViT) for processing the input remote sensing imagery and a text encoder for processing text cues. Its training process employs a bidirectional contrastive learning loss function, allowing the model to align visual features with semantic concepts in the same high-dimensional space, thus achieving powerful zero-shot classification capabilities.

[0029] Despite the progress made by remote sensing VLM, represented by RemoteCLIP, it still has several significant and interrelated shortcomings when dealing with complex OOD detection tasks, especially when facing the inherent challenges of remote sensing imagery.

[0030] First, it lacks spatial awareness and is unable to identify local anomalies.

[0031] The core problem with existing VLMs (including RemoteCLIP, which is optimized for remote sensing) lies in their global image processing approach, generating a single, holistic feature vector for the entire image. While this "global pooling" strategy is suitable for natural images with well-defined subjects, it is a fatal flaw in remote sensing applications because it completely ignores crucial fine-grained spatial details and local information in the image. Out-of-Depth (OOD) patterns in remote sensing scenes often do not occupy the entire image but appear as local anomalies.

[0032] Second, it heavily relies on large-scale labeled data and performs poorly in scenarios with few samples.

[0033] A core challenge in remote sensing is data scarcity. Acquiring high-quality labeled samples is a costly and extremely time-consuming process, typically requiring professional field surveys or manual interpretation by remote sensing experts. Therefore, most remote sensing datasets have a very limited number of samples per category, often only tens to hundreds, forcing models to learn under "few-shot" conditions. Furthermore, with the development of visual language models, some image-text collaborative detection methods have recently emerged. These methods usually rely on predefined anomalous text descriptions (such as preset category labels like "fire" and "landslide") and match them by calculating the similarity between the image and the anomalous text. However, this "forward matching" strategy has significant limitations: it is essentially still a closed-set classification method, dependent on a comprehensive anomalous knowledge base. When faced with unseen, undefined, out-of-distribution (OOD) targets, these methods often fail due to the lack of corresponding text templates. Therefore, how to utilize visual language models to build a general detection mechanism that does not rely on preset anomaly types is a pressing problem that needs to be solved.

[0034] Third, existing few-shot learning methods have limitations.

[0035] Few-shot learning methods (such as prototype networks) assume that a limited number of training samples are sufficient to represent the complete distribution of the category. This assumption is completely invalid in the field of remote sensing, because the same land cover category can exhibit huge visual differences in different geographical regions, climates, or imaging conditions.

[0036] Fourth, the self-training strategy contains a fundamental contradiction.

[0037] To expand the data, some solutions employ self-training techniques, which utilize the model to generate "pseudo-labels" on a large amount of unlabeled data to aid training. However, these methods typically rely on a fundamentally contradictory assumption: they assume that all unlabeled data comes from a known class distribution. This completely contradicts the open-world premise of OOD detection, because in real-world applications, unlabeled data inevitably contains a large number of genuine OOD samples.

[0038] In view of the above, the present invention provides a method, apparatus, electronic device and storage medium for detecting the distribution of remote sensing images, so as to solve at least one of the above problems.

[0039] Figure 1 This is a flowchart illustrating the remote sensing image distribution out-of-distribution detection method provided by the present invention, as shown below. Figure 1 As shown, the remote sensing image distribution out-of-distribution detection method includes, but is not limited to, steps 101 to 106.

[0040] It should be noted that the execution subject of the remote sensing image distribution out-of-distribution detection method provided by the present invention can be a server, computer equipment, such as mobile phone, tablet computer, laptop computer, handheld computer, vehicle electronic equipment, wearable device, ultra-mobile personal computer (UMPC), netbook or personal digital assistant (PDA), etc.

[0041] Step 101: Obtain the spatial feature map of the remote sensing image to be detected.

[0042] Among them, the spatial feature map is a visual feature map extracted from the remote sensing image to be detected according to a certain feature extraction and processing method.

[0043] Specifically, the remote sensing image to be detected is input into the remote sensing image distribution out-of-distribution detection model. The image encoder of the remote sensing image distribution out-of-distribution detection model extracts the original visual feature map of the remote sensing image to be detected as the initial spatial feature map, and the initial spatial feature map output by the image encoder is obtained. The initial spatial feature map is then directly used as the spatial feature map of the remote sensing image to be detected, thereby obtaining the spatial feature map of the remote sensing image to be detected.

[0044] Among them, the remote sensing image distribution out-of-zone detection model is based on the VLMs model and obtained after pre-training.

[0045] For example, the VLMs model is the RemoteCLIP model.

[0046] Step 102: Determine the global similarity of the feature points based on the similarity between the pre-trained global scene text prompts and the feature points of the spatial feature map.

[0047] Step 103: Determine the category similarity of the feature points based on the similarity between the pre-trained specific category text prompts and the feature points of the spatial feature map.

[0048] Global scene text prompts It is a text prompt learned based on the overall distribution pattern of the scene within the encoding distribution (ID) (such as common spectral or texture features).

[0049] Specific category text prompts It is a text hint learned by capturing fine-grained semantic details of each ID category.

[0050] It is understandable that the multiple in-distribution text-image pairs labeled samples used to train the out-of-distribution detection model of remote sensing images generally belong to different target categories. Therefore, the global scene text prompts obtained through pre-training are... and specific category text prompts The number can be multiple.

[0051] Specifically, Figure 2 This is a schematic diagram of the structure of the remote sensing image distribution detection model provided by the present invention, combined with... Figure 2 As shown, this invention introduces a dual-prompt alignment mechanism containing two sets of learnable text prompts for spatial-semantic consistency verification. Firstly, it is necessary to perform spatial-semantic consistency verification at each spatial location. The similarity between the input spatial feature map and the two sets of text prompts is calculated separately. It is understood that the specific similarity can be determined based on any similarity metric such as maximum cosine similarity, and this invention does not limit this.

[0052] On one hand, the text encoder of the remote sensing image out-of-distribution detection model is used to process the pre-trained global scene text prompts to obtain a global scene text prompt feature map. For each feature point in the spatial feature map, the global similarity of that feature point is determined based on the different similarities between that feature point and other feature points in the global scene text prompt feature map. This step is repeated to obtain the global similarity of all feature points in the spatial feature map.

[0053] On the other hand, the text encoder of the remote sensing image out-of-distribution detection model is used to process the pre-trained specific-category text prompts to obtain a specific-category text prompt feature map. For each feature point in the spatial feature map, the category similarity of the feature point is determined based on the different similarities between the feature point and the feature points in the specific-category text prompt feature map. This step is repeated to obtain the category similarity of all feature points in the spatial feature map.

[0054] In one embodiment, the formulas for calculating the global similarity and category similarity of each feature point are as follows: ; ; in, For the spatial feature map, the first Line number Global similarity of a list of feature points; For the spatial feature map, the first Line number The category of each feature point is displayed. For the spatial feature map, the first Line number List the eigenvalues ​​of each feature point; For text encoders; This refers to the number of text prompts in the global scene. The number of text prompts for a specific category; For the first A global scene text prompt; For the first Text hints for specific categories.

[0055] Step 104: Determine the background similarity of the feature points based on the similarity between the negative background text prompt and the feature points of the spatial feature map.

[0056] Among them, negative background text prompts It is determined based on the general background interference description; the general background interference description is used to describe non-target features such as clouds and shadows that are likely to cause remote sensing images to be misclassified as out-of-distribution samples.

[0057] Generally, one type of non-target feature corresponds to a general background interference description; a general background interference description corresponds to a negative background text cue. Types of non-target features include, but are not limited to, thick clouds, building shadows, water reflections, blurred noise, etc.

[0058] Specifically, in order to address the problem that non-target features such as clouds and shadows in remote sensing images are easily misclassified as out-of-distribution (OOD) samples, a negative background suppression mechanism was innovatively introduced on the basis of the dual cue alignment mechanism.

[0059] Based on various types of non-target features, a negative background text cue pool is pre-constructed, which includes multiple negative background text cuees corresponding to multiple general background interference descriptions.

[0060] For each feature point in the spatial feature map, calculate the different similarities between that feature point and each feature point in the negative background text prompt feature map corresponding to each negative background text prompt in the negative background text prompt pool. Obtain the similarity between that feature point and each negative background text prompt, and determine the maximum value of the similarity between that feature point and each negative background text prompt as the background similarity of that feature point. Repeat this step to obtain the background similarity of all feature points in the spatial feature map.

[0061] Optionally, the negative background text cue corresponding to the general background interference description is obtained by pre-training the initial negative background text cue feature map corresponding to the general background interference description; the initial negative background text cue feature map is output by the text encoder of the remote sensing image distribution out-of-distribution detection model after the general background interference description is input into it.

[0062] Step 105: Determine the distribution consistency score of the feature points based on the global similarity, category similarity, and background similarity of the feature points.

[0063] Among them, global similarity and category similarity are positively correlated with the intra-distribution consistency score; background similarity is negatively correlated with the intra-distribution consistency score.

[0064] In addition to introducing a dual-cue alignment mechanism that includes two sets of learnable text cues, a region of the remote sensing image to be detected and its spatial feature map is only considered an ID when it is simultaneously aligned with the global pattern and the semantics of a specific category, a negative background suppression mechanism is also introduced.

[0065] Furthermore, an intra-distribution consistency score is introduced to evaluate the consistency between feature points of the spatial feature map and intra-distribution samples and general background interference, in order to determine whether the feature points of the spatial feature map are aligned with the global pattern and specific category semantics.

[0066] Specifically, for each feature point in the spatial feature map, the intra-distribution consistency score of that feature point is calculated based on its global similarity, category similarity, and background similarity. Repeating this step yields the intra-distribution consistency scores for all feature points within the spatial feature map.

[0067] In one embodiment, the formula for calculating the consistency score within the distribution of feature points is as follows: ; in, For the spatial feature map, the first Line number The consistency score of the distribution of each feature point; For the spatial feature map, the first Line number Global similarity of a list of feature points; For the spatial feature map, the first Line number The category similarity of a list of feature points; For the spatial feature map, the first Line number Background similarity of a list of feature points; For the spatial feature map, the first Line number The index of the global cue for the activation of a column of feature points, i.e. The first in the spatial feature map Line number The highest similarity is found among the feature points; For the spatial feature map, the first Line number The index of the category hints for the activation of each feature point, i.e. The first in the spatial feature map Line number The highest similarity is found among the feature points; A function for calculating the embedding cosine similarity between activated global cues and category cues; Indicator functions for verifying semantic consistency between activated global and category hints.

[0068] Specifically, if the embedding cosine similarity between the activated global cue and the category cue is lower than a preset threshold... Indicator function Returns 0; if the embedding cosine similarity between the activated global cue and the category cue is equal to or greater than a preset threshold. Indicator function Returns 1. Anomalies that conflict between fine-grained semantics and the global context can be filtered using the indicator function, such as filtering anomalies that incorrectly activate the "building" feature in a "forest" context.

[0069] By introducing a background penalty term into the formula for calculating the consistency score within the distribution, and utilizing the denominator term... Significant suppression is performed on regions that are highly similar to background interference, thereby improving the anti-interference ability of the detection results in complex environments.

[0070] Step 106: Determine the out-of-distribution detection result of the remote sensing image to be detected based on the in-distribution consistency score of the feature points.

[0071] Specifically, based on the in-distribution consistency scores of multiple or all feature points in the spatial feature map, the out-of-distribution detection score (OOD Score) of the remote sensing image to be detected is determined, and based on the out-of-distribution detection score of the remote sensing image to be detected, the out-of-distribution detection result of the remote sensing image to be detected is determined.

[0072] Optionally, the out-of-distribution detection results include, but are not limited to, the out-of-distribution detection score, predicted distribution type, predicted target category, and classification confidence of the remote sensing image to be detected.

[0073] Among them, the predicted distribution type includes in-distribution type (ID) and out-of-distribution type (OOD); the predicted target category is the category obtained by predicting the target in the remote sensing image to be detected, such as airplane, car, etc.; the category confidence is the confidence of the predicted target category.

[0074] Optionally, the out-of-distribution detection score of the remote sensing image to be detected is determined based on the maximum value of the intra-distribution consistency score of all feature points in the spatial feature map, the average value of the intra-distribution consistency score of all feature points in the spatial feature map, and the spatial entropy of the intra-distribution consistency score of all feature points in the spatial feature map.

[0075] The OOD score, which combines three complementary measures—maximum, mean, and spatial entropy—can measure the degree of consistency of sample offset ID patterns.

[0076] In one embodiment, the formula for calculating the out-of-distribution detection score of the remote sensing image to be detected is as follows: ; in, The out-of-distribution detection score of the remote sensing image to be detected; This represents the maximum consistency score within the distribution of all feature points in the spatial feature map. This is the average of the consistency scores within the distribution of all feature points in the spatial feature map; The consistency score of the distribution of all feature points in the spatial feature map; , , These are learnable weights; is the spatial entropy, used to capture the spatial uncertainty of the consistency score within a distribution.

[0077] The remote sensing image out-of-distribution detection method provided by this invention introduces a dual-hint spatial semantic alignment mechanism containing two sets of learnable text hints for consistency verification. It designs parallel global scene text hints and specific category text hints, taking into account fine-grained spatial details and local information in the remote sensing image through specific category text hint alignment. Image content is cross-validated by comparing the global context of the scene and the fine-grained category semantics. Furthermore, the spatial features of the remote sensing image to be detected must be aligned with both the global scene text hints and the specific category text hints to be considered within the distribution. Based on this, this invention further introduces a learnable negative background hint mechanism, constructing a bidirectional feature verification system. Negative hints are used to explicitly suppress common interference such as clouds and shadows, effectively filtering out abnormal predictions that conflict between fine-grained semantics and the global background. This helps improve the reliability of out-of-distribution detection, avoiding misdetection of background clutter as out-of-distribution samples or misjudging out-of-distribution samples containing interference as background. This enhances the ability of methods such as remote sensing image out-of-distribution detection models to distinguish between "different but similar" within-distribution samples and "truly unknown" out-of-distribution samples.

[0078] Based on the above embodiments, as an optional embodiment, acquiring the spatial feature map of the remote sensing image to be detected includes: The remote sensing image to be detected is input into the image encoder of the remote sensing image distribution external detection model to obtain the initial spatial feature map output by the image encoder; The initial spatial feature map is processed based on a dual-branch attention mechanism to obtain the spatial attention weight matrix of the initial spatial feature map; Based on Gaussian weighting and the spatial attention weight matrix, neighborhood information aggregation is performed on the feature points of the initial spatial feature map to obtain the spatial feature map.

[0079] It should be noted that each element in the spatial attention weight matrix represents the spatial attention weight of a feature point at a certain location in the initial spatial feature map, and different elements in the spatial attention weight matrix correspond to feature points at different locations in the initial spatial feature map. For example, the element at the first position in the spatial attention weight matrix... Line number The matrix elements of the column correspond to the first column in the initial spatial feature map. Line number The characteristic points of the column.

[0080] Specifically, the remote sensing image to be detected is input into the image encoder of the remote sensing image distribution out-of-distribution detection model, and the dimension of the image encoder output is obtained. Initial spatial feature map .

[0081] Furthermore, a dual-branch attention mechanism is used to process the initial spatial feature map, calculate the spatial attention weight of each feature point in the initial spatial feature map, and obtain the spatial attention weight matrix of the initial spatial feature map, thereby identifying key regions in the initial spatial feature map.

[0082] Based on Gaussian weighted spatial weight function smoothing and optimization of spatial attention weight matrix, neighborhood information aggregation processing is performed on feature points of the initial spatial feature map to obtain spatial feature map. The feature value of each feature point on the spatial feature map aggregates the neighborhood information of the feature point's neighboring feature points, so that the spatial distribution of each feature point in the spatial feature map not only considers the importance of individual pixels, but also the overall situation of its surrounding area, thereby capturing multi-scale relationships.

[0083] Furthermore, the neighboring feature points of each feature point can be all or part of the other feature points within a 3x3 grid centered on that feature point, or all or part of the other feature points within a 25x2 grid centered on that feature point; the present invention does not limit this.

[0084] By leveraging the physical properties of Gaussian distribution, the environmental context information around the target is weighted and integrated, enabling the model to not only focus on the target itself in subsequent steps, but also perceive its surrounding environment, which is something that single spatial adaptation cannot achieve.

[0085] Optionally, the initial spatial feature map is processed based on a dual-branch attention mechanism to obtain a spatial attention weight matrix of the initial spatial feature map, including: performing average pooling on the initial spatial feature map in the channel dimension to obtain an average pooled feature map; performing max pooling on the initial spatial feature map in the channel dimension to obtain a max pooled feature map; concatenating the average pooled feature map and the max pooled feature map to obtain a concatenated feature map; and performing a convolution operation on the concatenated feature map to obtain the spatial attention weight matrix of the initial spatial feature map.

[0086] In one embodiment, the expression for the matrix elements of the spatial attention weight matrix corresponding to each feature point of the initial spatial feature map is as follows: ; in, For the initial spatial feature map, the first Line number The spatial attention weights corresponding to the feature points in the column, that is, the th column in the spatial attention weight matrix. Line number Column matrix elements; For activation functions; The size of the convolutional block in the convolutional layer, for example ; This is a convolution operation; For connection operation; This is an average pooling operation; This is a max pooling operation; This is the initial spatial feature map.

[0087] In one embodiment, the formula for calculating the spatial aggregation score of each feature point in the spatial feature map is as follows: ; in, For the spatial feature map, the first Line number Spatial aggregation score of feature points in the column; For the initial spatial feature map, the first Line number The set of neighborhood feature points of a column's feature points; For the initial spatial feature map, the first Line number The spatial attention weights of the feature points in the column, i.e. ; To use a Gaussian-weighted spatial weighting function To highlight the contribution of the central position.

[0088] Typical remote sensing image out-of-distribution detection methods only adjust the feature scale using a spatial adapter during the feature extraction stage.

[0089] The remote sensing image out-of-distribution detection method provided by this invention introduces an explicit spatial feature enhancement mechanism based on Gaussian context aggregation during the spatial feature map extraction stage. It calculates spatial attention weights through a bi-branch attention mechanism and further combines neighborhood information to calculate aggregated spatial scores to capture multi-scale relationships. This extracts and enhances spatial perception features that can distinguish local anomalies, thereby enhancing spatial perception capabilities, effectively identifying local anomalies, and alleviating the problem of judging large scale variations of remote sensing ground objects. It also solves the problem of missed detection of localized OOD patterns.

[0090] Based on the above embodiments, as an optional embodiment, the image encoder is obtained by jointly training the text encoder and the image encoder with the goal of maximizing the first similarity and minimizing the second similarity; the first similarity is the similarity between the predicted spatial feature map and the predicted text feature map of the matching image-text pair sample; the second similarity is the similarity between the predicted spatial feature map and the predicted text feature map of the unmatched image-text pair sample. The joint training loss function is determined based on image-to-text loss and text-to-image loss.

[0091] Before using the VLMs-based remote sensing image out-of-distribution detection model for out-of-distribution detection, it is necessary to pre-train the remote sensing image out-of-distribution detection model (including the image encoder) using multiple in-distribution image-text pair annotation samples.

[0092] It should be noted that an in-distribution image-text pair annotation sample consists of image-text pair samples and their corresponding in-distribution labels. Image-text pair samples include remote sensing image samples and their corresponding text description samples, while in-distribution labels include the target category labels (e.g., airplane, car, etc.) corresponding to the image-text pair samples. The in-distribution labels of the in-distribution image-text pair annotation samples are generally obtained manually and are relatively few in number.

[0093] It should be noted that for a certain distribution of image-text pair annotation samples, the remote sensing image samples and their corresponding text description samples can be matched, in which case the image-text pair samples in the image-text pair annotation samples within the distribution are matched image-text pair samples; the remote sensing image samples and their corresponding text description samples can also be mismatched, in which case the image-text pair samples in the image-text pair annotation samples within the distribution are mismatched image-text pair samples.

[0094] Predicted spatial feature map is a feature embedding obtained by inputting remote sensing image samples from the distributed image-text pair annotation samples into the image encoder; predicted text feature map is a feature embedding obtained by inputting matched image-text pair samples from the distributed image-text pair annotation samples into the text encoder.

[0095] Specifically, in the process of pre-training the remote sensing image out-of-distribution detection model using in-distribution image-text pair labeled samples, training is performed through bidirectional comparison learning between the text encoder and the image encoder to align visual and text features. This includes maximizing the similarity between the predicted spatial feature map and the predicted text feature map of the matching image-text pair samples and minimizing the similarity between the predicted spatial feature map and the predicted text feature map of the mismatched image-text pair samples. The text encoder and the image encoder are jointly trained according to a joint training loss function determined based on image-to-text loss and text-to-image loss to complete the pre-training of the image encoder.

[0096] In one embodiment, the expression for the joint training loss function is as follows: ; ; ; in, Losses due to joint training; Image-to-text loss; Text-to-image loss; The number of image-text pair samples contained in a training batch; For the first training batch Predicted spatial feature map of labeled samples of text-image pairs within the distribution; For the first training batch Predicted text feature maps of labeled samples of text-image pairs within a distribution; The temperature parameter is learnable; Let be the cosine similarity.

[0097] The remote sensing image out-of-distribution detection method provided by the present invention, by jointly training the image encoder with the goal of maximizing the similarity of matching image-text pairs and minimizing the similarity of all mismatched image-text pairs, enables the image encoder to learn to align visual features and semantic concepts in a high-dimensional space, so as to have zero-shot classification capability.

[0098] Based on the above embodiments, as an optional embodiment, the global scene text prompt and the specific category text prompt are obtained by jointly optimizing the initial global scene text prompt feature map and the initial specific category text prompt feature map based on a composite loss function; The composite loss function is determined based on the global loss function, the category loss function, and the diversity loss function.

[0099] The initial global scene text prompt feature map is a global scene prompt feature extracted directly from the text description samples of image-text pairs of all distributed image-text pair labeled samples; the initial specific category text prompt feature map is a specific category prompt feature extracted directly from the text description samples of image-text pairs of distributed image-text pair labeled samples of a specific category.

[0100] The global loss function is used to optimize global scene text prompts and specific category text prompts to achieve ID / OOD binary classification; the category loss function is used to optimize global scene text prompts and specific category text prompts to achieve intra-ID multi-class classification; the diversity loss function is used to maintain the representational diversity between global scene text prompts and specific category text prompts.

[0101] In one embodiment, the formula for calculating the composite loss function is as follows: ; in, This is a composite loss; This results in a global loss. The weights for the global loss; For category loss; The weights for the category loss; For the loss of diversity; The weight for diversity loss.

[0102] In one embodiment, the expression for the diversity loss function is as follows: ; in, For the loss of diversity; This refers to the number of text prompts in the global scene. The number of text prompts for a specific category; Calculate the cosine similarity; For the first Feature maps of activated global scene text prompts; For the first Feature map of a specific category of activated text prompt.

[0103] By forcing the global scene text prompts and specific category text prompts to be orthogonal through the aforementioned diversity loss function, the representation diversity between the global scene text prompts and specific category text prompts can be maintained.

[0104] The remote sensing image out-of-distribution detection method provided by this invention jointly optimizes the initial global scene text prompt feature map and the initial specific category text prompt feature map by using a composite loss function determined based on a global loss function, a category loss function, and a diversity loss function. This results in global scene text prompts and specific category text prompts that can achieve ID / OOD binary classification, ID intra-class multi-class classification, and diversity representation. This enables the introduction of a dual-prompt spatial semantic alignment mechanism containing two sets of learnable text prompts for consistency verification, which helps improve the reliability of out-of-distribution detection. This, in turn, enhances the ability of methods such as remote sensing image out-of-distribution detection models to distinguish between "different but similar" in-distribution samples and "truly unknown" out-of-distribution samples.

[0105] Based on the above embodiments, as an optional embodiment, the initial global scene text prompt feature map is determined based on the mean of global scene text features and learnable perturbation terms; the mean of global scene text features is determined based on the text description samples of all in-distribution image-text pair annotation samples; The initial specific category text prompt feature map is determined based on the specific category text centroid features and optimized vectors; the specific category text centroid features are determined based on the text description samples of the specific category of the distributed text-image pair annotation samples in all distributed text-image pair annotation samples.

[0106] Specifically, before jointly optimizing the initial global scene text prompt feature map and the initial specific category text prompt feature map, it is necessary to first obtain the initial global scene text prompt feature map and the initial specific category text prompt feature map according to the set initialization strategy.

[0107] To accelerate convergence and introduce prior knowledge when obtaining the initial global scene text prompt feature map, instead of using a random initialization strategy, the mean of the global scene text features is calculated using the text description samples of all distributed text-image pairs labeled as a small number of samples, and a learnable perturbation term is superimposed. The initial global scene text prompt feature map is determined by a specific initialization strategy.

[0108] In determining the mean value of global scene text features, the text description samples of all distributed text-image pair annotation samples can be input into the text encoder one by one. The mean value of the multiple text feature maps output by the text encoder is obtained by adding them together.

[0109] When obtaining the initial feature map of text prompts for a specific category, a category-based prototype construction strategy is adopted. Based on the text description samples of the text-text pairs labeled in all distributed text-text pairs, the centroid features of the specific category of the text are determined. An optimization vector is introduced to adjust the execution features to obtain the initial feature map of text prompts for a specific category.

[0110] In determining the centroid features of a specific category of text, the text description samples of the distributed text-image pair annotation samples of the specific category can be input one by one into the text encoder. Based on the multiple text feature maps output by the text encoder, the centroid features of the specific category of text can be obtained.

[0111] In one embodiment, the expression for the initial global scene text hint feature map is as follows: ; in, This forms the initial global scene text prompt feature map; The mean of global scene text features; For learnable perturbation terms; It follows a normal distribution; For variance; It is an identity matrix.

[0112] In one embodiment, the expression for the initial category-specific text cue feature map is as follows: ; in, For the first Initial category-specific text hint feature map; For the first Centroid features of specific categories of text in labeled samples of image-text pairs within class distribution; For the first Optimized vectors for the class.

[0113] The remote sensing image out-of-distribution detection method provided by this invention obtains initial global scene text prompt feature maps and specific category text centroid features by employing specific initialization strategies before pre-training global scene text prompts and specific category text prompts. This includes obtaining initial global scene text prompt feature maps based on the mean of global scene text features and learnable perturbation terms, and obtaining initial specific category text prompt feature maps based on specific category text centroid features and optimized vectors. This method can introduce prior knowledge to accelerate convergence and improve the accuracy of spatial-semantic consistency verification based on the dual prompt alignment mechanism.

[0114] Based on the above embodiments, as an optional embodiment, the remote sensing image distribution out-of-distribution detection model is obtained by training on in-distribution image-text pair labeled samples and automatically labeled image-text pair samples; The automatically labeled image-text pairs are determined based on unlabeled image-text pairs whose predicted entropy is less than a preset entropy threshold, whose predicted consistency score is higher than a preset consistency score threshold, and whose predicted confidence is greater than an adaptive confidence threshold.

[0115] It should be noted that the in-distribution image-text pair labeled samples consist of image-text pair samples and their corresponding manually labeled in-distribution samples; the unlabeled image-text pair samples only include image-text pair samples; and the automatically labeled image-text pair samples consist of image-text pair samples and their corresponding automatically labeled pseudo-labels determined by the prediction classification results output by the remote sensing image out-of-distribution detection model trained according to the initialization prompts.

[0116] Prediction entropy is an indicator used to measure the uncertainty of the predicted classification results obtained by the remote sensing image out-of-distribution detection model for unlabeled image-text pairs.

[0117] The prediction consistency score is an intradistribution consistency score determined based on the similarity between feature points of the spatial feature map of the remote sensing image sample in the unlabeled image-text pair sample and the global scene text prompt and the specific category text prompt, respectively.

[0118] This invention proposes a self-training mechanism guided by confidence, prediction entropy, and consistency score to address the data scarcity problem in training remote sensing image out-of-distribution detection models in scenarios with few samples.

[0119] Specifically, in combination Figure 2 As shown, in the pre-training process of the remote sensing image out-of-distribution detection model, initial prompt training is first performed on a small number of in-distribution image-text pair annotation samples, including training the image encoder and text encoder of the remote sensing image out-of-distribution detection model, global scene text prompts and specific category text prompts.

[0120] Furthermore, after initialization prompt training, the remote sensing image out-of-distribution detection model predicts unlabeled image-text pairs from the unlabeled data pool, obtaining the predicted classification results output by the remote sensing image out-of-distribution detection model. It also obtains the in-distribution consistency score, i.e., the prediction consistency score, which is determined by the similarity between the feature points of the spatial feature maps of the remote sensing image samples in the unlabeled image-text pairs and the global scene text prompts and specific category text prompts, respectively, during the prediction process of the remote sensing image out-of-distribution detection model.

[0121] Based on the prediction classification results, unlabeled image-text pairs are selected that have a prediction entropy less than a preset entropy threshold, a prediction consistency score higher than a preset consistency score threshold (also known as a safety threshold), and a prediction confidence greater than an adaptive confidence threshold. These are then used as image-text pairs for automatic labeling. The in-distribution labels (including target category labels), or pseudo-labels, of the automatically labeled image-text pairs are determined based on the target category with the highest confidence in the prediction classification results of these unlabeled image-text pairs.

[0122] Finally, an augmented training sample set was constructed by combining the in-distribution image-text pair labeled samples and the automatically labeled image-text pair samples. The remote sensing image out-of-distribution detection model was then retrained on the augmented training sample set to obtain the pre-trained remote sensing image out-of-distribution detection model.

[0123] In one embodiment, the formula for calculating the prediction entropy is as follows: ; in, To predict entropy; These are remote sensing image samples from unlabeled image-text pairs. The predicted classification result output by the remote sensing image distribution out-of-range detection model is a vector containing the predicted probabilities of all categories. This represents the total number of known in-distribution categories in the remote sensing image distribution detection model. Remote sensing image samples Category The conditional probability; This is the preset entropy threshold.

[0124] In one embodiment, the adaptive confidence threshold is calculated using the following formula: ; in, An adaptive confidence threshold; The mean confidence score of correctly classified samples on the training set; The standard deviation of the confidence scores for correctly classified samples on the training set; This is a hyperparameter.

[0125] In one embodiment, the expression for the automatically labeled image-text pair sample set (pseudo-label dataset) is as follows: ; in, For automatically labeled image-text pair sample sets; For unlabeled data pools; These are remote sensing image samples from unlabeled image-text pairs. To utilize the remote sensing image out-of-distribution detection model trained with initialization prompts for remote sensing image samples The predicted classification results; To predict confidence levels; An adaptive confidence threshold; To predict the consistency score; Set a preset consistency score threshold.

[0126] By automatically labeling text-image pairs based on unlabeled text-image pairs whose prediction confidence is greater than an adaptive confidence threshold and whose prediction consistency score is greater than a preset consistency score threshold, it is possible to ensure that the construction of the pseudo-label dataset has prediction consistency under the data augmentation view.

[0127] In one embodiment, ensuring that the prediction consistency score is greater than a preset consistency score threshold and less than a safety threshold ensures that the automatically labeled images and text not only have a clear classification of the samples, but also strictly conform to the consistency characteristics of the distribution (ID) in logic, avoiding the misuse of "high-confidence unknown anomalies" as positive samples for training, and preventing "overconfidence" noise pollution.

[0128] In one embodiment, the loss function for training the remote sensing image out-of-distribution detection model using in-distribution image-text pair labeled samples and automatically labeled image-text pair samples is determined based on labeled loss, pseudo-label loss and regularization loss.

[0129] In one embodiment, the loss function for training the out-of-distribution detection model of remote sensing images using in-distribution image-text pair labeled samples and automatically labeled image-text pair samples is as follows: ; in, This represents the total loss value. This is a sample set of labeled text-image pairs within the distribution. Label loss is the in-distribution multi-class cross-entropy loss calculated on the original in-distribution image-text pair labeled sample set, used to measure the model's performance on the "standard answer"; The pseudo-label loss is the in-distribution multi-class cross-entropy loss calculated on automatically labeled image-text pairs, used to measure the model's performance on "pseudo-answers". The weighting coefficients for the pseudo-label loss are adjustable hyperparameters. This is a regularization loss term used to prevent the model from overfitting on the training data. By adding a penalty term to the model's parameters, the complexity of the model is limited, thereby improving the model's generalization ability on unseen data. is the weight coefficient of the regularization loss, used to control the strength of the regularization penalty, and is an adjustable hyperparameter.

[0130] By adjusting the weighting coefficients of the pseudo-label loss in the loss function, pseudo-labels can be deweighted to suppress noise.

[0131] The remote sensing image out-of-distribution detection method provided by this invention deeply integrates the self-training method with the OOD detection task by combining a spatially semantically consistent, confidence-guided self-training loop strategy. Without the need for manual annotation, it uses the high-quality confidence signal provided by the double-cue alignment mechanism to dynamically filter and mine reliable pseudo-labels from massive unlabeled data to expand the training set. This ensures that potential OOD samples can be effectively filtered out while expanding the training data, solving the problem of OOD detection with few samples and improving the performance and robustness of the remote sensing image out-of-distribution detection model in scenarios with few samples.

[0132] Table 1 is a comparison table of the accuracy of different methods provided by the present invention, and Table 2 is a comparison table of ablation experiments provided by the present invention. Figure 3 This is a separation effect diagram provided by the present invention. Figure 4 This is a comparison chart of different sample numbers provided by the present invention.

[0133] Table 1

[0134] Table 2

[0135] The remote sensing image out-of-distribution detection method provided by this invention has achieved significant performance improvements on multiple remote sensing OOD detection benchmark datasets, especially under the condition of few-sample training, it can significantly improve detection performance, effectively separate ID / OOD samples and effectively improve the few-sample learning ability.

[0136] Regarding the significant improvement in detection performance, as shown in Table 1, the framework of this invention, acting as a feature extractor, can significantly improve the performance of various baseline OOD detection methods (such as MSP, Energy, GradNorm, etc.). For example, under specific settings on the UCMerced dataset, the maximum improvements in AUROC (a metric for overall performance) and AUPR (a metric for performance on imbalanced data) can reach 10.03% and 10.09%, respectively. On the NWPU-RESISC45 dataset, when used in conjunction with the GradNorm method, AUROC increased from 55.19% without enhancement to 68.11%.

[0137] In terms of effective ID / OOD sample separation, combined with Figure 3 As shown, compared to the significant overlap in the score distributions of ID and OOD samples in the baseline model RemoteCLIP, the method of this invention can significantly push the scores of ID samples to the low-score region, while OOD samples remain in the high-score region, forming a clear separation boundary, thereby reducing the misclassification rate.

[0138] Regarding effectively improving few-shot learning ability, combining Table 2 and Figure 4 As shown, ablation experiments demonstrate the effectiveness of each component of this invention. The complete framework (including self-training) achieves an average AUROC of 65.25% across all test datasets, significantly outperforming the 44.14% achieved using only the baseline model RemoteCLIP. The experiments also revealed that the 4-shot training configuration achieves optimal performance across all datasets, demonstrating the effectiveness and robustness of this invention under conditions of extreme data scarcity.

[0139] Overall, the remote sensing image out-of-destination (OOD) detection method provided by this invention can be built on a visual language basic model specialized for the remote sensing field. By introducing three core modules—spatial feature enhancement, dual cue alignment, and confidence-guided self-training—it can solve the deficiencies in identifying local anomalies and learning with few samples. Under conditions of scarce data (few samples), it can perform robust and accurate remote sensing image OOD detection.

[0140] Figure 5 This is a schematic diagram of the structure of the remote sensing image distribution detection device provided by the present invention, as shown below. Figure 5 As shown, the remote sensing image distribution detection device includes, but is not limited to, a spatial feature map acquisition module 501, a global similarity determination module 502, a category similarity determination module 503, a background similarity determination module 504, a consistency score determination module 505, and a detection result determination module 506.

[0141] The spatial feature map acquisition module 501 is used to acquire the spatial feature map of the remote sensing image to be detected.

[0142] The global similarity determination module 502 is used to determine the global similarity of the feature points based on the similarity between the pre-trained global scene text prompts and the feature points of the spatial feature map.

[0143] The category similarity determination module 503 is used to determine the category similarity of the feature points based on the similarity between the pre-trained specific category text prompts and the feature points of the spatial feature map.

[0144] Background similarity determination module 504 is used to determine the background similarity of the feature points based on the similarity between the negative background text prompt and the feature points of the spatial feature map; the negative background text prompt is determined based on a general background interference description.

[0145] The consistency score determination module 505 is used to determine the distribution consistency score of the feature points based on the global similarity of the feature points, the category similarity of the feature points, and the background similarity of the feature points.

[0146] The detection result determination module 506 is used to determine the out-of-distribution detection result of the remote sensing image to be detected based on the in-distribution consistency score of the feature points.

[0147] It should be noted that the remote sensing image distribution out-of-region detection device provided by the present invention can execute the remote sensing image distribution out-of-region detection method described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0148] The remote sensing image out-of-distribution detection device provided by this invention performs consistency verification by introducing a dual-hint spatial semantic alignment mechanism containing two sets of learnable text hints. It designs parallel global scene text hints and specific category text hints, taking into account fine-grained spatial details and local information in the remote sensing image through specific category text hint alignment. Image content is cross-validated by comparing the global context of the scene and the fine-grained category semantics. Furthermore, the spatial features of the remote sensing image to be detected must be aligned with both the global scene text hints and the specific category text hints to be considered within the distribution. Based on this, the invention further introduces a learnable negative background hint mechanism, constructing a bidirectional feature verification system. Negative hints are used to explicitly suppress common interference such as clouds and shadows, effectively filtering out abnormal predictions that conflict between fine-grained semantics and the global background. This helps improve the reliability of out-of-distribution detection, avoiding misdetection of background clutter as out-of-distribution samples or misjudging out-of-distribution samples containing interference as background. This enhances the ability of methods such as remote sensing image out-of-distribution detection models to distinguish between "different but similar" within-distribution samples and "truly unknown" out-of-distribution samples.

[0149] Figure 6This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communications bus 640. The processor 610 can call logical instructions in the memory 630 to execute the remote sensing image out-of-distribution detection method provided in any of the above embodiments. The remote sensing image out-of-distribution detection method includes, but is not limited to, the following steps: acquiring a spatial feature map of the remote sensing image to be detected; determining the global similarity of the feature points based on the similarity between pre-trained global scene text prompts and feature points of the spatial feature map; determining the category similarity of the feature points based on the similarity between pre-trained specific category text prompts and feature points of the spatial feature map; determining the background similarity of the feature points based on the similarity between negative background text prompts and feature points of the spatial feature map; the negative background text prompts are determined based on a general background interference description; determining the intra-distribution consistency score of the feature points based on the global similarity, category similarity, and background similarity; and determining the out-of-distribution detection result of the remote sensing image to be detected based on the intra-distribution consistency score.

[0150] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0151] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the remote sensing image out-of-distribution detection method provided in any of the above embodiments. The remote sensing image out-of-distribution detection method includes, but is not limited to, the following steps: acquiring a spatial feature map of the remote sensing image to be detected; determining the global similarity of the feature points based on the similarity between pre-trained global scene text prompts and feature points of the spatial feature map; determining the category similarity of the feature points based on the similarity between pre-trained specific category text prompts and feature points of the spatial feature map; determining the background similarity of the feature points based on the similarity between negative background text prompts and feature points of the spatial feature map; the negative background text prompts are determined based on a general background interference description; determining the distributional consistency score of the feature points based on the global similarity, the category similarity, and the background similarity; and determining the out-of-distribution detection result of the remote sensing image to be detected based on the distributional consistency score.

[0152] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the remote sensing image out-of-distribution detection method provided in any of the above embodiments. The remote sensing image out-of-distribution detection method includes, but is not limited to, the following steps: acquiring a spatial feature map of the remote sensing image to be detected; determining the global similarity of the feature points based on the similarity between pre-trained global scene text prompts and feature points of the spatial feature map; determining the category similarity of the feature points based on the similarity between pre-trained specific category text prompts and feature points of the spatial feature map; determining the background similarity of the feature points based on the similarity between negative background text prompts and feature points of the spatial feature map; wherein the negative background text prompts are determined based on a general background interference description; determining the distributional consistency score of the feature points based on the global similarity, the category similarity, and the background similarity; and determining the out-of-distribution detection result of the remote sensing image to be detected based on the distributional consistency score.

[0153] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0154] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting out-of-distribution features in remote sensing images, characterized in that, include: Acquire spatial feature maps of the remote sensing image to be detected; The global similarity of the feature points is determined based on the similarity between the pre-trained global scene text prompts and the feature points of the spatial feature map. The category similarity of the feature points is determined based on the similarity between the pre-trained specific category text prompts and the feature points of the spatial feature map; The background similarity of the feature points is determined based on the similarity between the negative background text prompt and the feature points of the spatial feature map; The negative background text prompt is determined based on a general background interference description; The distributional consistency score of the feature points is determined based on the global similarity, the category similarity, and the background similarity of the feature points. Based on the in-distribution consistency score of the feature points, the out-of-distribution detection result of the remote sensing image to be detected is determined.

2. The remote sensing image distribution detection method according to claim 1, characterized in that, The global scene text prompts and the specific category text prompts are obtained by jointly optimizing the initial global scene text prompt feature map and the initial specific category text prompt feature map based on a composite loss function. The composite loss function is determined based on the global loss function, the category loss function, and the diversity loss function.

3. The remote sensing image distribution detection method according to claim 2, characterized in that, The initial global scene text prompt feature map is determined based on the mean of global scene text features and learnable perturbation terms; the mean of global scene text features is determined based on the text description samples of all in-distribution image-text pair annotation samples; The initial specific category text prompt feature map is determined based on the specific category text centroid features and optimized vectors; the specific category text centroid features are determined based on the text description samples of the specific category of the distributed text-image pair annotation samples in all distributed text-image pair annotation samples.

4. The remote sensing image distribution out-of-distribution detection method according to any one of claims 1-3, characterized in that, The acquisition of the spatial feature map of the remote sensing image to be detected includes: The remote sensing image to be detected is input into the image encoder of the remote sensing image distribution external detection model to obtain the initial spatial feature map output by the image encoder; The initial spatial feature map is processed based on a dual-branch attention mechanism to obtain the spatial attention weight matrix of the initial spatial feature map; Based on Gaussian weighting and the spatial attention weight matrix, neighborhood information aggregation is performed on the feature points of the initial spatial feature map to obtain the spatial feature map.

5. The remote sensing image distribution detection method according to claim 4, characterized in that, The image encoder is obtained by jointly training the text encoder and the image encoder with the goal of maximizing the first similarity and minimizing the second similarity; the first similarity is the similarity between the predicted spatial feature map and the predicted text feature map of the matching image-text pair sample; the second similarity is the similarity between the predicted spatial feature map and the predicted text feature map of the unmatched image-text pair sample. The joint training loss function is determined based on image-to-text loss and text-to-image loss.

6. The remote sensing image distribution out-of-distribution detection method according to any one of claims 1-3, characterized in that, The remote sensing image distribution out-of-distribution detection model is trained using in-distribution image-text pair labeled samples and automatically labeled image-text pair samples. The automatically labeled image-text pairs are determined based on unlabeled image-text pairs whose predicted entropy is less than a preset entropy threshold, whose predicted consistency score is higher than a preset consistency score threshold, and whose predicted confidence is greater than an adaptive confidence threshold.

7. A remote sensing image distribution detection device, characterized in that, include: The spatial feature map acquisition module is used to acquire the spatial feature map of the remote sensing image to be detected. The global similarity determination module is used to determine the global similarity of the feature points based on the similarity between the pre-trained global scene text prompts and the feature points of the spatial feature map. The category similarity determination module is used to determine the category similarity of the feature points based on the similarity between the pre-trained specific category text prompts and the feature points of the spatial feature map; The background similarity determination module is used to determine the background similarity of the feature points based on the similarity between the negative background text prompt and the feature points of the spatial feature map; The negative background text prompt is determined based on a general background interference description; The consistency score determination module is used to determine the distribution consistency score of the feature points based on the global similarity of the feature points, the category similarity of the feature points, and the background similarity of the feature points. The detection result determination module is used to determine the out-of-distribution detection result of the remote sensing image to be detected based on the in-distribution consistency score of the feature points.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the remote sensing image distribution out-of-distribution detection method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the remote sensing image distribution out-of-distribution detection method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the remote sensing image distribution out-of-distribution detection method as described in any one of claims 1 to 6.

Images