Remote sensing image recognition model training method and system based on CLIP guidance
By generating sample difficulty scores and implementing a three-stage learning process using the CLIP model, this approach addresses the issues of weak domain adaptability and inaccurate sample difficulty assessment in remote sensing image understanding. It achieves efficient learning of remote sensing features and enhances robustness in complex scenes, making it suitable for tasks such as scene classification, object detection, change detection, and semantic segmentation.
Patent Information
- Application Number
- CN202511090240.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-05
- Publication Date
- 2025-11-21
AI Technical Summary
Existing technologies for understanding remote sensing images suffer from problems such as weak domain adaptability, inaccurate assessment of sample difficulty, and lack of semantic guidance, making it difficult to effectively utilize the semantic complexity and multi-scale characteristics of remote sensing data.
The CLIP model is used to generate sample difficulty scores. Combined with the sample set of the three-stage course, the robustness of the model is improved through progressive learning, dynamic updating of easy and difficult samples, and feature association mechanism, so as to achieve efficient learning of remote sensing features.
By using cross-modal similarity measurement and dynamic sample weighting, the complexity of samples is accurately characterized, improving the model's generalization ability and robustness to complex remote sensing scenes. It is applicable to downstream tasks such as scene classification, object detection, change detection, and semantic segmentation.
Smart Images
Figure CN120997644A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of computer vision, and particularly relates to a CLIP-guided remote sensing image recognition model training method and system. BACKGROUND
[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.
[0003] In recent years, high-resolution remote sensing images have been increasingly widely used in the fields of city planning, environmental monitoring, disaster assessment, and even military affairs. However, there are large domain differences in the categories, colors, textures, and shapes of targets contained in remote sensing images and natural images. Most of the current target detection backbone networks use pre-training networks in the field of computer vision, which can be fine-tuned based on remote sensing target datasets. However, the domain gap between remote sensing images and general images limits the fine-tuning performance of computer vision pre-training networks in specific remote sensing scenarios, and the system bias introduced by computer vision pre-training networks also has an adverse effect on remote sensing image target detection.
[0004] Pre-training methods are mainly divided into supervised learning and self-supervised learning: In 2019, the AAAI Artificial Intelligence Conference published an algorithm named "Tile2Vec: Unsupervised Representation Learning for Spatially Distributed Data". Inspired by Word2Vec in natural language processing, this method assumes that geographically adjacent remote sensing tiles have semantic similarity and achieves unsupervised feature extraction through metric learning. Specifically, it uses geographical coordinates to associate adjacent tiles and constructs a "spatial context" supervision signal to enable the model to learn a feature representation with spatial consistency. This method overcomes the defect of traditional unsupervised methods that rely on manual task design and directly uses the spatial distribution characteristics of remote sensing data for self-supervised learning, providing a basic framework for subsequent pre-training based on geographical information.
[0005] In the prior art, some studies attempt to solve the remote sensing pre-training problem through self-supervised learning. In 2021, IEEE International Conference on Computer Vision (ICCV) published an algorithm for unsupervised pre-training of remote sensing data, named "Seasonal Contrast: Unsupervised Pre-training from Uncurated Remote Sensing Data" in Chinese. The specific method of this method is to use remote sensing data of the same geographical location in different seasons as natural contrast samples to construct an unsupervised pre-training framework, extract season-invariant features and change features through contrast learning, and achieve model pre-training without manual annotation data. This method overcomes the shortcomings of traditional unsupervised pre-training methods that rely on manual design of tasks or specific data enhancement strategies, and uses the time series characteristics of remote sensing data itself to realize self-supervised learning and improve the generalization ability of the model to remote sensing scenes. However, these methods often ignore the semantic complexity and multi-scale characteristics of remote sensing data. In addition, traditional curriculum learning methods are mostly based on data-driven difficulty assessment, and lack effective use of semantic information of remote sensing scenes, making it difficult to accurately depict the complexity of samples.
[0006] In summary, the prior art has the problems of weak domain adaptation ability, inaccurate sample difficulty assessment, and lack of semantic guidance in remote sensing image understanding. SUMMARY
[0007] To solve at least one of the technical problems in the above background art, the present application provides a CLIP-guided remote sensing image recognition model training method and system, which generates sample difficulty scores based on the CLIP model and divides the sample set of three-stage curriculum learning. Gradual learning of three-stage curriculum learning sample set, combined with dynamic updating of difficult and easy samples and feature correlation mechanism, not only retains the gradualness of curriculum learning, but also enhances difficult sample learning through correlation transmission, improving the robustness of the model; without manual annotation, efficient learning of remote sensing features is realized, and the generalization ability of the model to complex remote sensing scenes is improved.
[0008] To achieve the above purpose, the present application adopts the following technical solutions: The first aspect of the present application provides a CLIP-guided remote sensing image recognition model training method, comprising: Constructing an image dataset, preprocessing the image dataset; obtaining an image feature vector and a text feature vector based on the dataset; the text feature vector includes a difficult text feature vector and a simple text feature vector; a first similarity between the image feature vector and the difficult text feature vector is calculated, and a second similarity between the image feature vector and the simple text feature vector is calculated; and the difference between the first similarity and the second similarity is calculated as a difficulty score; based on the difficulty score, the image feature vector can be divided into a low difficulty sample and a high difficulty sample; A remote sensing image recognition model is constructed, and the remote sensing image recognition model is trained in three stages using different difficulty samples, specifically including: in the first stage, low difficulty samples are used to train the model in the first stage; in the second stage, low difficulty samples and high difficulty samples are mixed to train the model in the second stage; in the third stage, high difficulty samples are used to train the model in the third stage; and finally, a trained remote sensing image recognition model is obtained.
[0009] As an implementation, the calculation formula of the first similarity similarity[dif_cls] between the image feature vector and the simple text feature vector is:
[0010] The calculation formula of the second similarity similarity[ez_cls] between the image feature vector and the complex text feature vector is:
[0011] wherein, the feature vector of the image, the simple text feature vector, the simple text feature vector, the dot product of the vectors; the difficulty score The calculation formula of the difficulty score is: .
[0012] As an implementation, based on the difficulty score, the image feature vector with a difficulty score ≥0.5 is divided into a high difficulty sample, and the image feature vector with a difficulty score <0.5 is divided into a low difficulty sample.
[0013] As an implementation, the first stage uses low difficulty samples to train the model in the first stage; specifically including: The first stage focuses on the basic training of low difficulty samples, and low difficulty samples are screened based on the difficulty score, and a cross-entropy loss function is used to help the model establish a basic remote sensing feature representation.
[0014] As an implementation, the second stage mixes low-difficulty samples and high-difficulty samples to perform second stage training on the model; specifically including: The second stage enters a balanced learning process, gradually introduces mixed difficulty samples with training rounds, and gradually transitions from a low-difficulty sample: high-difficulty sample ratio of 1:9 to a difficulty sample: high-difficulty sample ratio of 9:1, and calculates dynamic weights based on sample difficulty scores, adopts weighted cross-entropy loss to balance the learning process, so that the model can consolidate basic features while gradually adapting to complex scenarios, distinguishing similar categories and initially dealing with mild interference.
[0015] As an implementation, the third stage uses high-difficulty samples to perform third stage training on the model; specifically including: The third stage selects high-difficulty samples and switches to Focal Loss function at the same time, reduces the loss weight of easy-to-classify samples, and makes the model focus on learning high-difficulty samples.
[0016] The second aspect of the application provides a CLIP-guided remote sensing image recognition model training system based on the steps of the CLIP-guided remote sensing image recognition model training method provided in the first aspect of the application, comprising: An image dataset construction module configured to obtain an image dataset and pre-process the image dataset; A base feature vector acquisition module configured to obtain image feature vectors and text feature vectors based on the dataset; A similarity calculation module configured to calculate a first similarity between the image feature vectors and the difficult text feature vectors, and a second similarity between the image feature vectors and the simple text feature vectors; A difficulty score calculation module configured to calculate the difference between the first similarity and the second similarity; The difference is used as the difficulty score; based on the difficulty score, the image feature vectors can be divided into different difficulty samples; A model training module configured to build a remote sensing image recognition model and train the remote sensing image recognition model in three stages using different difficulty samples, specifically including: the first stage uses low-difficulty samples to perform first stage training on the model; the second stage mixes low-difficulty samples and high-difficulty samples to perform second stage training on the model; the third stage uses high-difficulty samples to perform third stage training on the model; A model output module: obtains the trained remote sensing image recognition model.
[0017] The third aspect of the application provides a computer readable storage medium.
[0018] A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of a CLIP-guided remote sensing image recognition model training method as described above.
[0019] A fourth aspect of the application provides a computer program product.
[0020] A computer program product comprising computer programs / instructions which, when executed by a processor, implement the steps of a CLIP-guided remote sensing image recognition model training method as described above.
[0021] A fifth aspect of the application provides an electronic device.
[0022] An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of a CLIP-guided remote sensing image recognition model training method as described above when executing the program.
[0023] Compared with the prior art, the application has the following beneficial effects: 1. The application utilizes the cross-modal alignment capability of the CLIP model, combines a three-stage training mechanism, and uses a pre-trained model for downstream remote sensing tasks, including scene classification, target detection, change detection, and semantic segmentation.
[0024] 2. The application measures the degree of association between the image and different semantic texts by calculating the similarity between the image feature vector and each text feature vector. Through this cross-modal similarity measurement, the association strength between the image and different complexity semantic texts can be accurately described, providing a quantitative basis for subsequent sample division based on difficulty scores.
[0025] 3. The application uses the similarity difference between the image feature vector and the different difficulty text feature vectors as a score to measure the difficulty of the sample. If the difficulty score is high, it means that the similarity between the image and the complex category text is much higher than that with the simple category text, which means that the sample contains more complex semantic features, such as blurred edges, severe background interference, and unobvious target features, etc., and thus determines that the sample is a difficult sample. Conversely, if the difficulty score is low, the sample is relatively simple. Through this rigorous evaluation process, the complexity of the sample can be accurately described.
[0026] Advantages of the additional aspects of the application will be partly given in the following description, partly will become apparent from the following description, or will be learned by practicing the application. BRIEF DESCRIPTION OF DRAWINGS
[0027] The accompanying drawings, which form a part of this specification, are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification. The embodiments of the application, and their
[0028] Figure 1 is a whole scheme flow chart of the application; Figure 2 is a CLIP model architecture diagram of the application; Figure 3 is a pre-training model network architecture diagram of the application; Figure 4 is a downstream task target detection visualization generation diagram of the application; Figure 5 is a downstream task change detection visualization generation diagram compared with other pre-training models. DETAILED DESCRIPTION
[0029] The application will be further described below in conjunction with the accompanying drawings and embodiments.
[0030] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the application belongs.
[0031] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit exemplary embodiments according to the application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and furthermore, it should be understood that when the terms "comprise" and / or "include" are used in the specification, there is a feature, step, operation, device, component and / or combination thereof.
[0032] Embodiment one As shown in Figure 1 The embodiment provides a CLIP-guided remote sensing image recognition model training method, which comprises the following steps: Step 1: Construct an image data set and pre-process the image data set; The constructed image dataset includes: selecting multi-resolution remote sensing images covering six continents, with resolution ranging from 0.1 meters to 30 meters, including panchromatic, multispectral and fused images; most of the images come from Gaofen-2 satellite, and the rest come from public datasets, ensuring the diversity and global coverage of the data. The original images captured by Gaofen-2 satellite are processed for radiation correction, geometric precision correction, etc. to eliminate atmospheric scattering, terrain distortion and other interference; fuzzy images are filtered through edge detection algorithm to ensure the effectiveness of the samples. The total dataset image size varies from 256x256 pixels to 5000x5000 pixels, taking into account satellite vertical shooting and aerial oblique shooting angles.
[0033] Step 2: Obtain image feature vectors and text feature vectors based on the image dataset; the text feature vectors include complex text feature vectors and simple text feature vectors.
[0034] According to the scene characteristics of remote sensing images, different difficulty levels of text feature vectors are manually divided: simple text feature vector acquisition: in addition to "clear satellite images", supplement "unobstructed road remote sensing images" and "high-resolution building outline images", etc., that is, images with clear target features and less background interference. Complex text feature vector acquisition: based on "low-quality remote sensing images", expand "cloud cover target images", "edge blurred small target images", "tilt angle dense building group" and other images, that is, images with fuzzy target features and high background complexity.
[0035] As shown in Figure 2 , image feature vector acquisition: after preprocessing (resize to 256x256, standardization), input the image encoder of the CLIP model (adopting ViT-B / 32 architecture), extract deep features through multiple layers of Transformer, generate 512-dimensional and L2 normalized image feature vectors. At the same time, input the text description into the text encoder after template normalization (such as "this is a remote sensing image of {text description}"), process through word embedding, position encoding and attention mechanism, generate text feature vectors of the same dimension and normalization, ensure the consistency of cross-modal feature measurement. Unified text input format: different text descriptions (such as "clear satellite image", "cloud cover target image", etc.) are converted into a consistent input form. This standardization avoids interference from differences in text expression methods (such as long and short sentences, different sentence structures) on the output features of the text encoder, ensuring that the text features are determined only by the core semantic content. The subsequent difficulty score calculation based on the similarity difference between image features and text features of different difficulties requires that the measurement standards of the two types of features be consistent. This step provides a basis for accurate calculation of difficulty scores through standardized text input and feature processing procedures, ensuring the reliability of the division results of high-difficulty samples and low-difficulty samples.
[0036] Step 3: Calculate the first similarity between the image feature vector and the complex text feature vector, the second similarity between the image feature vector and the simple text feature vector, and the difference between the first similarity and the second similarity, and take the difference as the difficulty score; specifically: By calculating the similarity between the image feature vector and the text feature vector of different difficulties, the association degree of the image and the different semantic texts is measured. For the characteristics of remote sensing scenes, this similarity calculation needs to cover multi-dimensional semantic association: for simple texts such as "clear satellite image", high similarity indicates that the image has easy-to-learn features such as target prominence and simple background; for complex texts such as "low-quality remote sensing image", high similarity reflects that the image has difficult-to-learn features such as target blur and background interference. Through this cross-modal similarity measurement, the association strength of the image and different complexity semantic texts can be accurately described, providing a quantitative basis for subsequent sample division based on the difficulty score.
[0037] The calculation of the similarity between the image feature vector and the text feature vector of different difficulties specifically includes: The calculation formula of the first similarity similarity[dif_cls] between the image feature vector and the simple text feature vector is: (1) The calculation formula of the second similarity similarity[ez_cls] between the image feature vector and the complex text feature vector is: (2) Wherein, is The feature vector of the image, is the simple text feature vector, is the simple text feature vector, is the dot product of the vector.
[0038] The calculation formula of the difficulty score is: (3) Step 4: Based on the difficulty score, the image feature vector with a difficulty score ≥0.5 is divided into a high-difficulty sample, and the image feature vector with a difficulty score Image feature vectors with a value less than 0.5 are classified as low-difficulty samples. The difference between the similarity of the image feature vector and the text feature vector of different difficulties is used as a score to measure the difficulty of the image sample. If the difficulty score is high, it means that the similarity of the image with the complex text feature vector is much higher than that with the simple text feature vector, indicating that the sample contains more complex semantic features, such as image edge blur, serious background interference, and unobvious target features, etc., and thus the sample is determined to be a high-difficulty sample. Conversely, if the difficulty score is low, the sample is relatively simple, i.e., a low-difficulty sample. Through this rigorous evaluation process, the complexity of the image sample can be accurately described.
[0039] Step 5: As shown in Figure 3 , a remote sensing image recognition model is constructed, and different difficulty samples are used to train the remote sensing image recognition model in three stages to finally obtain a trained remote sensing image recognition model. Specifically, it includes: The first stage uses low-difficulty samples for first-stage training to establish a basic representation. The first stage focuses on the basic training of low-difficulty samples, and based on the difficulty score, low-difficulty samples (such as high-definition buildings and unobstructed road images) are selected, and a cross-entropy loss function is used to help the model establish a basic remote sensing feature representation, such as distinguishing water from land and identifying obvious building outlines, etc., to lay a foundation for subsequent learning.
[0040] The cross-entropy loss is as follows: (4) The prediction probability of the model for the first class is , the encoding of the true label is 1 if it is the first class, and 0 otherwise.
[0041] The second stage mixes low-difficulty samples and high-difficulty samples to train the model in the second stage to balance the learning process.
[0042] In the second stage, the learning process is balanced, and mixed difficulty samples are gradually introduced with training rounds, with the sample ratio gradually transitioning from a low-difficulty sample: high-difficulty sample ratio of 1:9 to a difficulty sample: high-difficulty sample ratio of 9:1, and dynamic weights are calculated based on the sample difficulty score, and a weighted cross-entropy loss is used to balance the learning process, so that the model can consolidate the basic features while gradually adapting to complex scenarios, learning to distinguish similar categories and initially dealing with mild interference.
[0043] The weighted cross-entropy loss is as follows: (5) where is the prediction probability of the model for the first The predicted probability of the class, The encoding of the true label, The first The dynamic weight of the class sample.
[0044] The third stage uses high-difficulty samples to train the model in the third stage. The third stage strengthens the learning of high-difficulty samples through Focal Loss, and improves the adaptability of the model to complex scenes.
[0045] The third stage strengthens the learning of difficult samples to improve the adaptability of the model to complex scenes, selects difficult samples (such as edge blurred targets and complex background small targets), and switches to the Focal Loss function at the same time. By reducing the loss weight of easy-to-classify samples, the model focuses on difficult example learning, such as identifying small facilities in dense building groups, detecting road changes under cloud and fog interference, etc., to ultimately improve the robustness of the model to complex remote sensing scenes.
[0046] The Focal Loss loss is as follows: (6) wherein, is a "focus parameter" for adjusting the loss weight of easy-to-classify samples and difficult-to-classify samples. The greater the value, the lower the loss proportion of easy-to-classify samples, and the more attention the model pays to difficult-to-classify samples, highlighting the learning of difficult samples; is the probability of the model predicting that the current sample belongs to the true class, such as the probability of predicting "this is a building". The closer to 1, the more certain the model is that it is the class; is a weight coefficient for balancing the loss of positive and negative samples (such as "is a building" and "is not a building" positive and negative classes). It can give higher weight to the class with fewer samples, avoid the model due to uneven sample quantity, and make the loss of positive and negative samples reasonably affect the model learning.
[0047] The ResNet 50 pre-training weight obtained by the pre-trained remote sensing image recognition model is used for downstream remote sensing tasks. Specifically: in scene classification, EuroSAT and NWPU-RESISC45 datasets are used to verify the scene recognition ability of the model; in target detection, DIOR dataset is used to evaluate the performance of multiple detection models; in change detection, CDD and LEVIR datasets are used to evaluate the change recognition effect by F1 score; in semantic segmentation, LoveDA dataset is used to measure the segmentation accuracy by mIOU.
[0048] For example, Figure 4As shown, the first row shows the results of Faster R-CNN using CLIP-RSCLNet-ResNet50 on the DIOR dataset, the second row shows the results of RetinaNet using CLIP-RSCLNet-ResNet50 on the DIOR dataset, and the third row shows the results of DyHead using CLIP-RSCLNet-ResNet50 on the DIOR dataset.
[0049] like Figure 5 As shown, the visualization generated by comparing downstream task change detection with other pre-trained models is presented. The first and second rows are the change detection results of sample images from the LEVIR and CDD datasets, respectively. Among them, (a), (n), (b), and (o) are the first and second time phases of the same region. (c) and (p) are the ground truth change annotations. (d) and (p), (e) and (r), (f) and (s), (g) and (t), (h) and (u), (i) and (v), (j) and (w), (k) and (x), (l) and (y), (m) and (z) are the results based on IMP-ResNet-50, CLIP-RSCLNet-ResNet50, TOV, SeCo, CaCo, GeRSP, GeRSP 200, Geo-Aware, MoCo-IN, and MoCov2 BIT, respectively.
[0050] Example 2 This embodiment provides a CLIP-guided remote sensing image recognition model training system, employing a CLIP-guided remote sensing image recognition model training method as provided in Embodiment 1, including: The image dataset building module is configured to: acquire image datasets and preprocess the image datasets; The base feature vector acquisition module is configured to acquire image feature vectors and text feature vectors based on the dataset. The similarity calculation module is configured to: calculate the first similarity between the image feature vector and the difficult text feature vector, and the second similarity between the image feature vector and the simple text feature vector, respectively; The difficulty score calculation module is configured to calculate the difference between the first similarity and the second similarity. The difference is used as a difficulty score; based on the difficulty score, the image feature vector can be divided into samples of different difficulty levels. The model training module is configured to build a remote sensing image recognition model, and train the remote sensing image recognition model in three stages by using different difficulty samples, specifically including: the first stage uses low difficulty samples to perform first stage training on the model; the second stage mixes low difficulty samples and high difficulty samples to perform second stage training on the model; and the third stage uses high difficulty samples to perform third stage training on the model. The model output module: obtains the trained remote sensing image recognition model.
[0051] Embodiment three The embodiment provides a computer readable storage medium, which has a computer program stored thereon, and the program is executed by a processor to implement the steps of the remote sensing image recognition model training method guided by CLIP.
[0052] Embodiment four A computer program product includes computer programs / instructions, which are executed by a processor to implement the steps of the remote sensing image recognition model training method guided by CLIP.
[0053] Embodiment five The embodiment provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the program to implement the steps of the remote sensing image recognition model training method guided by CLIP.
[0054] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems) and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The device for implementing the functions specified in one flow or multiple flows and / or blocks Figure 1 The device for implementing the functions specified in one flow or multiple flows and / or blocks
[0055] The above only describes the preferred embodiments of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. A CLIP-guided remote sensing image recognition model training method, characterized in that, include: An image dataset is constructed and preprocessed. Image feature vectors and text feature vectors are obtained based on the dataset. The text feature vectors include difficult text feature vectors and simple text feature vectors. A first similarity between the image feature vectors and the difficult text feature vectors, and a second similarity between the image feature vectors and the simple text feature vectors are calculated. The difference between the first similarity and the second similarity is calculated and used as a difficulty score. Based on the difficulty score, the image feature vectors can be divided into low-difficulty samples and high-difficulty samples. A remote sensing image recognition model is constructed and trained in three stages using samples of varying difficulty. Specifically, the training stages are as follows: the first stage uses low-difficulty samples for the first stage training; the second stage mixes low-difficulty and high-difficulty samples for the second stage training; and the third stage uses high-difficulty samples for the third stage training. Finally, a well-trained remote sensing image recognition model is obtained.
2. The CLIP-guided remote sensing image recognition model training method of claim 1, wherein, The formula for calculating the first similarity [dif_cls] between the image feature vector and the simple text feature vector is as follows: The formula for calculating the second similarity [ez_cls] between image feature vectors and complex text feature vectors is as follows: wherein a feature vector of an image, is a simple text feature vector, is a simple text feature vector, is a dot product of the vectors; the difficulty score The formula for calculating the difficulty score is: 。 3. The CLIP-guided remote sensing image recognition model training method of claim 1, wherein, Based on the difficulty score, image feature vectors with difficulty scores ≥ 0.5 are classified as high difficulty samples, and image feature vectors with difficulty scores < 0.5 are classified as low difficulty samples. 4.The CLIP-guided remote sensing image recognition model training method of claim 1, wherein, The first stage uses low-difficulty samples to train the model; specifically, it includes: The first stage focuses on basic training with low-difficulty samples. Low-difficulty samples are selected based on difficulty scores, and the cross-entropy loss function is used to help the model establish basic remote sensing feature representations.
5. The CLIP-guided remote sensing image recognition model training method of claim 1, wherein, The second stage involves training the model by mixing low-difficulty and high-difficulty samples; specifically, it includes: The second stage involves a balanced learning process, in which mixed difficulty samples are gradually introduced with each training round. The sample ratio changes from 1:9 for low-difficulty samples to 9:1 for high-difficulty samples. Dynamic weights are calculated based on the sample difficulty scores, and a weighted cross-entropy loss is used to balance the learning process. This allows the model to consolidate its basic features while gradually adapting to complex scenarios, learning to distinguish similar categories, and initially dealing with mild interference.
6. The CLIP-guided remote sensing image recognition model training method of claim 1, wherein, The third stage involves training the model using high-difficulty samples; specifically, it includes: In the third stage, high-difficulty samples are selected, and the Focal Loss function is switched to reduce the loss weight of easily classified samples, so that the model focuses on learning high-difficulty samples.
7. A CLIP-guided remote sensing image recognition model training system based on any one of the CLIP-guided remote sensing image recognition model training methods of claims 1-6. include: The image dataset building module is configured to: acquire image datasets and preprocess the image datasets; The base feature vector acquisition module is configured to acquire image feature vectors and text feature vectors based on the dataset. The similarity calculation module is configured to: calculate the first similarity between the image feature vector and the difficult text feature vector, and the second similarity between the image feature vector and the simple text feature vector, respectively; The difficulty score calculation module is configured to calculate the difference between the first similarity and the second similarity. The difference is used as a difficulty score; based on the difficulty score, the image feature vector can be divided into samples of different difficulty levels. The model training module is configured to build a remote sensing image recognition model and train the remote sensing image recognition model in three stages by using different difficulty samples, specifically including: using low difficulty samples to perform first stage training on the model in the first stage; using mixed low difficulty samples and high difficulty samples to perform second stage training on the model in the second stage; and using high difficulty samples to perform third stage training on the model in the third stage. The model output module obtains the trained remote sensing image recognition model.
8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps in the remote sensing image recognition model training method based on CLIP guidance according to any one of claims 1-6.
9. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the steps in the remote sensing image recognition model training method based on CLIP guidance according to any one of claims 1-6.
10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps in the remote sensing image recognition model training method based on CLIP guidance according to any one of claims 1-6.