SAR (Synthetic Aperture Radar) image description method, device and equipment based on multi-modal large model driving

Abstract

The invention relates to an SAR image description method, device and equipment based on multi-modal large model driving, and the method comprises the steps: constructing a target-relation-scene three-layer collaborative layered labeling strategy according to a multi-layer relation of a target in a scene; constructing a structural cue word comprising a description cue instruction, a category cue instruction and a restriction word cue instruction, inputting the SAR image and the structural cue word into a multi-modal large model, and generating hierarchical semantic description corresponding to each hierarchy in a hierarchical labeling strategy by the model according to the structural cue word, and reconstructing and fusing the semantic description of each level to obtain a preliminary semantic description, performing quality analysis on the preliminary semantic description, optimizing the structural cue word according to a quality analysis result, and correcting the preliminary semantic description according to a semantic description standard to obtain the semantic description. By adopting the method, the semantic description annotation of the SAR image can be realized aiming at the problems that the image structure is complicated, the annotation mode is limited and the data scale is insufficient.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of SAR image processing, in particular to a SAR image description method, device and equipment based on a multi-modal large model driving. BACKGROUND

[0002] Synthetic Aperture Radar (SAR) has become an indispensable key image type in the field of remote sensing due to its all-weather and all-day imaging capability, and is widely used in pollution monitoring, disaster assessment, terrain exploration and national security scenes. Compared with visible light images, SAR images are based on microwave reflection imaging principles and can provide stable and reliable image information in complex environmental conditions. However, current research on SAR images is mostly focused on scene classification, target detection and segmentation tasks, which are limited to "what" and "where" in terms of object understanding, but are difficult to capture semantic relationships between objects and overall scene structure. The SAR image description task aims to automatically generate semantic text descriptions, which can express objects, attributes and their relationships, thereby providing key support for high-level semantic understanding of remote sensing images.

[0003] However, remote sensing images, especially SAR images, have complex structures, chaotic textures, and abstract semantics, making it difficult to directly transfer description models on natural images to this field. Existing remote sensing image description datasets such as UCM-Captions, RSICD, etc. are mainly based on visible light images, and only SSICD and SARLANG-1M-Cap involve SAR image description. SSICD uses artificial annotation, has limited sample size, and is time-consuming and labor-intensive; SARLANG-1M-Cap relies on visible light and SAR image pairs for transfer annotation, and the annotation process is highly dependent on visible light images, limiting its independent applicability. In addition, the sample size of these two datasets is insufficient, making it difficult to support the construction of high-level understanding models for SAR images. SUMMARY

[0004] Therefore, it is necessary to provide a SAR image description method based on a multi-modal large model driving for semantic description annotation of SAR images.

[0005] A SAR image description method based on a multi-modal large model driving, the method comprising: constructing a hierarchical annotation strategy for target-relation-scene three-layer cooperation according to the multi-layer relationship of the target in the scene; construct a structural prompt word, the structural prompt word comprising a description prompt instruction, a category prompt instruction, and a restriction word prompt instruction, wherein the description prompt instruction is constructed based on the hierarchical annotation strategy; obtaining a SAR image, inputting the SAR image and the structural prompt word into a multi-modal large model, and generating, by the multi-modal large model, a hierarchical semantic description corresponding to each layer in the hierarchical annotation strategy through an API calling mode according to the structural prompt word, reconstructing and fusing the hierarchical semantic descriptions to obtain a preliminary semantic description of the SAR image; performing quality analysis on the preliminary semantic description, optimizing the structural prompt word according to a quality analysis result, and correcting the preliminary semantic description according to a semantic description standard to obtain a semantic description of the SAR image.

[0006] In an embodiment, the hierarchical annotation strategy comprises: target layer annotation for describing basic attributes of a core target; relationship layer annotation for encoding spatial and structural relationships between targets; and scene layer annotation for understanding overall semantic and functional information of an image.

[0007] In an embodiment, the target layer annotation comprises target category information, target quantity and distribution information, and estimated size and related size information of the target; the relationship layer annotation comprises position and arrangement relationships between the target and other targets, interaction relationships between the target and the scene, and relative distance and direction attributes of the target and other targets; the scene layer annotation comprises scene type, scene functionality judgment, and scene environment information.

[0008] In an embodiment, when constructing the structural prompt word: the description prompt instruction sets three types of structured prompts, namely, a target layer, a relationship layer, and a scene layer, according to the hierarchical annotation strategy; the category prompt instruction sets a plurality of target category prompts related to a scene in which a target is located according to prior knowledge; the restriction word prompt instruction sets a constraint sentence format of the preliminary semantic description according to description requirements.

[0009] In an embodiment, after obtaining the semantic description of the SAR image: the semantic description is taken as a semantic description label of a corresponding SAR image, a plurality of SAR images are used to generate corresponding semantic description labels, and a training data set and a test data set are constructed; The target model is trained by using the training data set, so that the trained target model has the ability to semantically describe the SAR image.

[0010] In one embodiment, the target model is constructed by an encoder and a decoder, and when the target model is trained: An SAR image in the training data set is input into the encoder for feature extraction to obtain image features; The semantic description label corresponding to the SAR image is processed by using an embedding layer to obtain language embedding; The image features and the language embedding are input into the decoder to obtain a semantic description pseudo label; The adjustable parameters in the encoder and the decoder are adjusted by using a loss function calculated by the semantic description pseudo label and the semantic description label until convergence, to obtain the trained target model.

[0011] In one embodiment, the encoder uses a ResNet network and the decoder uses an LSTM network.

[0012] The application also provides a SAR image description device based on a multi-modal large model driving, which comprises: A hierarchical annotation strategy construction module is configured to construct a hierarchical annotation strategy on target-relation-scene three-layer cooperation according to the multi-layer relationship of the target in the scene; A structural prompt word construction module is configured to construct a structural prompt word, which comprises a description prompt instruction, a category prompt instruction, and a restriction word prompt instruction, wherein the description prompt instruction is constructed based on the hierarchical annotation strategy; A preliminary semantic description acquisition module is configured to acquire an SAR image, input the SAR image and the structural prompt word into a multi-modal large model, and generate a hierarchical semantic description corresponding to each level in the hierarchical annotation strategy by the multi-modal large model through an API calling mode according to the structural prompt word, reconstruct and fuse the hierarchical semantic descriptions to obtain a preliminary semantic description of the SAR image; An SAR image semantic description fine-tuning module is configured to analyze the quality of the preliminary semantic description, optimize the structural prompt word according to the quality analysis result, and correct the preliminary semantic description according to a semantic description standard to obtain the semantic description of the SAR image.

[0013] A computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the SAR image description method based on a multi-modal large model driving when executing the computer program.

[0014] A computer readable storage medium, having stored thereon a computer program, the computer program being executed by a processor to implement the steps of the SAR image description method driven by the multi-modal large model described above.

[0015] The SAR image description method, device and equipment based on the multi-modal large model described above, by constructing a hierarchical labeling strategy about the target-relation-scene three-layer cooperation according to the multi-layer relationship of the target in the scene, constructing a structural prompt word including description prompt instructions, category prompt instructions and restriction word prompt instructions, inputting the SAR image and the structural prompt word into the multi-modal large model, the multi-modal large model generating the hierarchical semantic description corresponding to each level in the hierarchical labeling strategy through the API calling mode according to the structural prompt word, reconstructing and fusing each level semantic description to obtain the preliminary semantic description of the SAR image, performing quality analysis on the preliminary semantic description, optimizing the structural prompt word according to the quality analysis result, and correcting the preliminary semantic description according to the semantic description standard to obtain the semantic description of the SAR image. The method can solve the problems of complex image structure, limited labeling mode and insufficient data scale, and realize the semantic description labeling of the SAR image. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A flowchart of the SAR image description method driven by the multi-modal large model in one embodiment; Figure 2 A flowchart of the hierarchical labeling rule design and the semi-automatic annotation framework based on the multi-modal large model in one embodiment; Figure 3 A related situation diagram of the image dataset constructed by the SAR image description method driven by the multi-modal large model in one embodiment, wherein, Figure 3 (a) represents a probability density distribution diagram for sentence length, Figure 3 (b) represents a word cloud distribution of main high-frequency words, Figure 3 (c) represents a statistical overview diagram of the dataset in multiple dimensions such as image quantity, labeling density and vocabulary coverage; Figure 4 The overall framework of the encoder-decoder model and the related network diagram in one embodiment; Figure 5 A sentence diagram generated by the SAR image description in one embodiment; Figure 6 A structure block diagram of the SAR image description device driven by the multi-modal large model in one embodiment; Figure 7 An internal structure diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0018] Due to the fuzzy target contour, incomplete structure, and not obvious difference between target and background in the SAR image, the great difference inside the scene greatly increases the difficulty of semantic extraction and expression. At the same time, the existing labeling method relies on manual labeling or optical-SAR image pairing, the former is low in efficiency and high in cost, and the latter is seriously dependent on the visible light image, and cannot realize semantic generation independently of the SAR image. These problems all lead to the fact that the scale of high-quality data samples is generally small, which is difficult to meet the training needs of deep learning models in semantic modeling and sentence generation, and restricts the improvement of model performance.

[0019] In the present application, in view of the above problems, as shown in Figure 1 A SAR image description method based on a multi-modal large model is proposed, which specifically includes the following steps: Step S100, according to the multi-layer relationship of the target in the scene, a hierarchical labeling strategy about target-relation-scene three-layer cooperation is constructed.

[0020] Step S110, a structural prompt word is constructed, the structural prompt word includes a description prompt instruction, a category prompt instruction and a restriction word prompt instruction, wherein the description prompt instruction is constructed based on the hierarchical labeling strategy.

[0021] Step S120, the SAR image is acquired, and the SAR image and the structural prompt word are input into the multi-modal large model, the multi-modal large model generates the hierarchical semantic description corresponding to each level in the hierarchical labeling strategy according to the structural prompt word through the API calling mode, and reconstructs and fuses each hierarchical semantic description to obtain a preliminary semantic description of the SAR image.

[0022] Step S130, the quality of the preliminary semantic description is analyzed, the structural prompt word is optimized according to the quality analysis result, and the preliminary semantic description is corrected according to the semantic description standard to obtain the semantic description of the SAR image.

[0023] In this embodiment, a novel hierarchical annotation rule is designed, starting from three semantic dimensions of target, relationship and scene, to generate three types of representative and complementary description sentences, covering key objects, interactive relationships between objects and overall scene structure in the image, effectively improving the expression completeness and information structure of the description sentences. On this basis, a semi-automatic annotation framework is constructed by introducing a multi-modal large model, which directly generates a description draft based on the SAR image through multi-modal reasoning ability, and is reviewed by human beings, significantly reducing the annotation cost while maintaining high semantic quality.

[0024] In the field of natural images, image description often relies on clear edge structure and prominent semantic features. However, due to its unique imaging mechanism, SAR images exhibit high uncertainty in terms of structural completeness, target recognizability and contextual consistency. These differences result in a significant decline in the effectiveness of traditional end-to-end image description methods when transferred to the SAR field, often resulting in semantic loss, misjudgment or empty description. Therefore, how to systematically extract and organize semantic information from SAR images is a key challenge for high-quality description generation.

[0025] To address the above challenges, in step S100, a "target-relationship-scene" hierarchical annotation strategy is proposed for the characteristics of SAR images, to simulate the human visual process from local perception to overall understanding, and to model the semantics of SAR images from bottom to top, to alleviate problems such as image semantic ambiguity, structural loss and scene ambiguity. This strategy breaks down the overall description task into three interrelated levels, each focusing on information capture and organization at a specific granularity, such as Figure 2 as shown.

[0026] In this embodiment, the hierarchical annotation strategy includes target-level annotation for describing the basic attributes of core targets, relationship-level annotation for encoding the spatial and structural relationships between targets, and scene-level annotation for understanding the overall semantic and functional information of the image, to construct a three-level annotation strategy.

[0027] Specifically, the target level (Object-level) focuses on key target elements in the image. Due to the noise interference or structural occlusion of targets in SAR images, only the illuminated surface is retained while the outline is missing, making it difficult for traditional target detection to restore their complete form. To alleviate this problem, the basic attributes of the target are focused on in the target layer annotation.

[0028] In one embodiment, the target level annotation includes target category information, target quantity and distribution information, and target estimated size and related size information. The target category information can be "passenger plane", "transport plane", "helicopter", etc., that is, various categories related to a target theme. The target quantity and distribution information is, for example, "two aircrafts in parallel distribution", "one isolated aircraft", etc. The target estimated size and related size information is, for example, "small passenger plane", "large transport plane", etc. These basic elements constitute the core atoms of semantic construction, and subsequent relationship and scene modeling are expanded based on these anchors.

[0029] Specifically, the relationship level models the structural association between targets and between targets and the environment. The target edges in the SAR image are blurred, and the background echo intensity is unstable, making it difficult to directly model semantic relationships. To improve interpretability, spatial structural relationships are explicitly labeled.

[0030] In one embodiment, the relationship level annotation includes the position and arrangement relationship between targets and other targets, the interaction relationship between targets and scenes, and the relative distance and direction attributes of targets and other targets. The position and arrangement relationship between targets and other targets is, for example, "pair distribution", "in-line arrangement", etc. The interaction relationship between targets and scenes is, for example, "parked on the runway", "close to the tower area", etc. The relative distance and direction attributes of targets and other targets are, for example, "target A is located in the left front of target B", etc.

[0031] Specifically, the scene level is responsible for encoding the overall semantic environment of the image. In the SAR image, the echo characteristics of the same target in different scenes differ significantly, and the background texture and feature distribution are also extremely inconsistent, making it difficult to fully understand the image by relying only on the target and relationship level.

[0032] In one embodiment, the scene level annotation includes scene type, scene functionality judgment, and scene environment information. The scene type is, for example, "city airport", "sea port", "plateau mountain", etc. The scene functionality judgment is, for example, "pre-flight task", "parking state", etc. The scene environment information is, for example, "adjacent to the coast", "background is mountain terrain".

[0033] In this embodiment, the three-level semantic annotation constitutes a complete description framework from bottom to top. The process is formalized as: Caption=F(O(x),R(x),S(x)) In the above formula, x represents the input SAR image, O(x), R(x), and S(x) correspond to the target, relationship, and scene three-level semantic information, respectively, and F is the semantic fusion and natural language generation module.Figure 2 The core role of this hierarchical strategy in the overall annotation process is demonstrated: by extracting semantic units layer by layer, a structured sketch is formed, and then a multi-modal large model is used to complete language generation, ensuring that the output sentence is semantically complete and structurally clear.

[0034] In this embodiment, according to the reference scene of the subsequent downstream task, a corresponding hierarchical annotation strategy is constructed.

[0035] Compared with the simple and repetitive annotation method in traditional image description, the hierarchical annotation strategy proposed in this method ensures the explainability and controllability of the generated sentence through structured semantic guidance, to enhance the semantic controllability. Through the diversification of the sentence template composed of hierarchical semantics, different scenes are adapted to improve the annotation diversity. This hierarchical annotation strategy is specially designed for SAR images, taking into account signal distortion, background interference, and semantic coefficient problems to better adapt to the characteristics of SAR data. At the same time, in the model training and evaluation of subsequent tasks, the annotation division of different levels can support phased training and fine-grained evaluation to improve the model performance and generalization ability.

[0036] Then, after clarifying the annotation strategy of the "target-relation-scene" three-layer cooperation, in this method, a semi-automatic annotation process framework is further constructed to generate structured semantics for SAR images with the help of multi-modal large models (MMLMs). This process aims to maximize the combination of structured knowledge of human annotation and the generation ability of large models, while improving annotation efficiency, ensuring sentence quality, structural consistency, and SAR context adaptability.

[0037] In this embodiment, the semi-automatic annotation process framework consists of five stages: the construction of structural prompts in step S110, the injection of multi-modal input and language generation output in step S120, the interactive feedback optimization and manual semantic verification in step S130. Among them, the design of structural prompts is the key to generation quality, which directly affects the ability of MMLM to capture the semantic structure of SAR.

[0038] In this embodiment, the structural prompts are divided into three categories to support the generation and control of three-layer semantics. When constructing the structural prompts: Descriptive Prompts set three types of structured prompts according to the hierarchical annotation strategy: target layer, relationship layer, and scene layer. Category Prompts set a variety of target category prompts related to the scene according to prior knowledge. Constraint Prompts set the constraint sentence format of the preliminary semantic description according to the description requirements.

[0039] Specifically, when constructing description prompts, corresponding to the generation requirements of layered semantics, three types of structured prompts are designed: "target layer," "relationship layer," and "scene layer." For example, for the target layer, prompts might include: "Please describe the type, number, and size of the objects in the image"; for the relationship layer, prompts might include: "Please describe the spatial relationships between multiple objects and their interactions with the scene"; and for the scene layer, prompts might include: "Please describe the environmental context of the image and its functional purpose."

[0040] Specifically, considering that the large multimodal model has not been trained on SAR images, category prompts are used to provide it with relevant prior knowledge in specific application scenarios. For example, "The image may contain a fighter jet, transport aircraft, or drone. Please determine the target category based on image features."

[0041] Specifically, to prevent the model-generated content from deviating from the semantic structure or introducing hallucinations, constrained instructions are designed to limit the sentence style (such as using "shared...frame...distributed in..."), vocabulary boundaries (avoiding vague expressions such as "looks like" and "suspected") and structural patterns ("goal first - then relationship - then scene").

[0042] After the construction of the structural prompt words is completed, the SAR image with the target detection box is used as input, so that the MMLM can focus on the core target area during the generation process and suppress the misleading background interference on the semantic generation. The input image, i.e., the SAR image, is fed into the multimodal large model together with the structural prompt words, and the preliminary annotation results are generated through API calls. In order to fully utilize the model generation capabilities and ensure the correspondence of the description levels, in this embodiment, the strategy of "layered independent generation-structured fusion output" is adopted, that is, sentence fragments are first generated for each semantic layer, and then uniformly reconstructed into a complete natural language sentence. For a given SAR image x, the overall generation process can be formalized as:

[0043] In the above formula They represent the generative semantics of the target, relationship, and scene levels respectively, and F is the fusion function used to construct the final natural language description.

[0044] Furthermore, considering that although the above design significantly improves the structural consistency of the generation, there are still problems with hallucination content, structural confusion or linguistic ambiguity in the initial output results. For this reason, an interactive feedback optimization mechanism is introduced into the process. After each round of generation, the quality of the sentences output by the multimodal large model is analyzed, and common errors are fed back to the prompt template layer. For example, when it is found that the model frequently misjudges "tarmac" as "runway", definition constraints and semantic comparison instructions are added to the next round of prompts. After multiple rounds of iterations, the prompt words gradually converge to form a stable template, which greatly improves the description quality and structural consistency.

[0045] In this embodiment, an artificial review mechanism is also introduced to refine the semantic of the automatically generated annotation results. The review process follows the standards including semantic structure alignment, semantic validity verification, and language cleaning and polishing. Among them, the semantic structure alignment standard includes whether the three-layer structure is complete and whether it strictly follows the order of target-relation-scene. Semantic validity verification includes whether the sentence has unfounded guesses or image content unrelated information. Language cleaning and polishing includes removing redundant language, eliminating grammatical errors, and improving language fluency and clarity.

[0046] The annotation that passes the review will be used as the final annotation corpus for dataset construction and model training. This annotation process combines human-machine collaborative generation and screening mechanisms while maintaining structural control, ensuring the professionalism, accuracy, and usability of the SAR image description corpus.

[0047] In this embodiment, the multi-modal large model can use GPT-4V, DeepSeek, etc.

[0048] Compared with traditional manual annotation methods, the semi-automatic annotation process framework proposed in this method significantly reduces labor costs, improves annotation efficiency, generates content with clear structure and unified style, and is highly scalable and adaptable to other SAR categories. It provides high-quality semantic supervision for subsequent model training.

[0049] Further, based on the above SAR image description method, after obtaining the semantic description of the SAR image, the semantic description is used as the semantic description label of the corresponding SAR image, and the corresponding semantic description label is generated according to multiple SAR images to construct a training data set. The target model is trained using the training data set, so that the trained target model has the ability to semantically describe the SAR image.

[0050] In this embodiment, based on the above image description method, a structured image-text dataset focusing on SAR aircraft targets is constructed for aircraft target-related scenes. This dataset aims to provide high-quality semantic support for language modeling of complex SAR images, addressing the long-standing lack of image-text alignment resources in the field of remote sensing images. It provides a standardized data foundation for image description, semantic understanding, and cross-modal retrieval tasks. At the same time, this dataset introduces a multi-level modeling paradigm of "target-relation-scene" for the first time in its structured organization, providing system support for the subsequent improvement of model semantic structure modeling capabilities.

[0051] Specifically, more than 5000 images were selected from two published high-resolution SAR data sources, SADD and SAR-Aircraft-1.0, combined with task requirements, image quality, and target diversity, to form a selected set of 5251 aircraft SAR images. To ensure consistency in subsequent model input, all images were standardized to 224x224 pixels, maintaining the integrity of key information while improving compatibility with mainstream visual models and unifying input formats across models and tasks.

[0052] In terms of description annotation, the hierarchical semantic modeling and semi-automatic annotation process proposed by the present method generate 9 structured description sentences for each image, totaling 47259 natural language annotations. These sentences correspond to the "target, relationship, scene" three semantic levels, ensuring that each image has both details and overall content, as well as target positioning and scene induction, providing training corpus support for hierarchical learning and multi-granularity expression of downstream models.

[0053] To characterize the language characteristics of the dataset, the vocabulary coverage and semantic distribution of the corpus were further analyzed to reflect the richness of language expression and task alignment: total number of words: 548,795, vocabulary size (without repetition): 4180, average length of each sentence: 11.6 words.

[0054] As shown in Figure 3 (a), the probability density distribution of sentence length indicates that the generated sentences are concentrated between 10 and 13 words, with refined structure and clear expression, conforming to standard sentence type specifications. As shown in Figure 3 (b), the word cloud distribution of main high-frequency words shows that key words such as "aircraft", "target", "runway", and "parking" frequently appear, reflecting the close relationship between the annotation language and the SAR semantic task. As shown in Figure 3 (c), the system presents a statistical overview of the dataset in terms of image quantity, annotation density, and vocabulary coverage, verifying the scale, balance, and systematicity of semantic coverage of the data.

[0055] The dataset is designed to fully combine the imaging characteristics and cognitive structure of SAR images, demonstrating the following three advantages: diversity and complexity coexist: samples cover multiple imaging perspectives and target forms, with significant differences in intra-class structure, target posture, and background environment, providing good support for model generalization. Semantic coverage is comprehensive: each image has multi-level, multi-angle, and multi-style language expression, supporting more refined image description and reasoning tasks. Strong reusability: based on unified templates and structural rules, it can be extended to other SAR target types such as ships and vehicles.

[0056] It is worth emphasizing that ATRNet-SARCap 1.0 is the first large-scale SAR aircraft image-text alignment dataset currently, and its construction not only realizes the system closed loop from SAR semantic perception to natural language generation, but also realizes the targeted response to the core difficulties of remote sensing language tasks in data structure design. Its structured semantic organization method and high-quality annotation results provide a clear semantic and clear structural supervision signal for subsequent image-language modeling tasks, emphasizing the progress of the SAR image language understanding task from sparse sample construction to systematic corpus construction.

[0057] By constructing the training dataset according to the method, on the one hand, the effectiveness of the method can be proved, and on the other hand, training data for subsequent downstream SAR image description tasks is provided.

[0058] In this paper, a specific implementation method of the SAR image description task is also provided, which trains a target model to have SAR image description capability by using the training dataset constructed according to the SAR image description method proposed above, and further proves the effectiveness of the SAR image description method in this paper.

[0059] In this embodiment, the target model is constructed by an encoder and a decoder (Encoder-Decoder). The target model uses the mainstream encoder-decoder (Encoder-Decoder) framework to model and train the SAR image description task. This framework has achieved great success in natural image description tasks, and its modular structure also facilitates the combination of different types of visual and language modelers, and adapts to the structured corpus characteristics constructed in this paper.

[0060] In this embodiment, when training the target model: inputting one SAR image in the training dataset into the encoder for feature extraction to obtain image features, at the same time, processing the semantic description label corresponding to the SAR image by using the embedding layer to obtain the language embedding, inputting the image features and the language embedding into the decoder to obtain the semantic description pseudo label, adjusting the adjustable parameters in the encoder and the decoder by using the loss function calculated by the semantic description pseudo label and the semantic description label until convergence, and obtaining the trained target model.

[0061] Specifically, the encoder is responsible for extracting high-level visual features from the input SAR image, generating an intermediate representation of the image semantics. Considering the characteristics of SAR images in the imaging process, such as target edge blur, incomplete structure, and strong background clutter, the direct migration of traditional natural image feature extractors has significant performance differences. Therefore, multiple visual models are introduced for systematic comparison, covering traditional convolutional neural networks, Transformer-based structures, and multi-modal pre-training models to comprehensively evaluate the ability of different encoders in SAR image semantic modeling.

[0062] In this embodiment, CNN network, VIT network, and CLIP network are selected as the encoder architecture. VGGNet: as a clear structure deep CNN, it has stable performance in images with strong target visibility, but its edge detail modeling capability in SAR images is limited. ResNet: with the help of residual connection to enhance gradient flow ability, it is suitable for training deeper networks and helps to extract more discriminative global and local features. ViT: divides the image into fixed-size patches and constructs a global self-attention mechanism, which has stronger robustness to unstructured noise in SAR images. CLIP-Image Encoder: as a multi-modal alignment model, it is trained in the image-text alignment feature space and has strong cross-modal semantic expression ability, which is expected to capture more semantically oriented image features.

[0063] Specifically, for each input image , the encoder extracts its corresponding feature vector , and the calculation formula is:

[0064] In the above formula, represents the high-level semantic representation vector of the image. In the experiment, the influence of different feature spaces on the subsequent decoding effect is compared, and it is found that the medium dimension (such as 512 or 1024) achieves a good balance between accuracy and training stability.

[0065] The motivation for choosing these models as encoders is to consider the characteristics of SAR images, such as weak edge information and stable target region structure. The self-attention mechanism-based encoders, including ViT and CLIP, can improve the model's feature modeling capability for key regions such as aircraft bodies, wings, and tail structures. These models naturally have regional perception advantages, which can help alleviate the problems of incomplete structure and noise interference in SAR images.

[0066] Further, the decoder is used to jointly model the image features and language embeddings to generate natural language descriptions that are semantically consistent and linguistically coherent. To model the "target-relation-scene" three-layer structured semantics in the training data set, two types of mainstream language models based on LSTM and Transformer are used to compare their abilities in sentence structure, semantic consistency, and long-distance dependency modeling. Among them, LSTM decode can gradually generate semantic fragments through time series structure, and performs well in controlling syntactic logic and short-term dependency, which is suitable for modeling local descriptions such as target class and relation class. Transformer decode is based on the self-attention mechanism, which allows the model to reference the entire history sequence and image features at each time step, improving the modeling ability of global structure and multi-semantic interaction of the scene layer.

[0067] Specifically, the decoder at each time step receives image features and the hidden state of the previous time step , calculates the output probability distribution of the current time step , and the core recursive form is:

[0068] wherein , , are the weight matrices of different network layers, and the Softmax mapping function of the output layer is represented as the output probability distribution of all words on the word table.

[0069] In addition, different modeling strategies are used for the generated sentences according to the different semantic characteristics of the target layer, the relation layer and the scene layer to improve the consistency and diversity of the description. Among them, the hierarchical language description mode with clear structure and independent semantics is crucial for structured description of SAR images.

[0070] In this embodiment, the core goal of the target model training is to minimize the difference between the model-generated sentences and the reference annotations to achieve accurate, fluent and structurally consistent natural language generation. To this end, a cross-entropy loss function based on word-level supervision is used as the optimization objective to perform maximum likelihood learning on the output word probability distribution and the reference word at each time step.

[0071] Let the i-th image in the training set be , the corresponding reference description be , and the model-generated sentence be The training goal is to maximize the conditional probability:

[0072] The transformation into the form of a minimization loss function is:

[0073] wherein, denotes the total number of training images, is the maximum length of a single sentence, is the probability distribution predicted by the model.

[0074] Further, in terms of training details, in order to further adapt to the structural complexity of SAR data, two strategies of hierarchical supervision training and sentence-level robustness enhancement are adopted to improve the convergence speed and model stability. Among them, hierarchical supervision training: in the training process, the loss of the generated sentence at the target, relationship and scene levels is calculated respectively, and weighted joint optimization is performed to ensure sufficient coverage of semantic learning signals at each level. Sentence-level robustness enhancement: introduce Scheduled Sampling strategy, gradually guide the model to use its own historical generated words as input in the later training stage, and alleviate the exposure bias problem brought by Teacher Forcing.

[0075] The Adam optimizer is used in the training stage, and the learning rate is , and the training epoch number is 50. In the evaluation stage, the standard indicators widely used in image description tasks such as BLEU, CIDEr, ROUGE-L and METEOR are used to comprehensively measure the comprehensive performance of the model in terms of syntactic quality, semantic relevance and structural accuracy.

[0076] This training scheme fully utilizes the hierarchical language supervision information provided by the dataset under the guidance of structured semantics, and builds a solid modeling foundation for the structural understanding and natural language expression of SAR images.

[0077] In the foregoing, not only to verify the effectiveness of the SAR image description method based on multi-modal large model driving proposed in this paper, but also to propose an implementable SAR image description model training method for the downstream task of the method, and a target model with SAR image description capability. As Figure 4 shown.

[0078] In this embodiment, the target model includes an encoder and a decoder, wherein the encoder adopts one of a CNN network, or a ResNet network, or a VIT network, or a CLIP network, and the decoder adopts one of an LSTM network or a Transform network.

[0079] Further, to comprehensively evaluate the effectiveness of the proposed hierarchical annotation strategy and encoder-decoder framework in the SAR image description task, a series of comparative experiments were designed on the SARAircraft-Captions dataset. Specifically, two annotation strategies were set: 1) based on the "target-relation-scene" hierarchical annotation strategy; 2) the traditional simple annotation strategy. To ensure fairness in comparison, we carefully selected 5 descriptions from the 9 generated by the hierarchical annotation as traditional annotations, ensuring that both were comparable in quantity and coverage.

[0080] In exploring the impact of different encoder and decoder combinations on SAR image description performance, a variety of network architectures widely used in image feature extraction were selected as image encoders, including CNN, ViT, and CLIP. These encoders are widely used in various image processing tasks due to their excellent performance in handling complex image features. At the same time, considering the special requirements of the image description task, we selected LSTM and Transformer, two decoders that perform well in the image description field. Under two different data annotation strategies, these encoder-decoder combinations were systematically experimented to comprehensively evaluate their performance in the SAR image description task.

[0081] To further improve model performance, all encoders have been pre-trained on the ImageNet dataset, which enables the model to have good perception of image features from the initial stage. In the decoder module, the embedding dimension of LSTM and Transformer is set to 512, and the dimension of the attention mechanism is also set to 512. This high-dimensional setting helps the model capture more rich semantic information and contextual relationships. A batch size of 32 is used, and the maximum training period is set to 50 epochs. In addition, to prevent model overfitting and save computing resources, an early stopping mechanism is introduced, with a patience value of 6, i.e., if the validation accuracy does not improve for six consecutive training periods, training is stopped. In the beam search phase, a beam width of 5 is used to balance computational efficiency and the quality of generated results during decoding. The experimental dataset is divided as follows: the training set contains 4200 images, the test set contains 525 images, and the validation set contains 526 images.

[0082] Table 1. Performance of different encoder networks on the ATRNet-SARCap 1.0 dataset when LSTM is used as the decoder

[0083]

[0084] To comprehensively evaluate the quality of generated sentences, this study employs multiple mainstream evaluation metrics, including BLEU-1 to BLEU-4, METEOR, ROUGE-L, and CIDEr. These metrics measure the similarity and consistency between generated and reference texts from different perspectives.

[0085] Specifically, the BLEU metric assesses lexical similarity by counting the n-gram overlaps between reference and generated sentences, with BLEU-1 to BLEU-4 measuring the similarity from 1-gram to 4-gram. The METEOR metric considers inflection, synonymy, and word order, measuring the matching degree between sentences, with higher scores indicating better semantic consistency and fluency. The ROUGE-L metric focuses on the longest common subsequence (LCS) between two sentences, evaluating structural similarity, with higher scores implying better preservation of the reference sentence's structural information. The CIDEr metric uses TF-IDF weights to calculate lexical similarity, reducing the influence of common words, with higher scores indicating stronger semantic and structural consistency between generated and reference sentences.

[0086] BLEU is a modified accuracy metric, while the other three metrics are calculated by combining accuracy and recall. The values of BLEU, METEOR, and ROUGE-L range from 0 to 1, while the value of CIDEr ranges from 0 to 5. Higher values indicate higher similarity between the predicted description and the human-labeled reference description.

[0087] Table 3. Performance of different encoder networks on the ATRNet-SARCap 1.0 dataset when using the Transformer as the decoder

[0088]

[0089] To evaluate the effectiveness of the hierarchical annotation strategy in the SAR image description task, a comparative experiment was designed, selecting multiple encoders and decoders for combination and conducting experiments on datasets generated by hierarchical and traditional annotation strategies. The experimental results are recorded in Tables 1 to 4, where Tables 1 and 3 are based on the hierarchical annotation strategy, and Tables 2 and 4 are based on the traditional annotation strategy.

[0090] By comparing the experimental data in Table 1 and Table 2, it can be found that the experimental results obtained by the annotation data based on the hierarchical annotation strategy are superior to those based on the traditional annotation strategy in many key evaluation indicators. Specifically, these indicators include BLUE_1, BLUE_2, BLUE_4, METEOR, and ROUGE_L. Further, by comparing the experimental data in Table 3 and Table 4, it can be observed that even if the decoder is changed, the annotation data generated by the hierarchical annotation strategy still exhibits significant advantages in multiple indicators, especially in the BLUE_1 indicator, which shows outstanding performance. This series of experimental results fully demonstrate that the hierarchical annotation strategy indeed has significant advantages in the SAR image description task, and compared with the traditional annotation strategy, it can more effectively improve the performance of the model, providing strong support for the further development of the SAR image description task.

[0091] However, the hierarchical annotation strategy performs poorly in the CIDEr indicator. Under the traditional annotation strategy, the CIDEr values of different combinations are stable at about 0.15, while under the hierarchical annotation strategy, it is only about 0.8. This indicates that the hierarchical annotation strategy lacks in semantic richness and diversity, which may limit the semantic range of the generated description, resulting in insufficient semantic matching with the reference description. Therefore, although the hierarchical annotation strategy performs well in other indicators, the CIDEr indicator reveals its limitations in semantic consistency, prompting us to consider the advantages and disadvantages of different annotation strategies in practical applications and further optimize the hierarchical annotation strategy to improve its semantic consistency performance and meet the diversified needs of the SAR image description task.

[0092] In order to further explore the performance of different architecture combinations in the SAR image description task, we conducted extensive experiments on the SARAircraft-Captions dataset with various combinations of encoders and decoders, and the specific experimental results are shown in Table 1 and Table 3.

[0093] The experimental results show that different encoder architectures have their own characteristics in feature extraction and semantic information expression, and the choice of decoder significantly affects the quality and fluency of the generated description. Among them, ResNet network as an encoder performs well when paired with different decoders. Especially when LSTM is used as the decoder, the performance of ResNet is particularly outstanding, with about 1 percentage point higher than other network combinations in BLUE_1 and BLUE_2 indicators, and slightly better in other indicators. This shows that the ResNet network has a significant advantage in extracting SAR image features and expressing semantic information, especially when combined with the LSTM decoder, it can generate high-quality and fluent descriptions. However, when the Transformer is used as the decoder, although its effect is not as significant as LSTM, it is still better than other network combinations in BLUE_3, BLUE_4, and ROUGE_L three indicators.

[0094] In summary, the ResNet network as an encoder is superior to other networks in the SAR image description task, especially when combined with the LSTM decoder, it can generate higher quality descriptions. This advantage may be attributed to the depth architecture and residual connection mechanism of the ResNet network, which enables it to effectively extract complex features in the image and provide rich semantic information for the decoder. For the choice of decoder, the experimental results show that the performance of the Transformer model is not as superior as expected. This may be limited by the characteristics of the SARAircraft-Captions dataset: although it is more complex in terms of sentence number and hierarchy, the sentences are relatively short and the sentence structure is relatively simple. The Transformer model has a significant advantage in generating long sequence sentences, but the simplicity of this sentence structure may limit its performance. In addition, the Transformer model usually needs a larger dataset for training to fully learn the diversity and complexity of language. In this experiment, the size of the dataset may still not be sufficient to support the full potential of the Transformer model.

[0095] To evaluate the performance of the encoder-decoder model architecture in generating natural sentences, ResNet101 was used as the encoder and LSTM as the decoder, trained on the training set of the SARAircraft-Captions dataset, and verified on the test set for its sentence generation effect on SAR images. The specific generation results are shown in Figure 5

[0096] Figure 5 ​The experimental results in the above show that the generated sentences have different description characteristics at three levels of target, relationship and scene. At the target level, the generated sentences mainly focus on the description of the number and size of the target, and at the target type level, the description is limited to the aircraft category and lacks more refined classification description. At the relationship level, the generated sentences can express the relationship between targets and the relationship between targets and scenes, but these descriptions are mostly general and mainly reflect the connection or proximity relationship between targets, lacking more complex interaction description. At the scene level, the generated sentences can better reflect the topographic features of the scene and the function or role of the scene.

[0097] Therefore, through the above experiments, preferably, the encoder of the target model adopts the ResNet network, and the encoder adopts the LSTM network.

[0098] In the above SAR image description method based on a multi-modal large model, in view of the structural complexity and labeling difficulty of SAR images in the semantic description task, a set of structured and expandable image-language modeling scheme is proposed. A hierarchical semantic modeling rule of "target-relation-scene" is designed, and a semi-automatic labeling framework based on a multi-modal large model is constructed, and a high-efficiency and high-consistency semantic labeling process is realized combined with artificial proofreading. Based on the above strategy, the largest SAR aircraft image description dataset SARAircraft-Captions is constructed. To verify the effectiveness of the dataset and the method, the mainstream encoder-decoder model is used to conduct system experiments on the image description task. The results show that the constructed dataset has significant effects on supporting semantic structure learning and improving language generation quality, especially under the architecture of ResNet network as the encoder and LSTM network as the decoder, the optimal performance is achieved. In addition, the experiment also reveals the influence of the decoder structure on the fluency and hierarchical expression ability of the sentence, which provides practical reference for subsequent model selection.

[0099] It should be understood that, although Figure 1 The steps in the flowchart of the above are displayed in sequence according to the direction of the arrow, but these steps are not necessarily executed in sequence according to the direction of the arrow. Unless explicitly stated in this article, the execution of these steps has no strict sequence limitation, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps in the above can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with other steps or at least part of the sub-steps or stages of other steps.

[0100] In one embodiment, as shown in Figure 6 A multi-modal large model driven SAR image description device is provided, comprising: a hierarchical annotation strategy construction module 200, a structural prompt word construction module 210, a preliminary semantic description acquisition module 220, and a SAR image semantic description fine-tuning module 230. The hierarchical annotation strategy construction module 200 is configured to construct a hierarchical annotation strategy for target-relation-scene three-layer cooperation according to the multi-layer relationship of the target in the scene. The structural prompt word construction module 210 is configured to construct a structural prompt word, which includes a description prompt instruction, a category prompt instruction, and a restriction word prompt instruction, wherein the description prompt instruction is constructed based on the hierarchical annotation strategy. The preliminary semantic description acquisition module 220 is configured to acquire a SAR image, input the SAR image and the structural prompt word into a multi-modal large model, and generate a hierarchical semantic description corresponding to each level in the hierarchical annotation strategy through API calling based on the structural prompt word. The hierarchical semantic descriptions are reconstructed and fused to obtain a preliminary semantic description of the SAR image. The SAR image semantic description fine-tuning module 230 is configured to analyze the quality of the preliminary semantic description, optimize the structural prompt word based on the quality analysis result, and correct the preliminary semantic description according to the semantic description standard to obtain the semantic description of the SAR image.

[0101] The specific limitations of the multi-modal large model driven SAR image description device can be referred to the limitations of the multi-modal large model driven SAR image description method described above, which will not be repeated here. Each module in the above multi-modal large model driven SAR image description device can be realized by software, hardware, and their combinations. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations of each module.

[0102] In one embodiment, a computer device is provided, which can be a terminal, and its internal structure diagram can be as shown in Figure 7As shown in the figure. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external terminal through the network connection. The computer program is executed by the processor to implement a SAR image description method based on a multi-modal large model driving. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0103] Those skilled in the art can understand that, Figure 7 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0104] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the following steps: According to the multi-layer relationship of the target in the scene, a hierarchical labeling strategy about target-relation-scene three-layer cooperation is constructed; A structural prompt word is constructed, the structural prompt word comprising a description prompt instruction, a category prompt instruction and a restriction word prompt instruction, wherein the description prompt instruction is constructed based on the hierarchical labeling strategy; An SAR image is obtained, and the SAR image and the structural prompt word are input into a multi-modal large model. The multi-modal large model generates the hierarchical semantic description corresponding to each level in the hierarchical labeling strategy through API calling according to the structural prompt word, reconstructs and fuses the hierarchical semantic descriptions to obtain a preliminary semantic description of the SAR image; The preliminary semantic description is analyzed in quality, and the structural prompt word is optimized according to the quality analysis result, and the preliminary semantic description is corrected according to the semantic description standard to obtain the semantic description of the SAR image.

[0105] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the following steps: According to the multi-layer relationship of the target in the scene, a hierarchical labeling strategy about target-relation-scene three-layer cooperation is constructed; A structural prompt word is constructed, the structural prompt word including a description prompt instruction, a category prompt instruction, and a restriction word prompt instruction, wherein the description prompt instruction is constructed based on the hierarchical labeling strategy; A SAR image is acquired, the SAR image and the structural prompt word are input into a multi-modal large model, the multi-modal large model generates a hierarchical semantic description corresponding to each layer in the hierarchical labeling strategy through an API calling mode according to the structural prompt word, and the hierarchical semantic descriptions are reconstructed and fused to obtain a preliminary semantic description of the SAR image; The preliminary semantic description is analyzed in quality, the structural prompt word is optimized according to the quality analysis result, and the preliminary semantic description is corrected according to a semantic description standard to obtain a semantic description of the SAR image.

[0106] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0107] Each technical feature of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present disclosure.

[0108] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A SAR image description method based on multimodal large model driving, characterized by: The method comprises: Based on the multi-layered relationships of the target in the scene, a hierarchical labeling strategy for the three-layer collaboration of target, relationship, and scene is constructed; Constructing structural prompt words, wherein the structural prompt words include description prompt instructions, category prompt instructions, and restriction word prompt instructions, wherein the description prompt instructions are constructed based on the hierarchical labeling strategy; Acquire a SAR image, input the SAR image and structural prompt words into a multimodal large model, and generate hierarchical semantic descriptions corresponding to each level of the hierarchical annotation strategy based on the structural prompt words through API calls, reconstruct and fuse the semantic descriptions of each level to obtain a preliminary semantic description of the SAR image; The preliminary semantic description is subjected to quality analysis, the structural prompt words are optimized according to the quality analysis result, and the preliminary semantic description is corrected according to the semantic description standard to obtain the semantic description of the SAR image.

2. The SAR image description method based on multimodal large model driving according to claim 1 is characterized in that: The hierarchical labeling strategy includes: Target-level annotations used to describe the basic attributes of core targets; Relationship-level annotations for encoding spatial and structural relationships between objects; And, scene-level annotation for understanding the overall semantic and functional information of the image.

3. The SAR image description method based on multimodal large model driving according to claim 2 is characterized in that: The target level annotation includes target category information, target quantity and distribution information, and target estimated size and related size information; The relationship level annotation includes the position and arrangement relationship between the target and other targets, the interaction relationship between the target and the scene, and the relative distance and direction attributes between the target and other targets; The scene level annotation includes scene type, scene functionality judgment and scene environment information.

4. The SAR image description method based on multimodal large model driving according to claim 3 is characterized in that: When constructing the structural prompt words: The description prompt instruction sets three types of structured prompts at the target layer, the relationship layer, and the scene layer according to the hierarchical annotation strategy; The category prompt instruction sets a variety of target category prompts related to the scene where the target is located based on prior knowledge; The restrictive word prompt instruction sets the constraint sentence format of the preliminary semantic description according to description requirements.

5. The SAR image description method based on multimodal large model driving according to any one of claims 1 to 4, characterized in that: After obtaining the semantic description of the SAR image: The semantic description is used as a semantic description label of the corresponding SAR image, and corresponding semantic description labels are generated according to multiple SAR images to construct a training data set; The target model is trained using the training data set, so that the trained target model has the ability to semantically describe the SAR image.

6. The SAR image description method based on multimodal large model driving according to claim 5 is characterized in that: The target model is constructed by an encoder and a decoder. When the target model is trained: Inputting a SAR image in the training data set into the encoder to extract features and obtain image features; Processing the semantic description labels corresponding to the SAR images using an embedding layer to obtain language embedding; Inputting the image features and language embedding into the decoder to obtain semantic description pseudo labels; By using the loss function calculated using the semantic description pseudo-label and the semantic description label, the adjustable parameters in the encoder and decoder are adjusted until convergence to obtain a trained target model.

7. The SAR image description method based on multimodal large model driving according to claim 6 is characterized in that: The encoder adopts ResNet network, and the decoder adopts LSTM network.

8. A SAR image description device driven by a multimodal large model, characterized in that: The device comprises: A hierarchical annotation strategy construction module is used to construct a hierarchical annotation strategy for the three-layer coordination of target, relationship, and scene based on the multi-layer relationship of the target in the scene. A structural prompt word construction module, used to construct structural prompt words, wherein the structural prompt words include description prompt instructions, category prompt instructions, and restriction word prompt instructions, wherein the description prompt instructions are constructed based on the hierarchical labeling strategy; A preliminary semantic description acquisition module is used to acquire a SAR image, input the SAR image and structural prompt words into a multimodal large model, and the multimodal large model generates hierarchical semantic descriptions corresponding to each level of the hierarchical annotation strategy based on the structural prompt words through API calls, and reconstructs and fuses the semantic descriptions of each level to obtain a preliminary semantic description of the SAR image; The SAR image semantic description fine-tuning module is used to perform quality analysis on the preliminary semantic description, optimize the structural prompt words according to the quality analysis results, and correct the preliminary semantic description according to the semantic description standard to obtain the semantic description of the SAR image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images