Synthetic aperture radar ship identification method and device

Abstract

The application provides a synthetic aperture radar ship identification method and device, which can be applied to the field of synthetic aperture radar image interpretation and artificial intelligence technology. In view of the problems of fuzzy ship structure component identification, difficulty in using expert prior knowledge and insufficient small sample recognition ability in the prior art, the application constructs a ship structure semantic label system, obtains the point selection operation of a user on key components in an image, records component label and spatial position information, combines a pre-constructed standard component layout description to construct a structured prompt word for a large language model, uses the large language model for semantic matching and topological reasoning, outputs a candidate ship model and a confidence degree, and iteratively optimizes the identification result through user feedback. The application realizes fine-grained identification of man-machine cooperation, improves the identification precision and interpretability, and has the generalization ability in the small sample scene.

Description

Technical Field

[0001] This invention relates to the fields of synthetic aperture radar image interpretation and artificial intelligence technology, and more specifically to a synthetic aperture radar ship identification method and apparatus. Background Technology

[0002] Synthetic Aperture Radar (SAR) possesses all-weather, all-day Earth observation capabilities, unaffected by environmental conditions such as clouds, lighting, and sea fog, playing a crucial role in marine surveillance and ship target identification. In recent years, deep learning-based methods for ship identification in SAR images have made significant progress, particularly convolutional neural networks, which have demonstrated excellent performance in overall detection rate.

[0003] However, existing methods still have the following technical problems in actual shipping scenarios:

[0004] First, structural component identification is ambiguous. Existing algorithms mostly focus on the overall features of a ship, making it difficult to accurately distinguish key functional components such as the bow, stern, superstructure, cargo hold openings, and pipeline areas. This results in insufficient fine-grained identification capabilities for ships, making it difficult to meet the needs of refined interpretation.

[0005] Second, prior expert knowledge is difficult to utilize effectively. Ship type identification heavily relies on structural layout knowledge. For example, bulk carriers typically have large open cargo holds and a regular structural layout; container ships are equipped with dedicated column bases and deck stabilization structures; oil tankers have dedicated liquid cargo tank areas and explosion-proof work areas; and ferries have continuous passenger decks and dedicated boarding areas. Traditional deep learning models are "black box" structures, making it difficult to explicitly incorporate and reason about the aforementioned structured expert knowledge, thus limiting the reliability and interpretability of the identification results.

[0006] Third, the problems of small sample size and long tail are prominent. For new or rare ships, due to the lack of sufficient training data, the generalization ability of purely data-driven deep learning methods is poor, making it difficult to accurately identify ship models with a small sample size. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] To address the problems of fuzzy identification of ship structural components, difficulty in explicitly utilizing expert prior knowledge, and insufficient identification capabilities in small sample and long-tail scenarios in existing technologies, this invention provides a synthetic aperture radar (SAR) ship identification method and apparatus. By constructing a semantic tagging system for ship structures, it obtains the point selection operations performed by users on key components in SAR images, and combines them with pre-constructed standard component layout descriptions. The component tags, spatial location information, and component layout features are jointly constructed into structured prompt words for a large language model. The large language model performs semantic matching and topological reasoning, outputs candidate ship models and confidence levels, and iteratively optimizes the identification results through user feedback, thereby improving the fine-grained accuracy, interpretability, and adaptability to rare targets.

[0009] (II) Technical Solution

[0010] To address the aforementioned technical problems, embodiments of the present invention propose a synthetic aperture radar (SAR) method and apparatus for ship identification.

[0011] According to a first aspect of the present invention, a synthetic aperture radar (SAR) ship identification method is provided, comprising: constructing a semantic tagging system for ship structures, the semantic tagging system including multiple semantic tags, each semantic tag describing a key component of the ship; acquiring a point selection operation performed by a user on a SAR image, the point selection operation being the user's annotation of at least one key component of the ship in the image based on the semantic tags, and recording the component tag and spatial location information corresponding to the point selection operation; acquiring a pre-constructed standard component layout description, the standard component layout description including at least one ship model and component layout features corresponding to the ship model; and, based on the component tag, spatial location information, and... Component layout features retrieved from standard component layout descriptions are used to construct structured prompts for a large language model. These structured prompts are then input into the large language model, which performs semantic matching and topological inference based on them, outputting at least one candidate ship model and its corresponding confidence score. The candidate ship models and their corresponding confidence scores are then fed back to the user. If the user confirms the feedback, the final recognition result is output. If the user corrects their selection or adds a new selection, the structured prompt construction step and the large language model inference step are repeated based on the updated selection until the recognition result converges or the user confirms.

[0012] In some exemplary embodiments, the semantic labels include at least one of the following: labels for describing the bow, labels for describing the stern, labels for describing pipeline areas, labels for describing containers, and labels for describing island superstructures; the standard component layout description includes descriptive text for concretely identifiable targets and / or appearance descriptive text for targets that cannot be concretely identifiable; wherein, the appearance descriptive text replaces the component name with a description of scattering features in synthetic aperture radar images, and the scattering features include strong scattering areas, strip-shaped strong scattering bands, block-shaped structures, or white strip-shaped targets.

[0013] In some exemplary embodiments, the spatial location information is the bounding box coordinates or center point coordinates corresponding to the user's click operation. Before constructing the structured prompt words, the method further includes: normalizing the bounding box coordinates or center point coordinates to obtain normalized coordinates; the normalization process uses the ship's main axis direction, the estimated ship length, and the estimated ship width as reference factors; wherein, the ship's main axis direction is obtained by fitting the coordinates of the bow component and the stern component selected by the user; the estimated ship length is the distance between the bow and the stern; and the estimated ship width is the maximum width of the hull perpendicular to the ship's main axis direction.

[0014] In some exemplary embodiments, constructing structured prompts for a large language model includes: when the number of labeled components is at least two, for any two labeled components, calculating the relative orientation relationship between the two components based on their normalized coordinates, the relative orientation relationship including the relative distance between the two components in the ship's length direction, the relative distance between the two components in the ship's width direction, and the angle of the line connecting the two components; converting the relative orientation relationship into a natural language description string, and using the natural language description string as spatial topological relationship text; generating user selection information text based on component labels and spatial location information; obtaining preset role setting text and task description text, wherein the role setting text is used to set the identity role of the large language model, and the task description text is used to describe the current recognition task; concatenating the role setting text, task description text, user selection information text, and spatial topological relationship text to generate structured prompts.

[0015] In some exemplary embodiments, the user-selected information text includes at least: the label name of the selected component, the numerical values ​​of the component's bounding box coordinates or center point coordinates, and the numerical values ​​of the component's normalized coordinates.

[0016] In some exemplary embodiments, after feeding back the candidate ship models and their corresponding confidence scores to the user, the method further includes: calculating the information entropy of the current recognition result, or calculating the confidence score difference between the candidate ship models; when the information entropy exceeds a preset threshold, or when the confidence score difference is less than a preset threshold, triggering the large language model to generate guiding query information; the guiding query information is used to prompt the user to supplement the point selection operation for specific key components, or to prompt the user to check specific areas of the image; after the user supplements the point selection operation or checks according to the guiding query information, the information supplemented by the user is used as the updated point selection operation, and the process returns to execute the structured prompt word construction step and the large language model inference step.

[0017] In some exemplary embodiments, after a user corrects a click operation or adds a new click operation, the corrected information or the added information is used as historical context; when the structured prompt word construction step is repeated, a new structured prompt word is constructed based on the historical context and the updated click operation; wherein, the historical context is composed of the historical context information of the previous round and the corrected information or the added information input by the user in the current round.

[0018] In some exemplary embodiments, the large language model generates an inference logic chain during the inference process. The inference logic chain can be a step-by-step textual inference chain, a visual heatmap inference chain, or a tree-like inference chain. When outputting the final recognition result, the following information is also output: ship model, key component distribution map, confidence level of each candidate ship model, and inference logic chain.

[0019] In some exemplary embodiments, the selection operation includes clicking or selecting by bounding box; wherein, when the selection operation is clicking, the spatial location information is the coordinates of the clicked point; when the selection operation is selecting by bounding box, the spatial location information is the coordinates of the bounding box of the selected area.

[0020] According to a second aspect of the present invention, a synthetic aperture radar (SAR) ship identification device is provided, comprising: a semantic construction module for constructing a semantic tagging system for ship structures, the semantic tagging system including multiple semantic tags, each semantic tag describing a key component of the ship; a first acquisition module for acquiring a point selection operation performed by a user on a SAR image, the point selection operation being the user's annotation of at least one key component of the ship in the image according to the semantic tags, and recording the component tags and spatial location information corresponding to the point selection operation; a second acquisition module for acquiring a pre-constructed standard component layout description, the standard component layout description including at least one ship model and component layout features corresponding to the ship model; and a first construction module for constructing a system based on component tags. The system uses spatial location information and component layout features retrieved from standard component layout descriptions to construct structured prompt words for a large language model. The result output module inputs these structured prompt words into the large language model, which performs semantic matching and topological inference based on the prompt words, outputting at least one candidate ship model and its corresponding confidence score. The result confirmation module provides the candidate ship models and their corresponding confidence scores to the user. If the user confirms the feedback, the final recognition result is output. If the user corrects their selection or adds a new selection, the structured prompt word construction step and the large language model inference step are repeated based on the updated selection until the recognition result converges or the user confirms.

[0021] (III) Beneficial Effects

[0022] As can be seen from the above technical solutions, the synthetic aperture radar ship identification method and apparatus provided by the embodiments of the present invention have at least the following beneficial effects:

[0023] (1) Improved fine-grained recognition accuracy. By constructing a semantic labeling system for ship structures and guiding users to select key components such as the bow, pipeline area, and superstructure, the structural cognition of human experts is integrated into the recognition process, overcoming the shortcomings of traditional methods that rely solely on overall features and are difficult to distinguish fine-grained components, thus significantly improving the accuracy of ship model identification.

[0024] (2) Enhanced interpretability of recognition results. This invention utilizes the semantic matching and topological reasoning capabilities of a large language model to not only output candidate ship models and confidence levels, but also generate an recognition report containing a logical chain of reasoning. This transforms the recognition process from a “black box” decision-making process into traceable and understandable symbolic reasoning, thereby improving the credibility of the system.

[0025] (3) It realizes the organic integration of expert prior knowledge and machine knowledge base. The present invention pre-constructs standard component layout descriptions (including the correspondence between ship models and component layout features) and explicitly guides the large language model to reason through structured prompts, which solves the problem that traditional deep learning methods are difficult to explicitly utilize structured ship layout knowledge and improves the robustness of identification of complex and rare ships.

[0026] (4) It has the ability to generalize in small sample and long-tail scenarios. This invention does not rely on a large number of labeled samples to train the network. Instead, it uses a knowledge-driven human-computer collaboration approach to reason using the internal knowledge of standard component layout descriptions and large language models. For new or rare ships with scarce data, users only need to select a few key components and combine them with template matching to complete the identification, which effectively alleviates the generalization bottleneck of pure data-driven methods in long-tail distribution.

[0027] (5) An iterative optimization mechanism for human-machine collaboration has been realized. This invention triggers a new round of reasoning through user feedback (correction or supplementation of point selection operation) and introduces uncertainty guidance strategies (such as information entropy and confidence difference judgment). The large language model actively generates guiding inquiry information to prompt users to pay attention to the feature regions with the most discriminative power, thereby reducing the cognitive load of humans and improving recognition efficiency and accuracy. Attached Figure Description

[0028] The above-described features, other objects, and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0029] Figure 1 A flowchart illustrating a synthetic aperture radar ship identification method based on semantic tag selection and a large language model according to an embodiment of the present invention is shown.

[0030] Figure 2 A flowchart illustrating the construction of structured prompt words for large language models according to an embodiment of the present invention is shown.

[0031] Figure 3 This schematic diagram illustrates the structural block diagram of a synthetic aperture radar ship identification device based on semantic tag selection and a large language model according to an embodiment of the present invention.

[0032] Figure 4 A block diagram of an electronic device for a synthetic aperture radar ship identification method based on semantic tag selection and a large language model according to an embodiment of the present invention is shown. Detailed Implementation

[0033] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0034] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0035] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0036] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0037] Definition of words:

[0038] Large Language Model: The large language model involved in this invention refers to a natural language processing model pre-trained on general text data based on a deep learning architecture. This type of model has semantic understanding, logical reasoning, and text generation capabilities. It can parse task intent based on input prompts, extract structured information, perform multi-step reasoning, and generate output text that conforms to language norms.

[0039] In this invention, the large language model serves as the inference engine, undertaking the following functions:

[0040] Semantic matching: Parse the component labels, spatial location information and component layout descriptions in the structured prompts and compare them with the ship structure knowledge stored in the model.

[0041] Topological reasoning: Based on the relative orientation relationships between user-selected components (such as front and rear positions, left and right offsets, distance ratios, etc.), and combined with the spatial constraints in the standard component layout description, determine whether the overall structure of the ship conforms to the typical layout of a certain model.

[0042] Confidence assessment: Output the candidate ship models and their corresponding confidence scores to indicate the degree of certainty of the reasoning results.

[0043] Uncertainty guidance: When the current identification information is insufficient or the distinction between candidate models is ambiguous, guiding questions are generated to prompt the user to supplement the annotation of key components or to further verify specific image areas.

[0044] The large language model used in this invention can be a general pre-trained model (e.g., a generative model based on an encoder-decoder structure). Its specific implementation is not limited to a particular version or vendor, as long as it has the aforementioned semantic understanding and reasoning capabilities. In use, a general open-source model can be loaded or a commercial model interface can be called. No additional fine-tuning is required for the ship identification task. Knowledge retrieval and reasoning can be achieved through structured prompts.

[0045] Figure 1 The flowchart illustrates a synthetic aperture radar ship identification method based on semantic tag selection and a large language model according to an embodiment of the present invention.

[0046] like Figure 1 As shown, the synthetic aperture radar ship identification method based on semantic tag selection and large language model according to an embodiment of the present invention includes steps S110 to S160.

[0047] In step S110, a semantic tagging system for ship structures is constructed. The semantic tagging system includes multiple semantic tags, each of which is used to describe a key component of the ship.

[0048] In embodiments of the present invention, the semantic tags include at least one of the following: a tag describing the bow of the ship, a tag describing the stern of the ship, a tag describing a pipeline area, a tag describing a container, and a tag describing an island superstructure. In practical applications, tags can be added or refined as needed.

[0049] For example, defining a semantic tagging system covering key functional components of a ship. L1 represents the bow, characterized by a sharp structure; L2 represents the stern, characterized by the helicopter landing area; L3 represents the pipeline area, characterized by strip-shaped bright scattering points on the foredeck; L4 represents containers, characterized by regularly arranged rectangular or honeycomb-shaped strong scattering areas in the fore or midsection; L5 represents the island superstructure, characterized by a tall, blocky structure in the mid-to-aft section of the hull. In practical applications, labels can be added or refined as needed, for example, including helicopter deck, deck piping, rectangular hatch covers, planar areas, and strong scattering areas.

[0050] At the same time, a ship knowledge template library was built. Knowledge Template Library Each template Includes ship model and its corresponding standard component layout description text The standard component layout description text uses at least one of the following two description methods:

[0051] The first type is a descriptive method targeting concrete cognitive goals. For example: "The superstructure of an oil tanker is usually the bridge at the stern, and the pipeline area extends from the bridge to the bow."

[0052] The second method is a descriptive approach for targets that cannot be visualized. This involves using scattering characteristics from synthetic aperture radar (SAR) images to replace component names. For example: "The stern of an oil tanker typically exhibits a strong scattering area, appearing as a tall bridge structure; the middle of the hull displays a strip-shaped strong scattering band that extends to the bow, appearing as a white strip-shaped target."

[0053] In practical applications, the standard component layout description text can be a combination of the two types of description texts mentioned above. For example: "The stern of an oil tanker is generally the superstructure of the bridge, and the pipeline area extends from the bridge forward to the bow; correspondingly, the stern of an oil tanker generally has a strong scattering area, which is represented by the high-rise bridge structure, and the middle of the hull presents a strip-shaped strong scattering band that extends all the way to the bow, which is represented by a white strip-shaped target."

[0054] In step S120, the point selection operation performed by the user on the synthetic aperture radar image is obtained. The point selection operation is the user's annotation of at least one key ship component in the image based on semantic tags. The component tags and spatial location information corresponding to the point selection operation are recorded.

[0055] In embodiments of the present invention, the selection operation includes clicking or selecting by bounding box; wherein, when the selection operation is clicking, the spatial location information is the coordinates of the clicked point; when the selection operation is selecting by bounding box, the spatial location information is the coordinates of the bounding box of the selected area.

[0056] The user loads the synthetic aperture radar image I to be identified on the front-end interface. Based on visual judgment, the user selects a label l from the label set L. k Then, select the corresponding area on image I. The selection operation includes clicking or selecting by bounding box; when the selection operation is clicking, the spatial location information is the coordinates of the clicked point; when the selection operation is selecting by bounding box, the spatial location information is the coordinates of the bounding box of the selected area.

[0057] The system records the user's click operation sequence. ,in The coordinates of the bounding box or center point of the i-th selected area.

[0058] In step S130, a pre-constructed standard component layout description is obtained. This description includes at least one ship model and component layout features corresponding to that model. In practice, the system retrieves the matching standard component layout description text D from the knowledge template base K based on the component tag selected by the user. spatial .

[0059] In embodiments of the present invention, the standard component layout description includes descriptive text for concretely cognizable targets and / or appearance description text for targets that cannot be concretely cognized; wherein, the appearance description text replaces the component name with a description of scattering features in synthetic aperture radar images, and the scattering features include strong scattering areas, strip-shaped strong scattering bands, block-shaped structures, or white strip-shaped targets.

[0060] In step S140, structured prompt words for a large language model are constructed based on component labels, spatial location information, and component layout features retrieved from standard component layout descriptions.

[0061] This step will select the sequence by the user. The retrieved component layout features are used to construct structured prompt words. . Formalized as:

[0062]

[0063] in, This represents the role setting text, used to define the identity role of the large language model (e.g., "You are a ship identification expert"). This indicates the task description text, such as "Please analyze the type of the target components based on their relative positions. Note that some components are difficult to describe and will be directly represented by their appearance." This indicates the text of the information selected by the user, i.e., the list of parts confirmed by the user, such as "User has confirmed the existence of: Bow:point1:1163,596 Stern:point2:757,684, Helicopter Deck:rectangle3:838,641,882,682 Island:rectangle5:1002,612,1042,648, Funnel:rectangle4:969,615,1008,638, Cargo Hold Opening:rectangle4:922,643,901,673"; The text representing spatial topological relationships is a description of the topological relationships between components retrieved from the knowledge template base K. For example, "The stern of an oil tanker is generally the superstructure of the bridge, and there is a pipeline area extending from the bridge to the bow; correspondingly, the stern of an oil tanker generally has a strong scattering area, which is represented by the high-rise bridge structure, and the middle of the hull presents a strip-shaped strong scattering band that extends to the bow, which is represented by a white strip-shaped target."

[0064] Before constructing the structured prompts, the method also includes normalization of spatial location information. Specifically, the coordinates of the bounding box or center point are normalized to obtain normalized coordinates. The normalization process uses the ship's main axis direction, estimated ship length, and estimated ship width as benchmark factors. The ship's main axis direction is obtained by fitting the coordinates of the bow and stern components selected by the user; the estimated ship length is the distance between the bow and stern; and the estimated ship width is the maximum width of the hull perpendicular to the ship's main axis direction.

[0065] Based on this, spatial relationship encoding is performed. Let the center coordinates of the component i selected by the user be... Normalized to the interval [0,1]. For any two marked components i and j, their relative orientation relationship... The encoding is:

[0066]

[0067] in, , These are the estimated ship length and width, respectively. The angle of the line connecting the center points of the two components.

[0068] Figure 2 The flowchart illustrating the construction of structured prompt words for large language models according to an embodiment of the present invention is shown.

[0069] like Figure 2 As shown, the method for constructing structured prompt words for large language models according to an embodiment of the present invention includes steps S210 to S250.

[0070] In step S210, when the number of marked components is at least two, for any two marked components, the relative orientation relationship between the two components is calculated based on the normalized coordinates of the two components. The relative orientation relationship includes the relative distance between the two components in the length direction of the ship, the relative distance between the two components in the width direction of the ship, and the angle of the line connecting the two components.

[0071] In step S220, the relative orientation relationship is converted into a natural language description string, and the natural language description string is used as the spatial topological relationship text.

[0072] For example, if the normalized coordinates of the pipeline area are (0.6, 0.5) and the normalized coordinates of the bow are (0.2, 0.5), then it can be transformed into "the pipeline area is located 0.4 times the length of the ship behind the bow, with no lateral offset".

[0073] In step S230, user selection information text is generated based on component labels and spatial location information.

[0074] In embodiments of the present invention, the user-selected information text includes at least: the label name of the selected component, the numerical values ​​of the component's bounding box coordinates or center point coordinates, and the numerical values ​​of the component's normalized coordinates.

[0075] For example: "User has confirmed the existence of: bow (center point: 1163, 596), stern (center point: 757, 684), pipeline area (boundary box: 922, 643, 901, 673)".

[0076] In step S240, preset role setting text and task description text are obtained. The role setting text is used to set the identity role of the large language model, such as "You are a ship identification expert". The task description text is used to describe the current identification task, such as "Please analyze the ship model based on the relative positional relationship of the target parts. Note that some parts may be described by appearance instead of name".

[0077] In step S250, the role setting text, task description text, user selection information text, and spatial topology relationship text are concatenated to generate structured prompt words.

[0078] The splicing order can be any predefined order. In this embodiment, the splicing is done in the order of "role-task-user selection-spatial relationship".

[0079] In step S150, the structured prompt words are input into the large language model, which performs semantic matching and topological reasoning based on the structured prompt words, and outputs at least one candidate ship model and the confidence level corresponding to each candidate ship model.

[0080] Structured prompts Input is fed into a pre-trained Large Language Model (LLM). The LLM is based on... Semantic matching and topological reasoning are performed to activate its internal knowledge base, and the similarity score between the user input features and each model in the knowledge template base K is calculated. The system outputs at least one candidate ship type and the corresponding confidence level for each candidate ship type. For example, the output might be: "Type A oil tanker: 0.85; Type D improved: 0.12; Bulk carrier: 0.03". Simultaneously, LLM generates inference logic chains during the inference process, which can be step-by-step textual inference chains, visual heatmap inference chains, or tree-structured inference chains.

[0081] In step S160, the candidate ship models and their corresponding confidence levels are fed back to the user. If the user confirms the feedback results, the final recognition results are output. If the user corrects the selection operation or adds a new selection operation, the structured prompt word construction step and the large language model inference step are repeated according to the updated selection operation until the recognition results converge or the user confirms.

[0082] To further optimize the iterative process, this embodiment also includes an uncertainty guidance mechanism. Specifically, after feeding back the candidate ship models and their corresponding confidence scores to the user, the mechanism further includes: calculating the information entropy of the current recognition result, or calculating the confidence score difference between the candidate ship models; when the information entropy exceeds a preset threshold (e.g., 0.6), or when the confidence score difference is less than a preset threshold (e.g., 0.2), the large language model is triggered to generate guiding query information. The guiding query information is used to prompt the user to supplement the point selection operation for specific key components, or to prompt the user to check specific areas of the image. After the user supplements the point selection operation or checks according to the guiding query information, the information supplemented by the user is used as the updated point selection operation, and the process returns to execute the structured prompt word construction step and the large language model inference step.

[0083] Furthermore, after a user corrects or adds a new click action, the corrected or added information is used as historical context. When the structured prompt word construction step is repeated, new structured prompt words are constructed based on the historical context and the updated click action. The historical context consists of the previous round's historical context information and the corrected or added information input by the user in the current round.

[0084] In this embodiment, the large language model generates inference logic chains during the inference process. These inference logic chains can be step-by-step text-based inference chains, visual heatmap inference chains, or tree-structured inference chains. When outputting the final recognition result, the following information is also output: ship model, key component distribution map, confidence level of each candidate ship model, and inference logic chain.

[0085] Example 1: A specific workflow example of user-selected oil tanker identification

[0086] Initial state: The operator loads a SAR image, and the system automatically outlines the ship's silhouette. The operator clicks the "Bow" tab to draw a frame on the bow; clicks the "Pipeline Area" tab to draw a long strip behind the bow; and clicks "Helicopter Deck" to draw a frame at the stern.

[0087] Reasoning: The system converts the positions and labels of these boxes into text: "Bow, coordinates XX, followed by pipeline area, coordinates XX, stern has helicopter deck, coordinates XX, and in the middle is a tall integrated mast." The LLM receives the instructions and performs retrieval and topological reasoning based on the ship knowledge template base.

[0088] LLM output: "Based on the strong scattering characteristics of the marked 'pipeline area,' it highly matches the characteristics of Type A oil tanker; since the midship section is an area that cannot be distinguished by SAR images, the possibility of Type D cannot be ruled out at this time. The core difference between Type D and Type A is whether there is a strong scattering area in the midship section with dedicated liquid cargo tanks. It is recommended to confirm the extension length and number of pipeline areas, or try to magnify the blurred area in the midship section to verify whether there are identifiable scattering characteristics."

[0089] Correction: The operator zoomed in on the image as prompted, added multiple continuous strip-shaped strong scattering features to the pipeline area, and rechecked the blurred area in the middle of the hull. After confirming that there were no identifiable scattering features of any ship components, the operator added the check results to the system annotation information.

[0090] Final judgment: LLM, based on the updated annotation information, reasoned again: "It was confirmed that there were strong scattering characteristics in multiple continuous pipeline areas on the hull, and there were no strong scattering characteristics of the liquid cargo tanks in the middle that are unique to the improved D type. Combined with the imaging characteristics of SAR images, the blurred area in the middle is a featureless area caused by sea state interference, not a lack of structural features of the ship itself, thus excluding the D series. The final judgment is that it is an oil tanker of type A."

[0091] Example 2: Active Learning Recommendation Strategy

[0092] When the information selected by the user is insufficient to uniquely identify the target, the system generates a recommended question using LLM: "To differentiate between Type A and Type B, please confirm whether the stern has a double hangar structure?" This guides the user to make further targeted selections, thereby minimizing identification uncertainty. For example, if the user initially only selected the bow and stern, the system outputs a confidence score of 0.52 for oil tankers of Type A and 0.48 for bulk carriers, exceeding the information entropy threshold. LLM proactively asks: "Please confirm whether there is a strip-shaped area of ​​strongly scattering pipelines extending from the bow to the stern in the midsection of the hull?" After the user adds the selection, the system re-infers the information, increasing the confidence score to 0.92.

[0093] In combination with the above Figures 1 to 2 Method and process Figure 3 From the perspective of functional modular design, the logical architecture of the system of the present invention during actual deployment is further demonstrated.

[0094] Figure 3 The diagram illustrates the structure of a synthetic aperture radar ship identification device based on semantic tag selection and a large language model according to an embodiment of the present invention.

[0095] like Figure 3As shown, the synthetic aperture radar ship identification device 300 based on semantic tag selection and large language model in this embodiment includes a semantic construction module 310, a first acquisition module 320, a second acquisition module 330, a first construction module 340, a result output module 350, and a result confirmation module 360.

[0096] The semantic construction module 310 is used to construct a semantic tagging system for ship structures. The semantic tagging system includes multiple semantic tags, each of which describes a key component of the ship. In one embodiment, the semantic construction module 310 can be used to perform the operation S110 described above, which will not be repeated here.

[0097] The first acquisition module 320 is used to acquire the point selection operation performed by the user on the synthetic aperture radar image. The point selection operation is the user's annotation of at least one key ship component in the image based on semantic tags, and the module records the component tags and spatial location information corresponding to the point selection operation. In one embodiment, the first acquisition module 320 can be used to execute the operation S120 described above, which will not be repeated here.

[0098] The second acquisition module 330 is used to acquire a pre-built standard component layout description, which includes at least one ship model and component layout features corresponding to the ship model. In one embodiment, the second acquisition module 330 can be used to perform the operation S130 described above, which will not be repeated here.

[0099] The first construction module 340 is used to construct structured prompt words for a large language model based on component labels, spatial location information, and component layout features retrieved from standard component layout descriptions. In one embodiment, the first construction module 340 can be used to perform the operation S140 described above, which will not be repeated here.

[0100] The result output module 350 is used to input structured prompt words into a large language model, which then performs semantic matching and topological reasoning based on the structured prompt words, and outputs at least one candidate ship model and the confidence score corresponding to each candidate ship model. In one embodiment, the result output module 350 can be used to perform the operation S150 described above, which will not be repeated here.

[0101] The result confirmation module 360 ​​is used to feed back the candidate ship models and their corresponding confidence levels to the user. If the user confirms the feedback result, the final recognition result is output. If the user corrects the selection operation or adds a new selection operation, the structured prompt word construction step and the large language model inference step are repeated according to the updated selection operation until the recognition result converges or the user confirms it. In one embodiment, the result confirmation module 360 ​​can be used to perform the operation S160 described above, which will not be repeated here.

[0102] According to embodiments of the present invention, any plurality of modules among the semantic construction module 310, the first acquisition module 320, the second acquisition module 330, the first construction module 340, the result output module 350, and the result confirmation module 360 ​​can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of the present invention, at least one of the semantic construction module 310, the first acquisition module 320, the second acquisition module 330, the first construction module 340, the result output module 350, and the result confirmation module 360 ​​can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in hardware or firmware, or in any one of software, hardware, and firmware implementations, or in a suitable combination of any of these. Alternatively, at least one of the semantic construction module 310, the first acquisition module 320, the second acquisition module 330, the first construction module 340, the result output module 350, and the result confirmation module 360 ​​can be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0103] Figure 4 A block diagram of an electronic device for a synthetic aperture radar ship identification method based on semantic tag selection and a large language model according to an embodiment of the present invention is shown.

[0104] like Figure 4 As shown, an electronic device 400 according to an embodiment of the present invention includes a processor 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage portion 408 into a random access memory (RAM) 403. The processor 401 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 401 may also include onboard memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0105] RAM 403 stores various programs and data required for the operation of electronic device 400. Processor 401, ROM 402, and RAM 403 are interconnected via bus 404. Processor 401 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 402 and / or RAM 403. It should be noted that programs may also be stored in one or more memories other than ROM 402 and RAM 403. Processor 401 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in one or more memories.

[0106] According to an embodiment of the present invention, the electronic device 400 may further include an input / output (I / O) interface 405, which is also connected to a bus 404. The electronic device 400 may also include one or more of the following components connected to the input / output (I / O) interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output (I / O) interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 410 as needed so that computer programs read from it can be installed into the storage section 408 as needed.

[0107] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0108] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of the present invention, a computer-readable storage medium may include ROM 402 and / or RAM 403 and / or one or more memories other than ROM 402 and RAM 403 described above.

[0109] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

Claims

1. A synthetic aperture radar method for ship identification, characterized in that, The method includes: A semantic tagging system for ship structures is constructed, wherein the semantic tagging system includes multiple semantic tags, and each semantic tag is used to describe a key component of the ship; The system acquires the point selection operation performed by the user on the synthetic aperture radar image. The point selection operation is the user's annotation of at least one key ship component in the image based on the semantic tags. The system records the component tags and spatial location information corresponding to the point selection operation. Obtain a pre-built standard component layout description, which includes at least one ship model and component layout features corresponding to the ship model; Based on the component labels, the spatial location information, and the component layout features retrieved from the standard component layout description, structured prompt words for large language models are constructed. The structured prompt words are input into a large language model, which performs semantic matching and topological reasoning based on the structured prompt words, and outputs at least one candidate ship model and the confidence level of each candidate ship model. The candidate ship models and their corresponding confidence levels are fed back to the user. If the user confirms the feedback, the final recognition result is output. If the user corrects the selection operation or adds a new selection operation, the structured prompt word construction step and the large language model inference step are repeated according to the updated selection operation until the recognition result converges or the user confirms.

2. The method according to claim 1, characterized in that, The semantic tags include at least one of the following: tags for describing the bow, tags for describing the stern, tags for describing the pipeline area, tags for describing containers, and tags for describing the island superstructure. The standard component layout description includes descriptive text for concrete, recognizable targets and / or appearance description text for targets that cannot be recognizable; wherein, the appearance description text replaces the component name with a description of scattering features in synthetic aperture radar images, and the scattering features include strong scattering areas, strip-shaped strong scattering bands, block-shaped structures, or white strip-shaped targets.

3. The method according to claim 1, characterized in that, The spatial location information is the bounding box coordinates or center point coordinates corresponding to the user's click operation. Before constructing the structured prompt, the method further includes: normalizing the bounding box coordinates or center point coordinates to obtain normalized coordinates; the normalization process uses the ship's main axis direction, the estimated ship length, and the estimated ship width as reference factors. The ship's main axis direction is obtained by fitting the coordinates of the bow component and the stern component selected by the user. The estimated length of the vessel is the distance between the bow and the stern. The estimated width of the vessel is the maximum width of the hull perpendicular to the direction of the vessel's main axis.

4. The method according to claim 3, characterized in that, The structured prompt words for constructing a large language model include: When the number of marked components is at least two, for any two marked components, the relative orientation relationship between the two components is calculated based on the normalized coordinates of the two components. The relative orientation relationship includes the relative distance between the two components in the length direction of the ship, the relative distance between the two components in the width direction of the ship, and the angle of the line connecting the two components. The relative orientation relationship is converted into a natural language description string, and the natural language description string is used as the spatial topological relationship text; Generate user selection information text based on the component label and the spatial location information; Obtain preset role setting text and task description text, wherein the role setting text is used to set the identity role of the large language model, and the task description text is used to describe the current recognition task; The structured prompt words are generated by concatenating the character setting text, the task description text, the user selection information text, and the spatial topology text.

5. The method according to claim 4, characterized in that, The user-selected information text includes at least: the label name of the selected component, the numerical values ​​of the bounding box coordinates or center point coordinates of the component, and the numerical values ​​of the normalized coordinates of the component.

6. The method according to claim 1, characterized in that, After feeding back the candidate ship models and their corresponding confidence levels to the user, the process also includes: Calculate the information entropy of the current identification result, or calculate the confidence difference between each candidate ship model; When the information entropy exceeds a preset threshold, or when the confidence difference is less than a preset threshold, the large language model is triggered to generate guiding inquiry information; the guiding inquiry information is used to prompt the user to supplement the selection operation of specific key components, or to prompt the user to check specific areas of the image; After the user supplements the selection operation or checks based on the guided inquiry information, the information supplemented by the user is used as the updated selection operation, and the process returns to execute the structured prompt word construction step and the large language model inference step.

7. The method according to claim 1, characterized in that, After a user corrects or adds a new click operation, the corrected or added information is used as historical context. When the structured prompt word construction step is repeated, a new structured prompt word is constructed based on the historical context and the updated click operation. The historical context is composed of the historical context information from the previous round and the corrected or added information input by the user in the current round.

8. The method according to claim 1, characterized in that, The large language model generates a reasoning logic chain during the reasoning process. The reasoning logic chain can be a step-by-step text-based reasoning chain, a visual heatmap reasoning chain, or a tree-like reasoning chain. When outputting the final recognition result, the following information is also output: ship model, key component distribution map, confidence level of each candidate ship model, and the reasoning logic chain.

9. The method according to claim 1, characterized in that, The selection operation includes clicking or selecting by box; When the selection operation is a click, the spatial location information is the coordinates of the clicked point; When the point selection operation is a box selection, the spatial location information is the coordinates of the bounding box of the selected area.

10. A synthetic aperture radar ship identification device, characterized in that, The device includes: A semantic construction module is used to construct a semantic tagging system for ship structures. The semantic tagging system includes multiple semantic tags, each of which describes a key component of the ship. The first acquisition module is used to acquire the point selection operation performed by the user on the synthetic aperture radar image. The point selection operation is the user's annotation of at least one key ship component in the image according to the semantic tags. The module records the component tags and spatial location information corresponding to the point selection operation. The second acquisition module is used to acquire a pre-built standard component layout description, which includes at least one ship model and component layout features corresponding to the ship model. The first construction module is used to construct structured prompt words for a large language model based on the component labels, the spatial location information, and the component layout features retrieved from the standard component layout description; The result output module is used to input the structured prompt words into the large language model, and the large language model performs semantic matching and topological reasoning based on the structured prompt words to output at least one candidate ship model and the confidence level corresponding to each candidate ship model. The result confirmation module is used to feed back the candidate ship model and its corresponding confidence level to the user. If the user confirms the feedback result, the final recognition result is output. If the user corrects the selection operation or adds a new selection operation, the structured prompt word construction step and the large language model inference step are repeated according to the updated selection operation until the recognition result converges or the user confirms.

Images