A maritime multi-modal trajectory optimization agent tool decision-making method

Abstract

The application discloses a kind of maritime multimodal trajectory optimization intelligent agent tool decision-making methods, obtain the problem text of maritime navigation task, maritime situation image and historical tool record, and construct trajectory input according to decision step order;Text representation, visual representation and tool trajectory representation are obtained by respectively encoding problem text, maritime situation image and historical tool record;The above-mentioned representation is input into multimodal time sequence attention model, the time sequence dependence relationship between modeling task semantics, image evidence and historical tool behavior is output fusion feature representation;Based on fusion feature representation, score candidate action set, determine tool selection result or stop decision;When not stopping, execute selected tool and write back output and feedback to historical tool record;In training stage, construct positive and negative trajectory set, and update decision strategy using trajectory level preference optimization. The application is used to improve the continuity, stability and traceability of maritime multi-step tool collaborative decision-making.

Description

Technical Field

[0001] This invention relates to the field of intelligent analysis technology for maritime data, specifically to a decision-making method for a maritime multimodal trajectory optimization intelligent agent tool. Background Technology

[0002] With the development of maritime informatization, ship-shore collaborative sensing, and intelligent decision support technologies, a large amount of text, images, rules, historical records, and external system feedback information is generated in business scenarios such as navigation safety, port scheduling, and maritime supervision. When handling tasks such as navigation situation assessment, risk alerts, rule queries, and decision support, related systems typically need to comprehensively utilize multi-source information and make continuous judgments between different analysis stages to support the business processing needs in complex maritime scenarios. While some existing systems can complete single-type information processing based on text retrieval, image recognition, or rule matching, when multiple sources of information participate in decision-making, there are still problems such as insufficient expression of information relationships, inadequate utilization of historical processing results, and difficulty in continuously tracking the decision-making chain.

[0003] In practical maritime applications, navigational situations often change with time, space, and interacting objects, and information from different sources is not simply parallel. For example, task descriptions, image evidence, historical query results, and intermediate feedback may all influence subsequent judgments. Existing decision support methods often focus on immediate responses to current inputs or rely on preset rules to complete local judgments, making it difficult to fully preserve preceding processing behaviors and their feedback constraints in continuous decision-making processes. When external tools are called multiple times or multiple rounds of reasoning are involved in the same task, issues such as duplicate processing, path redundancy, inconsistencies between preceding and subsequent judgments, or difficulty in tracing back intermediate evidence can easily arise, thus affecting the stability and auditability of maritime decision support results.

[0004] Therefore, how to uniformly organize task texts, situational images, and historical processing records without increasing the complexity of maritime operations, and how to enable the system to adjust subsequent decision-making processes based on existing processing trajectories, has become a problem that needs to be solved in the field of maritime intelligent decision support. Especially in scenarios involving multi-step analysis, multi-tool collaboration, and result feedback correction, a technical solution is needed that can balance multi-source information fusion, historical trajectory utilization, and decision-making process constraints to improve the continuity, traceability, and engineering applicability of maritime decision support processes.

[0005] In view of this, the present invention proposes a decision-making method for a maritime multimodal trajectory optimization intelligent agent tool. Summary of the Invention

[0006] The purpose of this invention is to provide a decision-making method for a maritime multimodal trajectory optimization intelligent agent tool, which addresses the problems of insufficient correlation of multi-source information, inadequate utilization of historical tool feedback, and inconsistent multi-step decision trajectories in existing maritime auxiliary decision-making.

[0007] This invention provides a decision-making method for a maritime multimodal trajectory optimization intelligent agent tool, comprising the following steps:

[0008] S1: Obtain the problem text, maritime situation image and historical tool records of the maritime navigation mission, and construct the problem text, maritime situation image and historical tool records into a trajectory input according to the decision step sequence. The historical tool records include the tool type, input parameters, output results and feedback information of the tools that have been called.

[0009] S2: Encode the problem text, maritime situation image and historical tool record respectively to obtain text representation, visual representation and tool trajectory representation, and map the text representation, visual representation and tool trajectory representation to a unified representation space;

[0010] S3: Input textual representation, visual representation, and tool trajectory representation into a multimodal temporal attention model to model the temporal dependencies between maritime mission semantics, situational image evidence, and historical tool behavior, and output a fused feature representation;

[0011] S4: Based on the fusion feature representation, perform action scoring on the candidate action set including maritime knowledge retrieval tools, maritime image retrieval tools, image description assistance tools, direct answering tools, and stop actions, and determine the tool selection result or stop decision for the current decision step;

[0012] S5: When the current decision step does not trigger a stop decision, execute the selected tool and obtain the tool output results and feedback information. Write the tool call information to the historical tool record to update the trajectory input, and return to execute S2 to S4 until a stop decision is triggered or the preset maximum number of iterations is reached.

[0013] S6: During the training phase, positive trajectory sets and negative trajectory sets are constructed based on maritime navigation missions. The trajectory-level normalized probability difference between the positive trajectory set and the negative trajectory set is calculated, and the trajectory-level preference optimization is used to optimize the decision-making strategy of the target update agent tool.

[0014] As a preferred embodiment of the present invention, the trajectory input is constructed as follows:

[0015] The problem text of maritime navigation missions is used as the semantic node of the mission, and the maritime situation image is used as the visual evidence node.

[0016] Each tool call encapsulates the tool type, input parameters, output results, and feedback information into a tool trajectory node;

[0017] Connect the tool trajectory nodes according to the order in which the tools are invoked, and organize the task semantic nodes and visual evidence nodes together with the tool trajectory nodes into a trajectory sequence;

[0018] At each decision step, the current trajectory sequence is read to form the trajectory input for use in multimodal coding.

[0019] As a preferred technical solution of the present invention, the maritime situation image includes at least one of electronic chart display image, radar situation image, shipborne camera image, port monitoring image or ship-shore collaborative situation image; the question text includes at least one of navigation rule query, collision risk warning, navigation situation description, route planning requirement or maritime safety question and answer requirement.

[0020] As a preferred embodiment of the present invention, encoding historical tool records includes:

[0021] Create action identifier vectors for the invoked tools based on the candidate action set;

[0022] Convert tool input parameters, tool output results, and feedback information into a tool invocation status representation;

[0023] By concatenating or mapping the action identifier vector with the tool call state representation, a single-step tool trajectory representation can be obtained.

[0024] Multiple single-step tool trajectory representations are aggregated according to the calling order to obtain tool trajectory representations that represent historical tool behavior.

[0025] As a preferred embodiment of the present invention, the multimodal temporal attention model includes a modality identifier embedding layer, a temporal position embedding layer, and an attention fusion layer; wherein:

[0026] The modality identification embedding layer is used to distinguish the information sources of text representation, visual representation and tool trajectory representation. The temporal position embedding layer is used to identify the decision step positions recorded by the historical tool in the trajectory sequence. The attention fusion layer is used to calculate the correlation weights between different modal information and different decision step tool behaviors, and generate fused feature representations based on the correlation weights.

[0027] As a preferred embodiment of the present invention, the tool selection result or decision-stopping step of the current decision step includes:

[0028] The fused feature representation is input into the decision head network to obtain the action score of each candidate action in the candidate action set.

[0029] The action score is scaled using a learnable temperature parameter to obtain the action probability distribution;

[0030] Select the candidate action corresponding to the current decision step based on the action probability distribution;

[0031] When the selected candidate action is a stop action, a stop decision is generated; when the selected candidate action is a tool action, tool call information containing the tool identifier and tool input content is generated.

[0032] As a preferred technical solution of the present invention, writing to the historical tool record includes: encapsulating the tool type, tool input parameters, tool output results, feedback information and corresponding decision step number of the current tool call into a new tool trajectory node, and appending the new tool trajectory node to the end of the current historical tool record, so that the trajectory input of the next decision step simultaneously includes the original maritime mission information and the executed tool call information.

[0033] As a preferred embodiment of the present invention, constructing the positive trajectory set and the negative trajectory set includes:

[0034] Multiple tool call trajectories are generated based on the same maritime navigation mission;

[0035] Based on the matching relationship between trajectory output results and task objectives, tool call steps, repeated calls, and consistency of intermediate feedback, multiple tool call trajectories are classified by quality.

[0036] Tool call trajectories that meet the trajectory quality conditions will be assigned to the positive trajectory set, while tool call trajectories that do not meet the trajectory quality conditions will be assigned to the negative trajectory set.

[0037] The positive trajectory set and negative trajectory set corresponding to the same maritime navigation mission constitute trajectory-level preference sample pairs.

[0038] As a preferred embodiment of the present invention, the construction of the trajectory-level preference optimization objective includes:

[0039] Calculate the log probability of each decision step action in each tool call trajectory;

[0040] The logarithmic probabilities of each decision step are normalized according to the trajectory length to obtain the trajectory-level normalized probabilities.

[0041] Introduce a reference policy and calculate the probabilistic advantage of the current decision policy over the reference policy on the positive trajectory set and the negative trajectory set, respectively.

[0042] A trajectory-level preference optimization loss function is constructed based on the probability advantage difference between the positive trajectory set and the negative trajectory set, and the agent tool decision-making strategy is updated based on the trajectory-level preference optimization loss function.

[0043] As a preferred embodiment of the present invention, the trajectory-level preference optimization loss function further includes an entropy regularization term. The entropy regularization term is constructed based on the action probability distribution of the current decision policy on the candidate action set, and is used to constrain the tool selection result from being overly concentrated on a single tool action during the training process. The agent tool decision policy updates its parameters through the joint objective of the trajectory-level preference optimization loss function and the entropy regularization term.

[0044] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0045] This invention constructs a trajectory input by organizing the problem text of maritime navigation missions, maritime situation images, and historical tool records in the order of decision steps. This allows the originally scattered task semantics, image evidence, and tool feedback to participate in subsequent decisions under the same trajectory structure, avoiding the problem of fragmented historical information caused by making a single judgment based on the current input. This improves the continuity of multi-source information organization and the traceability of intermediate evidence in the process of maritime auxiliary decision-making.

[0046] This invention unifies the encoding of textual representations, visual representations, and tool trajectory representations, and utilizes a multimodal temporal attention model to model the temporal dependencies between maritime task semantics, situational image evidence, and historical tool behavior. This enables the agent to combine the results of previous tool calls and feedback constraints when selecting tools or deciding to stop, reducing problems such as repeated calls, path redundancy, and inconsistencies between previous and subsequent judgments, thereby improving the stability of the multi-step tool collaborative decision-making process.

[0047] This invention constructs positive and negative trajectory sets during the training phase and builds a trajectory-level preference optimization objective based on trajectory-level normalized probability differences. This enables the decision-making strategy to no longer rely solely on single-step action fitting, but to learn path preference relationships in maritime auxiliary decision-making tasks at the scale of complete tool call trajectories. This suppresses semantic drift in multi-step inference and improves the consistency between tool selection results, stopping decisions, and final output. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0049] Figure 1 This is a flowchart of the method of the present invention;

[0050] Figure 2 This is a flowchart of the trajectory generation algorithm of the present invention;

[0051] Figure 3This is a schematic diagram of the multimodal temporal attention decision network structure of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below with reference to the accompanying drawings.

[0053] Throughout the accompanying drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. The described embodiments are only a part of the embodiments of this application, not all of them. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0054] To address two key issues in multimodal agent inference during multi-step processes: first, insufficient utilization of historical actions and tool feedback information, with cross-modal fusion often remaining at the static feature level and lacking explicit modeling of the dynamic dependencies in the inference chain; second, existing optimization processes are mostly based on single-step preference alignment, making it difficult to maintain policy consistency and stability at the trajectory scale, and prone to inference deviation, path redundancy, and getting trapped in local optima. This embodiment provides a maritime multimodal trajectory optimization agent tool decision-making method, applied to maritime navigation auxiliary decision-making scenarios. These scenarios include tasks such as ship navigation situation assessment, navigation rule query, collision risk warning, route auxiliary judgment, and port entry / exit auxiliary decision-making. The system receives question text that can be a navigation situation description, rule query statement, or risk warning request; maritime situation images can be electronic chart displays, radar situation images, shipborne camera footage, port monitoring images, or ship-shore collaborative situation images.

[0055] In this embodiment, the intelligent agent is configured with a set of candidate actions, which includes a maritime knowledge retrieval tool, a maritime image retrieval tool, an image description assistance tool, a direct response tool, and a stop action. The maritime knowledge retrieval tool is used to retrieve textual evidence from maritime rules, navigation knowledge, port management regulations, or historical case databases; the maritime image retrieval tool is used to retrieve similar images and their associated descriptions based on the input maritime situation image; the image description assistance tool is used to generate descriptions of the positional relationships or environmental information of targets, channels, and vessels in the situation image; the direct response tool is used to generate auxiliary decision-making results when existing evidence meets the task requirements; and the stop action is used to end the current multi-step tool invocation process.

[0056] Example 1

[0057] like Figure 1 As shown in the figure, the decision-making method for a maritime multimodal trajectory optimization intelligent agent tool provided in this embodiment includes the following steps:

[0058] S1: Acquire the problem text, maritime situation image, and historical tool records for the maritime navigation mission. Construct the problem text, maritime situation image, and historical tool records into a trajectory input according to the decision-making step sequence. The problem text represents the semantic requirements of the current maritime mission. For example, the problem text could be "Determine whether there is a risk of cross-encounter with the target vessel based on the current radar situation image," or "Determine whether to adjust the course based on the electronic chart and the vessel's current position." The maritime situation image provides visual evidence, including but not limited to images displayed on electronic charts, radar images, shipborne camera images, or port monitoring images. The historical tool records include the tool type, input parameters, output results, and feedback information of the tools invoked.

[0059] Further explanation: Before executing step S1, the following preparatory configurations are completed: An external tool set is constructed, including the text retrieval enhancement tool TextRAG, the image retrieval enhancement tool ImageRAG, the image description assistance tool ImageAssist, and the direct answer tool DirectAnswer. These external tools and the stop action STOP are unified into a candidate action set. A unique identifier and index mapping are established for the candidate action set for trajectory node recording and historical action encoding. The maximum number of iterations in the inference process is set to 4. Further, the TextRAG module constructs a text knowledge base and retrieval index based on Wiki5M; the ImageRAG module uses CLIP to extract visual embeddings and relies on FAISS to construct vector indexes to return similar images and their caption information; the ImageAssist module generates the caption corresponding to the input image and concatenates it with the question text to form enhanced prompts; the DirectAnswer module implements direct answers to the input text and image based on Qwen3-VL-8B-Instruct. For the training data construction stage, a preset tool traversal order T can be used to iterate through the candidate tools during trajectory generation and record the output and feedback at each step.

[0060] In one embodiment, step S1 is used to construct a trajectory input and generate a trajectory sequence, which consists of multiple trajectory nodes arranged in chronological order; each trajectory node is used to record key information of a tool call for use in subsequent multimodal coding and decision output.

[0061] A given sample set can be represented as: ;

[0062] in: Representative question text, Represents the corresponding image. This represents the answer label. For each sample, the system sequentially calls the tools in the toolbox in a predetermined order, which can be represented as:

[0063]

[0064] DirectAnswer generates answers directly using a large language model, TextRAG retrieves relevant text from the Wikipedia knowledge base, ImageAssist generates descriptive captions for the input image, and ImageRAG supplements caption information for similar images through image search. After each call, the correctness of the answer is checked; if correct, the iteration stops; if incorrect, the current tool and its result are added to the history, and the process continues to try the next tool. Through this process, each sample generates a tool call trajectory, and the multi-step reasoning sequence for each sample can be represented as:

[0065]

[0066] in: For tool actions, For results feedback, The definition of historical context state can be represented as:

[0067]

[0068] During trajectory generation, the system sequentially tries each tool, which can be represented as follows:

[0069]

[0070] like If the output matches the standard answer, the trajectory terminates and a Stop action is added; otherwise, other tools are tried to proceed to the next step. This process yields a complete tool call trajectory tree, with each decision node containing... Five-tuple information.

[0071] To achieve preference contrastive learning, we unfold and combine the multi-step inference trajectories of each question sample into a trajectory set. For the question... and its corresponding image The system generates several success and failure trajectories during multiple calls, which can be represented as follows:

[0072]

[0073]

[0074] This term reflects the model's average confidence across the entire decision sequence. Subsequently, the training sample set, constructed using the trajectory set as the basic unit, can be expressed as:

[0075]

[0076] Each sample set contains both positive and negative trajectories, used for preference optimization at the trajectory scale. During training, the model is encouraged to increase the average normalized probability of the positive trajectory set and decrease the corresponding values ​​of the negative trajectory set, thereby learning implicit multi-step inference preferences.

[0077] After completing step S1, the system has obtained the trajectory input state for the current decision step. The question text q, the input image p, and the historical state consisting of historical tool calls and feedback are included. The information has been organized into structured trajectory information that can be used for modeling. Based on this state, proceed to step S2, where the information of each modality is encoded and aligned to a unified representation space, providing an input basis for subsequent temporal attention fusion and action decision-making.

[0078] In one implementation, the trajectory input is used not only for the current decision during the inference phase but also for trajectory sample generation during the training phase. For example... Figure 2 As shown, for samples containing question text, image information, and answer tags, the system sequentially calls candidate tools according to a preset tool sequence. After each tool call, it obtains the corresponding output result and evaluates the output result through a judgment function. When the tool output meets the task requirements, the current tool call node and stop action are written into the trajectory; when the tool output does not meet the task requirements, the tool call result is written into the history record, and subsequent tools are called. This forms a tool call trajectory that includes question text, image information, historical tool records, tool actions, tool output results, and stop actions.

[0079] Furthermore, each trajectory is saved according to the tool call order, and the historical records in the trajectory are used to represent the tool call behaviors that have occurred up to the current decision step. Through the above trajectory generation process, the system can obtain trajectory samples corresponding to different tool call paths under the same maritime navigation mission, providing a data foundation for the subsequent construction of positive trajectory sets, negative trajectory sets, and trajectory-level preference optimization.

[0080] S2: Encode the problem text, maritime situation image and historical tool record respectively to obtain text representation, visual representation and tool trajectory representation, and map the text representation, visual representation and tool trajectory representation to a unified representation space;

[0081] Specifically, this invention employs corresponding encoding functions for the three modes and maps them to a vector space of uniform dimension. The encoding method can be expressed as follows:

[0082]

[0083] Specifically: The text representation q uses the semantic encoding model BGE to extract the semantic embeddings of the question text and aligns them to a unified dimension through linear mapping, enabling it to be fused and modeled in the same space as image features and historical features. This representation provides the current task objective and constraints, serving as the primary semantic basis for subsequent tool selection. The visual representation p uses the visual encoding model CLIP to extract the visual embeddings of the input image and projects them to a unified dimension through a linear layer. This characterizes key entities, attributes, and scene cues in the image, thus forming complementary evidence with the text question. The historical trajectory representation records the tool type, input, output, and feedback information of previous tool calls, providing a structured context that explicitly distinguishes reasoning branches, enabling the model to identify: which tools have been tried, what intermediate evidence has been obtained, which directions have been rejected, and what information is still missing. Historical encoding uses a lightweight one-hot encoding with feedback to construct historical action and feedback representations, avoiding the cumulative error and drift risk associated with complex historical sequence modeling. For each historical action-feedback pair... The actions Corresponding to four types of tools and STOP, a 5-dimensional one-hot representation is used; feedback The historical representation is constructed by concatenating scalars and actions and then aggregating them.

[0084]

[0085] Feedback during the training phase The value {0,1} can be used to indicate whether the tool output is consistent with the ground truth; during the inference stage, it can be fixed to a constant of 0.5 to avoid answer leakage, while maintaining the consistency of the historical input structure, which is conducive to model generalization.

[0086] After completing step S2, three types of modal representations (q, p, h) that can be used for temporal attention fusion are obtained. The three types of representations are mapped to a unified representation space, which is then used for subsequent multimodal temporal fusion in step S3.

[0087] After encoding the problem text, maritime situation images, and historical tool records, the system enters the multimodal temporal attention fusion and decision-making stage. For example... Figure 3As shown, the problem text is input into the text encoder to obtain a text representation, the maritime situation image is input into the image encoder to obtain a visual representation, and the historical tool record is input into the history encoder to obtain a tool trajectory representation. The text representation, visual representation, and tool trajectory representation are fed into the multi-head attention module as multimodal inputs. The relationship between task semantics, image evidence, and historical tool behavior is modeled through attention weights to obtain a fused feature representation. Subsequently, the fused feature representation is input into the linear layer to output the decision result of the current decision step.

[0088] based on Figure 3 The network structure shown indicates that the multimodal temporal attention model not only utilizes the current problem text and image information, but also incorporates the historical records formed by previous tool calls into the same decision network, so that the tool selection results and stopping decisions can be constrained by the historical tool output and feedback information.

[0089] S3: Input textual representation, visual representation, and tool trajectory representation into a multimodal temporal attention model to model the temporal dependencies between maritime mission semantics, situational image evidence, and historical tool behavior, and output a fused feature representation;

[0090] In this embodiment, the multimodal temporal attention model includes a modality identification embedding layer, a temporal position embedding layer, and an attention fusion layer. The modality identification embedding layer is used to distinguish the sources of text information, image information, and tool trajectory information; the temporal position embedding layer is used to represent the sequential position of tool trajectory nodes in the multi-step decision-making process; and the attention fusion layer is used to calculate the association weights between different modal information and tool behaviors in different decision steps.

[0091] The text representation, visual representation, and historical trajectory representation obtained in step S2 are stacked into a fixed-length multimodal sequence input, enabling the model to explicitly model the dynamic dependencies among text, image, history, tool behavior, and other elements within the same temporal attention framework. The stacking method can be represented as follows:

[0092]

[0093] Where d represents the unified representation dimension. To enable the model to distinguish different information sources during the fusion process, this invention adds a learnable modality identifier embedding to each modality, thereby avoiding the confusion of different modalities as the same type of token in the attention process. Subsequently, a multi-layer Transformer encoder is used to perform context modeling on sequence X. For any layer, its multi-head self-attention and feedforward update process can be represented as:

[0094]

[0095]

[0096]

[0097] in: , , Here, represents the query, key, and value matrix, respectively; LN denotes layer normalization; and FFN represents the feedforward neural network. Through this multi-layer stacking, the model can simultaneously encode in the fused representation: the semantics of the problem, image evidence, and the constraints of historical tool behavior on the current decision, thereby obtaining a more stable and interpretable cross-modal context representation. To obtain the single-vector fused features for action decision-making, this invention performs average pooling aggregation on the final layer output sequence to obtain the fused semantic vector c, the calculation of which can be expressed as:

[0098]

[0099] in It integrates features from three modalities: current problem, visual information, and historical decision-making. (Fusing vector) It is a multimodal temporal attention fusion feature representation, which simultaneously includes the semantics of the current maritime mission, situational image evidence, and the constraint information of historical tool behavior on the current decision step, for subsequent tool selection and stopping judgment.

[0100] S4: Based on the fusion feature representation, perform action scoring on the candidate action set including maritime knowledge retrieval tools, maritime image retrieval tools, image description assistance tools, direct answering tools, and stop actions, and determine the tool selection result or stop decision for the current decision step;

[0101] In this embodiment, the decision head network can adopt a multilayer perceptron structure, including a linear mapping layer, an activation function layer, a regularization layer, and an output layer. The output dimension of the output layer is consistent with the number of candidate actions in the candidate action set. Each output value corresponds to the action score of a candidate action.

[0102] Based on the fusion vector c obtained in step S3, a lightweight decision head network predicts the candidate action scores for the current decision step, and determines the tool selection result and the stopping decision accordingly. This decision head employs a two-layer fully connected structure, and its output is expressed as follows:

[0103]

[0104] in The GELU activation function is used, and each dimension of the output vector o corresponds one-to-one with the candidate action set, which includes at least four types of tool actions and stop actions.

[0105] Furthermore, to adjust the sharpness of the motion distribution and improve training stability, this invention introduces a learnable temperature parameter. Scaling the action scores yields the conditional probability distribution of the actions, represented as follows:

[0106]

[0107] Based on the above probability distribution, the current action is determined. When the output is STOP, a stop decision is triggered. When the output is a tool action, corresponding tool invocation information is generated. The tool invocation information includes at least a tool identifier and tool input content. The tool input content can consist of the current problem text, maritime situation image, historical tool output results, or fused intermediate evidence.

[0108] For example, if the current question text involves the applicable conditions of navigation rules, and the historical tool records do not yet contain rule retrieval results, the system can select the maritime knowledge retrieval tool; if the current task requires understanding the positional relationships of ships in radar images or electronic charts, the system can select the image description assistance tool or the maritime image retrieval tool; if the historical tool records already contain sufficient rule evidence and image evidence, the system can select the direct answer tool or stop the action.

[0109] S5: When the current decision step does not trigger a stop decision, execute the selected tool and obtain the tool output results and feedback information. Write the tool call information to the historical tool record to update the trajectory input, and return to execute S2 to S4 until a stop decision is triggered or the preset maximum number of iterations is reached.

[0110] If the decision to stop is negative, execute the tool selected in step S4, obtain the tool output and feedback information, and add the new trajectory node formed by this tool call back to the historical tool record, thereby updating the trajectory input and entering the next round of decision iteration.

[0111] Specifically, at each decision step, this invention calls the corresponding TextRAG, ImageRAG, ImageAssist, and DirectAnswer tool modules based on the tool type to obtain the tool output results; subsequently, feedback information is obtained according to a preset feedback generation mechanism. This feedback information can be used for preference construction and historical trajectory encoding during the training phase, serving as a historical state. The components are involved in subsequent temporal attention modeling; on the other hand, they are used to describe the effectiveness of this tool call in solving the problem, thereby supporting the construction of the target for subsequent trajectory-level preference optimization.

[0112] After this call is written back, the history tool record is updated to include the newly added node. This, in turn, forms the input state for the next decision step. The system repeats steps S2 to S4 based on the updated trajectory input until the stop decision is met or the preset maximum number of iterations (4) is reached.

[0113] After completing step S5, the final answer is output during the inference phase; during the training phase, step S6 is performed to align the trajectory-level preference samples to update the decision policy parameters.

[0114] After completing steps S1 to S5, the model is able to make decisions based on the state at each step. Output action probability distribution This forms a complete multi-step tool call trajectory. After the tool is executed, the system encapsulates the tool type, tool input parameters, tool output results, feedback information, and decision step number of this tool call into a new tool trajectory node, and appends this new tool trajectory node to the end of the historical tool record.

[0115] S6: During the training phase, based on... Figure 2 The trajectory generation process shown yields multiple tool call trajectories. Based on the matching relationship between the trajectory output and the task objective, the number of tool call steps, repeated calls, and feedback consistency, the trajectories are divided into a positive trajectory set and a negative trajectory set. The positive trajectory set and negative trajectory set are then constructed, the trajectory-level normalized probability difference between the positive trajectory set and the negative trajectory set is calculated, and the trajectory-level preference optimization is used to optimize the target update agent's tool decision-making strategy.

[0116] Specifically, a set of positive trajectories is constructed for each training sample. With negative trajectory set Each trajectory It includes multi-step state-action pairs, where the state is composed of question text, image information, and historical tool records, and the action corresponds to the candidate tool action or stop action. Considering that different trajectory lengths may vary, to ensure comparability, this invention uses the trajectory normalized log probability as the trajectory score. The log probabilities of each step on the trajectory are accumulated and then averaged over the trajectory length, defined as:

[0117]

[0118] Based on this, a reference model is introduced. As a benchmark, the optimization becomes more stable. For each trajectory, the difference between the current policy and the reference policy is calculated simultaneously, and a soft aggregation comparison of the positive and negative trajectory sets is performed at the group level to obtain the relative advantage of the positive and negative sets. Defined as:

[0119]

[0120] in This represents the score corresponding to the reference policy on the trajectory. The model maximizes... To enhance the advantage of the positive trajectory set relative to the negative trajectory set, a trajectory-level preference optimization loss is constructed to make the model tend to enhance the overall advantage of the positive trajectory set relative to the negative trajectory set during training. The loss function is:

[0121]

[0122] in For the Sigmoid function, This is a balancing coefficient used to control the contrast intensity. It is used to adjust the contrast intensity. Since this objective directly aligns preference relationships on the scale of the entire trajectory, it can suppress policy drift and long-range inconsistency problems caused by single-step alignment alone, and reduce path redundancy and semantic drift risks. To prevent the policy from collapsing into a few action patterns during training, this invention further introduces an entropy regularization term to encourage policy diversity, defined as:

[0123]

[0124] in Here is the regularization coefficient. Finally, the preference optimization objective and entropy regularization are combined to form the overall optimization objective:

[0125]

[0126] By minimizing the total loss L, the network parameters of the decision policy are updated, enabling the model to learn implicit multi-step preference relationships at the end-to-end trajectory level. This not only improves the rationality of the first-step decision but also strengthens the stability of cross-steps and the consistency of early stopping judgments, thereby achieving more robust, coherent, and controllable decision-making behavior in multi-tool collaborative multimodal reasoning tasks.

[0127] Example 2

[0128] Based on Example 1, this example provides a specific application where the user inputs the question text "Based on the current electronic chart and radar situation, determine whether this vessel needs to take evasive action", and simultaneously inputs a maritime situation image containing information on the vessel, the target vessel, the channel boundary, and their relative bearings.

[0129] First, the question text and maritime situation image are used as the initial trajectory input. Since the initial historical tool record is empty, the system primarily relies on the question text and image information for judgment in the first decision step. After fusion using a multimodal temporal attention model, the action probability output by the decision head network shows that the image description auxiliary tool scores higher; therefore, the system invokes the image description auxiliary tool. This tool outputs image description information such as the target vessel being located to the right front of the current vessel and the convergence trend of both tracks, and writes this output as a new tool trajectory node into the historical tool record.

[0130] In the second decision-making step, the system re-encodes the question text, maritime situation image, and newly added historical tool records. Since the current task also involves the judgment of avoidance rules, the system selects a maritime knowledge retrieval tool based on the fused feature representation, retrieves relevant rule clauses from the maritime navigation rule base related to cross encounters, yielding vessels, and straight-ahead vessels, and writes the retrieval results into the historical tool records.

[0131] In the third decision-making step, the system performs a fusion judgment based on the question text, image description results, and rule retrieval results. If the fused feature representation indicates that existing evidence can support the auxiliary decision result, the system can choose to directly generate auxiliary decision suggestions using the direct response tool, or directly output a stop action and end the tool call process. The final auxiliary decision result output by the system may include the current situation assessment, the relevant rule basis, the suggested course or speed adjustment method, and target information that needs to be continuously observed.

[0132] In the above process, each tool selection step of the system is related to the output and feedback information of the preceding tools, and the results of each tool call are written into the historical tool record to form a traceable trajectory. Compared with the method of making single-step judgments based solely on the current input, this embodiment enables the maritime auxiliary decision-making process to have a continuous chain of intermediate evidence, which facilitates subsequent review, verification, and engineering deployment.

[0133] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for maritime multi-modal trajectory optimization agent tool decision making, characterized in that, Includes the following steps: S1: Obtain the problem text, maritime situation image and historical tool records of the maritime navigation mission, and construct the problem text, maritime situation image and historical tool records into a trajectory input according to the decision step sequence. The historical tool records include the tool type, input parameters, output results and feedback information of the tools that have been called. S2: Encode the problem text, maritime situation image and historical tool record respectively to obtain text representation, visual representation and tool trajectory representation, and map the text representation, visual representation and tool trajectory representation to a unified representation space; S3: Input textual representation, visual representation, and tool trajectory representation into a multimodal temporal attention model to model the temporal dependencies between maritime mission semantics, situational image evidence, and historical tool behavior, and output a fused feature representation; S4: Based on the fusion feature representation, perform action scoring on the candidate action set including maritime knowledge retrieval tools, maritime image retrieval tools, image description assistance tools, direct answering tools, and stop actions, and determine the tool selection result or stop decision for the current decision step; S5: When the current decision step does not trigger a stop decision, execute the selected tool and obtain the tool output results and feedback information. Write the tool call information to the historical tool record to update the trajectory input, and return to execute S2 to S4 until a stop decision is triggered or the preset maximum number of iterations is reached. S6: During the training phase, positive trajectory sets and negative trajectory sets are constructed based on maritime navigation missions. The trajectory-level normalized probability difference between the positive trajectory set and the negative trajectory set is calculated, and the trajectory-level preference optimization is used to optimize the decision-making strategy of the target update agent tool.

2. The method of claim 1, wherein, The trajectory input includes: The problem text of maritime navigation missions is used as the semantic node of the mission, and the maritime situation image is used as the visual evidence node. Each tool call encapsulates the tool type, input parameters, output results, and feedback information into a tool trajectory node; Connect the tool trajectory nodes according to the order in which the tools are invoked, and organize the task semantic nodes and visual evidence nodes together with the tool trajectory nodes into a trajectory sequence; At each decision step, the current trajectory sequence is read to form the trajectory input for use in multimodal coding.

3. The method of claim 1, wherein, The maritime situation images include at least one of electronic chart display images, radar situation images, shipborne camera images, port monitoring images, or ship-shore collaborative situation images; the question texts include at least one of navigation rule queries, collision risk warnings, navigation situation descriptions, route planning requirements, or maritime safety Q&A requirements.

4. The method of claim 1, wherein, Encoding historical tool records includes: Create action identifier vectors for the invoked tools based on the candidate action set; Convert tool input parameters, tool output results, and feedback information into a tool invocation status representation; By concatenating or mapping the action identifier vector with the tool call state representation, a single-step tool trajectory representation can be obtained. Multiple single-step tool trajectory representations are aggregated according to the calling order to obtain tool trajectory representations that represent historical tool behavior.

5. The decision-making method for a maritime multimodal trajectory optimization intelligent agent tool according to claim 1, characterized in that, The multimodal temporal attention model includes a modality identifier embedding layer, a temporal location embedding layer, and an attention fusion layer; where: The modality identification embedding layer is used to distinguish the information sources of text representation, visual representation and tool trajectory representation. The temporal position embedding layer is used to identify the decision step positions recorded by the historical tool in the trajectory sequence. The attention fusion layer is used to calculate the correlation weights between different modal information and different decision step tool behaviors, and generate fused feature representations based on the correlation weights.

6. The decision-making method for a maritime multimodal trajectory optimization intelligent agent tool according to claim 1, characterized in that, The tools for selecting an outcome or stopping the decision in the current decision step include: The fused feature representation is input into the decision head network to obtain the action score of each candidate action in the candidate action set. The action score is scaled using a learnable temperature parameter to obtain the action probability distribution; Select the candidate action corresponding to the current decision step based on the action probability distribution; When the selected candidate action is a stop action, a stop decision is generated; when the selected candidate action is a tool action, tool call information containing the tool identifier and tool input content is generated.

7. The decision-making method for a maritime multimodal trajectory optimization intelligent agent tool according to claim 1, characterized in that, Writing to the historical tool record includes: encapsulating the tool type, tool input parameters, tool output results, feedback information, and corresponding decision step number of this tool call into a new tool trajectory node, and appending the new tool trajectory node to the end of the current historical tool record, so that the trajectory input of the next decision step simultaneously includes the original maritime mission information and the executed tool call information.

8. The decision-making method for a maritime multimodal trajectory optimization intelligent agent tool according to claim 1, characterized in that, Constructing the sets of positive and negative trajectories includes: Multiple tool call trajectories are generated based on the same maritime navigation mission; Based on the matching relationship between trajectory output results and task objectives, tool call steps, repeated calls, and consistency of intermediate feedback, multiple tool call trajectories are classified by quality. Tool call trajectories that meet the trajectory quality conditions will be assigned to the positive trajectory set, while tool call trajectories that do not meet the trajectory quality conditions will be assigned to the negative trajectory set. The positive trajectory set and negative trajectory set corresponding to the same maritime navigation mission constitute trajectory-level preference sample pairs.

9. The decision-making method for a maritime multimodal trajectory optimization intelligent agent tool according to claim 1, characterized in that, The construction of the trajectory-level preference optimization objective includes: Calculate the log probability of each decision step action in each tool call trajectory; The logarithmic probabilities of each decision step are normalized according to the trajectory length to obtain the trajectory-level normalized probabilities. Introduce a reference policy and calculate the probabilistic advantage of the current decision policy over the reference policy on the positive trajectory set and the negative trajectory set, respectively. A trajectory-level preference optimization loss function is constructed based on the probability advantage difference between the positive trajectory set and the negative trajectory set, and the agent tool decision-making strategy is updated based on the trajectory-level preference optimization loss function.

10. The decision-making method for a maritime multimodal trajectory optimization intelligent agent tool according to claim 9, characterized in that, The trajectory-level preference optimization loss function also includes an entropy regularization term, which is constructed based on the action probability distribution of the current decision policy on the candidate action set. This term is used to constrain the tool selection result from becoming overly concentrated on a single tool action during training. The agent's tool decision policy updates its parameters through the joint objective of the trajectory-level preference optimization loss function and the entropy regularization term.

Images