An unmanned ship environment perception decision-making method and system based on brain-like memory

Abstract

The application discloses an unmanned ship environment perception decision-making method and system based on brain-like memory, and relates to the technical field of unmanned ship decision-making. The method comprises the following steps: acquiring an environment image in front of the unmanned ship; inputting the environment image into an unmanned ship environment perception decision-making model to output an action instruction; the unmanned ship environment perception decision-making model comprises an image feature extractor, a BERT model, a full connection layer, a short-time scene memory module and a long-time memory module connected in sequence; the BERT model extracts image feature encoding containing text features from the image features; the short-time scene memory module is used for presetting multiple questions, and a large language model is used to answer each question in a set order to obtain multiple answers; the long-time memory module is used for outputting an action instruction based on the multiple answers by using long-time memory and context learning of the large language model; and the large language model is a large language model fine-tuned based on reinforcement learning. The application improves the accuracy of action decision-making.

Description

Technical Field

[0001] This invention relates to the field of unmanned surface vessel (USV) decision-making technology, and in particular to a method and system for USV environmental perception and decision-making based on brain-like memory. Background Technology

[0002] In the field of intelligent decision-making for unmanned surface vessels (USVs), existing methods often only utilize reinforcement learning, taking the USV's external environmental information as input and designing a reward model to obtain decisions that maximize reward scores. The problem with this type of approach is that accurate acquisition of external environmental information is difficult in complex sea conditions. Even after acquiring all external environmental information, the trained reinforcement learning model still struggles to make the most accurate decisions based on complex sea conditions. Summary of the Invention

[0003] The purpose of this invention is to provide an environmental perception and decision-making method and system for unmanned surface vessels based on brain-like memory, which improves the accuracy of action decisions.

[0004] To achieve the above objectives, the present invention provides the following solution:

[0005] A brain-like memory-based environmental perception and decision-making method for unmanned surface vessels includes:

[0006] Acquire images of the environment in front of the unmanned surface vessel;

[0007] The environmental image is input into the unmanned surface vessel's environmental perception and decision-making model, and action commands are output. The unmanned surface vessel's environmental perception and decision-making model includes an image feature extractor, a BERT model, a fully connected layer, a short-term scene memory module, and a long-term memory module connected in sequence.

[0008] The unmanned surface vessel is controlled by the aforementioned action commands;

[0009] The image feature extractor is used to extract image features from the environmental image; the BERT model is used to extract image feature encoding containing text features from the image features; the fully connected layer is used to map the image feature encoding to an image encoding suitable for recognition by a large language model; the short-term scene memory module is used to preset multiple questions, and use the short-term scene memory of the large language model to answer each question in a set order to obtain multiple answers; the long-term memory module is used to use the long-term memory and context learning of the large language model to output action instructions based on multiple answers; the large language model is a large language model fine-tuned based on reinforcement learning.

[0010] This invention also discloses an unmanned surface vessel environmental perception and decision-making system based on brain-like memory, comprising:

[0011] The environmental image acquisition module is used to acquire images of the environment in front of the unmanned surface vessel.

[0012] The unmanned surface vessel (USV) environmental perception and decision-making model decision module is used to input the environmental image into the USV environmental perception and decision-making model and output action commands; the USV environmental perception and decision-making model includes an image feature extractor, a BERT model, a fully connected layer, a short-term scene memory module and a long-term memory module connected in sequence.

[0013] The control module is used to control the movement of the unmanned surface vessel using the action commands;

[0014] The image feature extractor is used to extract image features from the environmental image; the BERT model is used to extract image feature encoding containing text features from the image features; the fully connected layer is used to map the image feature encoding to an image encoding suitable for recognition by a large language model; the short-term scene memory module is used to preset multiple questions, and use the short-term scene memory of the large language model to answer each question in a set order to obtain multiple answers; the long-term memory module is used to use the long-term memory and context learning of the large language model to output action instructions based on multiple answers; the large language model is a large language model fine-tuned based on reinforcement learning.

[0015] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0016] This invention combines large language models with reinforcement learning, utilizing the powerful multimodal representation and contextual reasoning capabilities of large language models to simulate the collaborative decision-making methods of perceptual memory, scene memory, long-term memory, and working memory in the human brain. This solves the problems of difficult environmental information perception and real-time accurate decision-making in the intelligent decision-making of unmanned surface vessels. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic diagram of an environmental perception and decision-making method for unmanned surface vessels based on brain-like memory is provided in an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram illustrating the training principle of the unmanned surface vessel environmental perception and decision-making model provided in an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of an unmanned surface vessel environmental perception and decision-making system based on brain-like memory, provided as an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The purpose of this invention is to provide an environmental perception and decision-making method and system for unmanned surface vessels based on brain-like memory, which improves the accuracy of action decisions.

[0023] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] Example 1

[0025] like Figure 1 As shown in the figure, this embodiment provides an environmental perception and decision-making method for unmanned surface vessels based on brain-like memory. The method specifically includes the following steps.

[0026] Step 101: Obtain an image of the environment in front of the unmanned surface vessel.

[0027] Step 102: Input the environmental image into the unmanned surface vessel's environmental perception and decision-making model, and output action commands; the unmanned surface vessel's environmental perception and decision-making model includes an image feature extractor, a BERT model, a fully connected layer, a short-term scene memory module, and a long-term memory module connected in sequence.

[0028] The operational commands include changes to the unmanned surface vessel's (USV) turning angle, speed, and diving depth, among others.

[0029] Step 103: Control the unmanned surface vessel's movement using the action command.

[0030] The image feature extractor is used to extract image features from the environmental image; the BERT model is used to extract image feature encoding containing text features from the image features; the fully connected layer is used to map the image feature encoding to an image encoding suitable for recognition by a large language model; the short-term scene memory module is used to preset multiple questions, and use the short-term scene memory of the large language model to answer each question in a set order to obtain multiple answers; the long-term memory module is used to use the long-term memory and context learning of the large language model to output action instructions based on multiple answers; the large language model is a large language model fine-tuned based on reinforcement learning.

[0031] Large language models include GPT-4 and other large language models. The BERT model mentioned is a trained BERT model.

[0032] The training principle of the unmanned surface vessel's environmental perception and decision-making model is as follows: Figure 2 As shown, firstly, a large number of images related to the ocean and unmanned surface vessels (USVs) and their corresponding descriptive texts were collected to create an image-text pair dataset. Then, low-quality images were filtered out, and the texts were manually reviewed to correct errors, including removing duplicate words and incoherent sentences. Finally, the manually selected image-text pairs were used as training data for the visual encoder (USV environment dataset).

[0033] The visual encoder comprises an image feature extractor and a BERT model. In the visual encoder, a pre-trained VisionTransformer is used as the image feature extractor to extract semantic features from the image, and a pre-trained BERT (Bidirectional Encoder Representations from Transformers) model is used as the text feature extractor to extract semantic features from the text. Both the pre-trained VisionTransformer and the pre-trained BERT model are then frozen.

[0034] In the pre-trained BERT model, a cross-attention module is added between the self-attention module and the feedforward neural network of each Transformer block.

[0035] The training process of the BERT model includes:

[0036] Collect an unmanned surface vessel (USV) environment dataset; each sample in the USV environment dataset includes an USV environment image and corresponding text description information.

[0037] The text and learnable encoding are fused as initial input, and a cross-attention module is added between the self-attention module and the feedforward neural network of the BERT model. This cross-attention module fuses image and text features. Image features extracted by the pre-trained VisionTransformer are input into the cross-attention module, then passed through the feedforward neural network to obtain an encoding that simultaneously contains text and image features, used to calculate the subsequent image-text matching loss. Simultaneously, the text is processed separately through a BERT pre-trained self-attention module and a feedforward neural network to obtain text features, which are combined with the image feature encoding obtained above to calculate the image-text contrast loss. Additionally, a mask is added to the text self-attention module, and the image encoding and the masked text are used to predict the masked content, resulting in the image-text generation loss. The learnable encoding is trained using these three losses. Then, an additional fully connected layer is trained at the model's output using the image-text generation loss. This fully connected layer maps the text features extracted from the image to the text encoding that the large language model can recognize. This allows the pre-trained large language model to recognize image features with almost no modification, and to perform text generation and inference based on these features.

[0038] Each sample data point includes the following during training:

[0039] The unmanned surface vessel's environmental images are input into a pre-trained image feature extractor, which outputs sample image features.

[0040] The text description information corresponding to the unmanned surface vessel's environmental image is input into the BERT model, and the sample image features are input into the cross-attention module of each Transformer block in the BERT model.

[0041] The features output by the cross-attention module are processed by a feedforward neural network to obtain the feature encoding of the first sample image.

[0042] The image-text matching loss is determined based on the feature encoding of the first sample image.

[0043] The text description information corresponding to the unmanned surface vessel's environmental image is input into the pre-trained network, which outputs the feature encoding of the second sample image. The pre-trained network includes a self-attention module and a feedforward neural network connected in sequence.

[0044] The image-text contrast loss is determined based on the feature encoding of the first sample image and the feature encoding of the second sample image.

[0045] Add a mask to the self-attention module of each Transformer block in the BERT model.

[0046] The text description information corresponding to the unmanned surface vessel's environmental image is input into the BERT model after adding a mask. The features of the sample image are input into the cross-attention module of each Transformer block in the BERT model. The features output by the cross-attention module are processed by a feedforward neural network to obtain the third sample image feature encoding.

[0047] The image text generation loss is determined based on the feature encoding of the third sample image and the label data corresponding to the feature encoding of the third sample image.

[0048] The mixture loss during the training process of the BERT model is represented as:

[0049]

[0050] Where ω1 is the image-text contrast loss. The weight hyperparameters, ω2 is the image-text matching loss. The weight hyperparameters, ω3 is the image text generation loss. The weight hyperparameters, It is a mixed loss value.

[0051] In the BERT model learning process, a hybrid loss function is used. The goal is to train a feature extractor from image-text pairs to extract image features and feed them into a large language model to complete multimodal information transmission.

[0052] Image-text contrast loss The specific calculation formula is as follows:

[0053]

[0054] Where s is the similarity calculation function, which will be implemented using cosine similarity in this invention; τ is a learnable parameter; and H is the cross-entropy loss function. This indicates that the image-text similarity is calculated using Softmax normalization for each image in a batch. This represents the text-image similarity calculated using Softmax normalization for each sentence in a batch, y. i2t (I) represents the one-hot similarity of the images calculated using label information (here, the label refers to whether the image and text are in the same image-text pair). t2i (T) represents the one-hot similarity of the text calculated using the label information. The final image-text contrast loss H is defined as the cross-entropy between the predicted data and the label data. I represents a single image, and T represents a single piece of text. m This indicates agreement with the m-th rising image in the batch, T. mThis refers to the m-th text segment in the same batch, where M represents the batch size. Let D' represent a batch of image-text pairs extracted from the entire dataset, where D' represents the entire dataset, i.e., the unmanned surface vessel (USV) environment dataset. Image-text contrast loss. Its function is to make positive sample pairs closer in the feature space and negative sample pairs farther apart through contrastive learning, thereby aligning image features and text features and maximizing their mutual information.

[0055] Image-text matching loss The specific calculation formula is as follows:

[0056]

[0057] Where, p itm It uses the output of image and text multimodal processing and passes it through the Softmax function to obtain binary classification predictions, determining whether the image and text belong to positive or negative samples. itm Two-dimensional one-hot vectors are generated using label information, and H is the cross-entropy loss function. Image-text matching loss. Its function is to ensure that the model can correctly identify positive and negative sample pairs by judging whether the image and text match, thereby aligning the features of the image and text.

[0058] Image text generation loss The specific calculation formula is as follows:

[0059]

[0060] in, p represents the masked text. msk It uses images and masked text to make predictions about masked content, y msk It uses tags to generate one-hot encodings representing the masked content, and H is the cross-entropy loss function. Image text generation loss. Its main function is to allow the model to complete the information of the masked part based on the image and the masked text, so as to ensure that the model can obtain the correct image features and generate the corresponding text representation.

[0061] The BERT model is optimized based on image-text matching loss, image-text contrast loss, and image-text generation loss.

[0062] Collect questions that characterize the state of the unmanned surface vessel (USV) as expert knowledge. Then, process the real-time collected USV state images through a trained BERT model to obtain corresponding image feature encodings.

[0063] Leveraging the long-term memory and contextual learning of the large language model, action instructions are output based on multiple answers, specifically including:

[0064] Based on the large language model, long-term memory and contextual learning are used to output a set of instructions consisting of multiple instructions based on multiple answers, and action instructions are output according to the set of instructions.

[0065] The large language model is fine-tuned using a reinforcement learning model.

[0066] The process of fine-tuning the large language model includes:

[0067] Construct an instruction training set; the sample data in the instruction training set includes input data and label data, the input data is the sample instruction set, and the label data is the sorting of each instruction in the sample instruction set, with the sorting order being from high to low scores.

[0068] The reward model is trained by taking the sample instruction set as input and the sorting of the sample instruction set as output, and the trained reward model is obtained.

[0069] The instruction set output by the large language model is input into the trained reward model, and the first-ranked instruction is fed back to the large language model as an action instruction to fine-tune the large language model.

[0070] The formula for the attention layer in a large language model to achieve long-term memory and contextual learning is as follows:

[0071]

[0072] Among them, W V and W K Both are transformation matrices with dimensions d'×d, where d and d' are constants. X' is the token vector representation of the example part of the input, while X represents the vector representation of all tokens after the example part and before the last word in the input. [X'; X] indicates matrix concatenation, V represents the query vector, K represents the key vector, and q represents the query vector. This represents the attention layer that plays a role in context learning. The formula described above details the computational steps of the attention mechanism during a forward propagation process. Comparing this with the formula below shows that the attention mechanism plays a role in context learning during forward propagation.

[0073] The specific formula derivation for large language models to achieve long-term memory and contextual learning is as follows:

[0074]

[0075] Among them, W was obtained by simplifying the forward propagation process of the large language model. ZSL (zsl represents Zero-shot Learning) and ΔW ICL(icl stands for In-Context Learning), W ZSL ΔW represents the sample learning weights. ICL x' represents the context-learned weights, Linear Attn represents the linear attention layer, and x' represents the linear attention layer. i W represents the input to the current attention module, where i represents the input order of the attention module. The simplification process involves using a fully connected layer and an attention transformation mechanism. V X' can be seen as the output gradient calculated in the previous fully connected circuit, W K X' can be seen as the input to the previous fully connected computation, and q can be seen as the current input. This formula specifically describes how the attention layer implicitly completes context learning during a forward propagation of a large language model.

[0076] The reward model is trained using human-annotated ordering of instructions in different scenarios as training data. It simulates human scoring of each incoming instruction with reference to the current state of the unmanned vessel. The goal is to give each instruction the most reasonable score possible without changing the order of manually annotated instructions.

[0077] The loss function for training the reward model is expressed as:

[0078]

[0079] Where, r θ () represents the reward model, and x represents the question and image input to the large language model. Let D represent a picture and a question (or multiple questions) in the instruction training set, and the first and second instructions corresponding to the picture and the question. Let D represent the instruction training set, and y represent the instruction training set. w Indicates the first instruction, y l Let K represent the second instruction, K represent the number of instructions in the sample instruction set, σ represent the sigmoid activation function, loss(θ) represent the loss value when training the reward model, and θ represent all parameters of the reward model. This represents the number of combinations of any two data points chosen from K data points, i.e.

[0080] The objective function for fine-tuning a large language model is expressed as:

[0081]

[0082] Where objective(φ) represents the objective function value. It is a reinforcement learning model, r θ () represents the reward model, π LLMIt is the initial, untuned large language model, E (x,y) This represents a reinforcement learning training set consisting of an image and a question, along with the action instructions output by the large language model corresponding to the image and the question. This represents the reinforcement learning training set. Let E represent the current reinforcement learning model, β represent the hyperparameter controlling the offset between the current trained reinforcement learning model and the initial reinforcement learning model, x represent the image and question input to the large language model, y represent the action command output by the large language model, γ represent the parameter controlling the weight of the large language model on the reinforcement learning training set and the original pre-training data, and E... x D represents reinforcement learning training focusing on a picture and a question. pretrain This represents the pre-training data used when the original large language model is pre-trained.

[0083] Each sample in the reinforcement learning training set and the pre-training data during pre-training includes an image and a question, as well as the action instructions output by the large language model corresponding to the image and question.

[0084] The image feature extractor is a trained visual converter.

[0085] Example 2

[0086] like Figure 3 As shown, this embodiment provides an unmanned surface vessel (USV) environmental perception and decision-making system based on brain-like memory. The system includes:

[0087] The environmental image acquisition module 201 is used to acquire environmental images in front of the unmanned surface vessel.

[0088] The unmanned surface vessel (USV) environmental perception and decision-making model decision module 202 is used to input the environmental image into the USV environmental perception and decision-making model and output action commands; the USV environmental perception and decision-making model includes an image feature extractor, a BERT model, a fully connected layer, a short-term scene memory module and a long-term memory module connected in sequence.

[0089] The control module 203 is used to control the movement of the unmanned surface vessel using the action commands.

[0090] The image feature extractor is used to extract image features from the environmental image; the BERT model is used to extract image feature encoding containing text features from the image features; the fully connected layer is used to map the image feature encoding to an image encoding suitable for recognition by a large language model; the short-term scene memory module is used to preset multiple questions, and use the short-term scene memory of the large language model to answer each question in a set order to obtain multiple answers; the long-term memory module is used to use the long-term memory and context learning of the large language model to output action instructions based on multiple answers; the large language model is a large language model fine-tuned based on reinforcement learning.

[0091] The training process of the unmanned surface vessel (USV) environmental perception and decision-making model includes the following steps:

[0092] Step A: First, collect a large number of images related to the ocean and unmanned surface vessels (USVs) and their corresponding descriptive texts to create an image-text pair dataset. Then, filter out low-quality images and manually review the text to correct errors, including removing duplicate words and incoherent sentences. Finally, use the manually selected image-text pairs as training data for the visual encoder.

[0093] Step B: In the visual encoder, the pre-trained Vision Transformer model is used as an image feature extractor to extract semantic features of the image, and the pre-trained BERT (Bidirectional Encoder Representations from Transformers) model is used as a text feature extractor to extract semantic features of the text. The pre-trained model is then frozen.

[0094] Step C: Insert randomly initialized cross-attention modules into each Transformer block of the BERT model, fusing text and learnable encoding as initial input. A cross-attention module is added between the BERT model's self-attention module and feedforward neural network. This module fuses image and text features. Image features extracted by the pre-trained Vision Transformer are input into the cross-attention module, then processed by the feedforward neural network to obtain an encoding that simultaneously contains text and image features, used to calculate the subsequent image-text matching loss. Simultaneously, the text is processed separately by a BERT pre-trained self-attention module and feedforward neural network to obtain text features, which are combined with the previously obtained image feature encoding to calculate the image-text contrast loss. Additionally, a mask is added to the text self-attention module. The image encoding and the masked text are used to predict the masked content, resulting in the image-text generation loss. Learnable encodings are trained using these three losses. Then, an additional fully connected layer is trained at the output of the model using image-to-text generation loss. The role of the fully connected layer is to realize the mapping between the text features extracted from the image and the text encoding that the large language model can recognize. This allows the pre-trained large language model to recognize the features of the image with almost no modification and to perform text generation and inference based on them.

[0095] In step C, the definition of the mixed loss function during training is as follows:

[0096]

[0097] Where ω1 is the image-text contrast loss. The weight hyperparameters, ω2 is the image-text matching loss. The weight hyperparameters, ω3 is the image text generation loss. The weight hyperparameter.

[0098] Step D: Collect questions that can characterize the state of the unmanned surface vessel (USV) as expert knowledge. The real-time collected USV state images are then processed by the BERT model trained in Step C to obtain the corresponding image feature encodings.

[0099] Step E: After organizing the questions collected in Step D, they are sequentially concatenated with image codes in order of increasing difficulty and input into the language model. This easy-to-difficult input leverages the short-term contextual memory of the large language model. Based on the progressively increasing difficulty, the answer to the previous question serves as short-term contextual memory, aiding in the solution of the next question. Next, the attention layer in the large language model is utilized to implicitly optimize parameters during forward reasoning, thereby achieving long-term memory of the input text (corresponding to long-term memory of the flowchart) and contextual learning. This further guides the language model to fuse multimodal information and long-term memory to obtain various possible instructions for the unmanned surface vessel's next action.

[0100] Step F: Find professionals to analyze the instruction set obtained in Step E and rank the rationality of each instruction. Finally, compile the various images and questions along with their corresponding instructions into a small dataset to simulate working memory in the human brain, where the ranking of the instruction set is considered label information.

[0101] Step G: Train a reward model using the small dataset collected in F. This reward model is trained by using human-annotated sorting of instructions in different scenarios as training data. It simulates human scoring of each incoming instruction with reference to the current state of the unmanned vessel. The goal is to give each instruction the most reasonable score possible without changing the order of manually annotated instructions.

[0102] Furthermore, in step G, the loss function for training the reward model is defined as follows:

[0103]

[0104] Where r θ This represents the reward model, where x represents the input model's question and image. This represents an image and question in a small dataset, along with its two corresponding instructions, y. w and y l Let K represent the output instruction obtained, K represent the number of instructions in the instruction set, and σ represent the sigmoid activation function. The specific meaning of this loss function is that it arbitrarily selects two instructions y from the instruction set. w and y l When y w Instruction than y l When instructions are prioritized, we expect the reward model to favor y. w The score for the instruction should be higher than y. l The instructions are scored, thus cleverly using the manually specified order as a loss function that can train the model.

[0105] Step H: Use the trained reward model to train the reinforcement learner, and fine-tune the large language model in E, aiming to make the output of the large language model obtain a higher score in the reward function. Use the finally trained model to obtain the final command under the current sea state, and then make autonomous decisions for the unmanned surface vessel based on the obtained command.

[0106] Furthermore, in step H, the objective function for training the reinforcement learner is defined as follows:

[0107]

[0108] in, It is a reinforcement learning model, r θ It is the reward model in step G, π LLM This is the initial, untuned large language model. In this objective function, the first term r... θ (x,y) is used to enable the instructions trained by the model to achieve higher scores. It is worth noting that E (x,y) The sampled data can be viewed as the state in a classic reinforcement learning algorithm, and it changes as the model is updated. (Second term) This is a regularization term that uses the KL divergence between the probability distributions of the new model learned through reinforcement learning and the initial model to constrain the reinforcement learning model. The aim is to prevent the learned reinforcement learning model from deviating excessively from the initial model. The third term... It is the loss function of the pre-trained language model, which allows the model obtained after reinforcement learning to maintain good performance on the original pre-trained data.

[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0110] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A brain-like memory-based environmental perception and decision-making method for unmanned surface vessels, characterized in that, include: Acquire images of the environment in front of the unmanned surface vessel; The environmental image is input into the unmanned surface vessel's environmental perception and decision-making model, and action commands are output. The unmanned surface vessel's environmental perception and decision-making model includes an image feature extractor, a BERT model, a fully connected layer, a short-term scene memory module, and a long-term memory module connected in sequence. The unmanned surface vessel is controlled by the aforementioned action commands; The image feature extractor is used to extract image features from the environmental image; the BERT model is used to extract image feature codes containing text features from the image features; the fully connected layer is used to map the image feature codes to image codes applicable to large language models for recognition; the short-term scene memory module is used to preset multiple questions, and use the short-term scene memory of the large language model to answer each question in a set order to obtain multiple answers; The input questions, progressing from easy to difficult, leverage the short-term contextual memory of the large language model. Based on this progressively increasing difficulty, the answer to the previous question serves as short-term contextual memory, aiding in the solution of the next question. The long-term memory module utilizes the attention layer of the large language model to implicitly optimize parameters during forward reasoning, enabling long-term memory of the input text and contextual learning, and outputting action commands based on multiple answers. The large language model is a finely tuned version based on reinforcement learning. The method of using the long-term memory and contextual learning of the large language model to output action instructions based on multiple answers specifically includes: based on the large language model, using long-term memory and contextual learning to output a set of instructions composed of multiple instructions based on multiple answers, and outputting action instructions according to the set of instructions; The large language model is fine-tuned using a reinforcement learning model. The process of fine-tuning the large language model includes: Construct an instruction training set; the sample data in the instruction training set includes input data and label data, the input data is a sample instruction set, and the label data is the sorting of each instruction in the sample instruction set, with the sorting order being from high to low scores; The reward model is trained by taking the sample instruction set as input and the sorting of the sample instruction set as output, and the trained reward model is obtained. The instruction set output by the large language model is input into the trained reward model, and the first-ranked instruction is fed back to the large language model as an action instruction to fine-tune the large language model. The loss function for training the reward model is expressed as: ； in, () represents the reward model. x This represents the input question and image for a large language model. This represents an image and a question in the instruction training set, along with the first and second instructions corresponding to the image and the question. D This represents the instruction training set. Indicates the first instruction. Indicates the second instruction. K Indicates the number of instructions in the sample instruction set. This represents the sigmoid activation function. This represents all the parameters of the reward model.

2. The unmanned surface vessel environmental perception and decision-making method based on brain-like memory according to claim 1, characterized in that, The BERT model is a trained BERT model; a cross-attention module is added between the self-attention module and the feedforward neural network of each Transformer block in the BERT model; The training process of the BERT model includes: Collect an unmanned surface vessel (USV) environment dataset; each sample in the USV environment dataset includes an image of the USV environment and corresponding text description information. Each sample data point includes the following during training: The unmanned surface vessel's environmental images are input into a pre-trained image feature extractor, which outputs sample image features. The text description information corresponding to the unmanned surface vessel environment image is input into the BERT model, and the sample image features are input into the cross attention module of each Transformer block in the BERT model; The features output by the cross-attention module are processed by a feedforward neural network to obtain the feature encoding of the first sample image. The image-text matching loss is determined based on the feature encoding of the first sample image; The text description information corresponding to the unmanned surface vessel's environmental image is input into a pre-trained network, which outputs the feature encoding of the second sample image. The pre-trained network includes a self-attention module and a feedforward neural network connected in sequence. The image-text contrast loss is determined based on the feature encoding of the first sample image and the feature encoding of the second sample image; Add a mask to the self-attention module of each Transformer block in the BERT model; The text description information corresponding to the unmanned surface vessel environment image is input into the BERT model after adding a mask. The sample image features are input into the cross attention module of each Transformer block in the BERT model. The features output by the cross attention module are processed by a feedforward neural network to obtain the third sample image feature encoding. The image text generation loss is determined based on the feature encoding of the third sample image and the label data corresponding to the feature encoding of the third sample image. The BERT model is optimized based on image-text matching loss, image-text contrast loss, and image-text generation loss.

3. The unmanned surface vessel environmental perception and decision-making method based on brain-like memory according to claim 1, characterized in that, The objective function for fine-tuning a large language model is expressed as: ； in, Represents the objective function value. Represents a reward model. It is the initial, untuned large language model. This represents a reinforcement learning training set consisting of an image and a question, along with the action instructions output by the large language model corresponding to the image and the question. This represents the reinforcement learning training set. This represents the current reinforcement learning model. The hyperparameter represents the degree of deviation between the currently trained reinforcement learning model and the initial reinforcement learning model. x This represents the image and question input to the large language model. y This represents the action instructions output by the large language model. This parameter represents the weighting of the large language model on the reinforcement learning training set and the original pre-training data. This indicates that reinforcement learning training focuses on a single image and a question. This represents the pre-training data used when the original large language model is pre-trained.

4. The unmanned surface vessel environmental perception and decision-making method based on brain-like memory according to claim 1, characterized in that, The image feature extractor is a trained visual converter.

5. An unmanned surface vessel (USV) environmental perception and decision-making system based on brain-like memory, characterized in that, include: The environmental image acquisition module is used to acquire images of the environment in front of the unmanned surface vessel. The unmanned surface vessel (USV) environmental perception and decision-making model decision module is used to input the environmental image into the USV environmental perception and decision-making model and output action commands; the USV environmental perception and decision-making model includes an image feature extractor, a BERT model, a fully connected layer, a short-term scene memory module and a long-term memory module connected in sequence. The control module is used to control the movement of the unmanned surface vessel using the action commands; The image feature extractor is used to extract image features from the environmental image; the BERT model is used to extract image feature codes containing text features from the image features; the fully connected layer is used to map the image feature codes to image codes suitable for recognition by a large language model; the short-term scene memory module is used to preset multiple questions, and use the short-term scene memory of the large language model to answer each question in a set order to obtain multiple answers; The input questions, progressing from easy to difficult, leverage the short-term contextual memory of the large language model. Based on this progressively increasing difficulty, the answer to the previous question serves as short-term contextual memory, aiding in the solution of the next question. The long-term memory module utilizes the attention layer of the large language model to implicitly optimize parameters during forward reasoning, enabling long-term memory of the input text and contextual learning, and outputting action commands based on multiple answers. The large language model is a finely tuned version based on reinforcement learning. The method of using the long-term memory and contextual learning of the large language model to output action instructions based on multiple answers specifically includes: based on the large language model, using long-term memory and contextual learning to output a set of instructions composed of multiple instructions based on multiple answers, and outputting action instructions according to the set of instructions; The large language model is fine-tuned using a reinforcement learning model. The process of fine-tuning the large language model includes: Construct an instruction training set; the sample data in the instruction training set includes input data and label data, the input data is a sample instruction set, and the label data is the sorting of each instruction in the sample instruction set, with the sorting order being from high to low scores; The reward model is trained by taking the sample instruction set as input and the sorting of the sample instruction set as output, and the trained reward model is obtained. The instruction set output by the large language model is input into the trained reward model, and the first-ranked instruction is fed back to the large language model as an action instruction to fine-tune the large language model. The loss function for training the reward model is expressed as: ； in, () represents the reward model. x This represents the input question and image for a large language model. This represents an image and a question in the instruction training set, along with the first and second instructions corresponding to the image and the question. D This represents the instruction training set. Indicates the first instruction. Indicates the second instruction. K Indicates the number of instructions in the sample instruction set. This represents the sigmoid activation function. This represents all the parameters of the reward model.

Images