Marine remote sensing ship image segmentation method based on visual segmentation large model

Abstract

The invention discloses an offshore remote sensing ship image segmentation method based on a visual segmentation large model, and the method comprises the following steps: obtaining a plurality of to-be-detected offshore remote sensing ship images, and taking the images as a to-be-detected data set; obtaining a visual segmentation large model, wherein the visual segmentation large model comprises an image encoder, a prompt encoder and a mask decoder; replacing a plurality of attention network layers of an image encoder of the obtained visual segmentation large model by adopting a low-rank adapter network; training the replaced visual segmentation large model to obtain a trained marine remote sensing ship image segmentation large model; and inputting a to-be-detected marine remote sensing ship image into the trained marine remote sensing ship image segmentation large model, and outputting a semantic segmentation prediction map. According to the method, ship semantic segmentation can be rapidly and accurately carried out under the complex background of the marine remote sensing ship image, and the efficiency and accuracy of marine remote sensing ship image segmentation are improved.

Description

Technical Field

[0001] The invention relates to the technical field of image segmentation, and in particular to a method for segmenting marine remote sensing ship images based on a large visual segmentation model. Background Art

[0002] SAR remote sensing technology is becoming increasingly important because it can reveal surface features regardless of day or night and weather conditions. However, due to the inherent characteristics of SAR images, including speckle noise, shadow effects, and complex environmental backgrounds, accurate segmentation of ship images poses significant challenges. Compared with synthetic aperture radar (SAR) images, high-definition optical remote sensing images provide richer details, greatly facilitating the accurate identification of ship targets.

[0003] At present, obtaining useful visual information from images has become a basic research idea in the field of computer vision. Image semantic segmentation is one of the core tasks in the field of computer vision. It aims to subdivide the input image into multiple semantic regions, each of which is identified by a corresponding mask. Compared with early visual technologies that can only identify image edges or simple features, semantic segmentation achieves fine pixel-level analysis. However, current image segmentation methods use convolutional networks for simple stacking or simple attention mechanisms, which cannot handle the problem of marine remote sensing ship image segmentation well.

[0004] When the existing large-scale image segmentation and visual segmentation models are applied to the field of marine remote sensing ship image segmentation, there may be problems such as long training time, high difficulty in parameter adjustment, and insufficient adaptability to complex sea conditions. Specifically, there are the following disadvantages:

[0005] 1. Traditional marine ship monitoring methods mainly rely on manual observation, which not only causes missed detection and false detection, but also consumes a lot of manpower and material resources;

[0006] 2. Existing image segmentation methods, such as those based on edge information, grayscale statistics, fractal models, and saliency vision, have poor detection effects when the sea conditions are complex. These methods have weak background suppression capabilities and take a long time to complete, especially in bad weather or when there is cloud and fog interference, where the performance is significantly reduced.

[0007] 3. Although the large visual segmentation model performs well in other fields, its application in the field of marine remote sensing ship image segmentation still needs more exploration and optimization. The existing large visual segmentation model may have problems such as long training time, high difficulty in parameter adjustment, and insufficient adaptability to complex sea conditions when processing marine remote sensing ship images.

[0008] 4. Existing image segmentation methods use convolutional networks for simple stacking or simple attention mechanisms, which cannot handle the problem of marine remote sensing ship image segmentation well, especially in terms of feature extraction and segmentation efficiency.

[0009] In summary, in view of the problems of existing marine remote sensing ship image segmentation methods, such as reliance on manual observation, poor detection effect when the sea conditions are complex, poor fine-tuning effect of large visual segmentation models, and low feature extraction and segmentation efficiency, there is an urgent need for a technology that can quickly and accurately perform ship semantic segmentation. Summary of the invention

[0010] In order to overcome the defects and shortcomings of the prior art, the present invention provides a method for segmenting marine remote sensing ship images based on a large visual segmentation model. The present invention utilizes the image segmentation large model fine-tuning technology to assign each pixel in the image to a specific category, complete pixel-level classification, and can quickly and accurately perform ship semantic segmentation under the complex background of marine remote sensing ship images, reduce manual recognition and operation, and use a low-rank adapter network to replace several attention network layers of the acquired large model, and train on a data set, which can improve the efficiency and accuracy of marine remote sensing ship image segmentation.

[0011] In order to achieve the above object, the present invention adopts the following technical solutions:

[0012] The present invention provides a method for segmenting marine remote sensing ship images based on a large visual segmentation model, comprising the following steps:

[0013] Acquire several images of marine remote sensing ships to be detected as the data set to be tested;

[0014] Obtaining a large visual segmentation model, wherein the large visual segmentation model includes an image encoder, a hint encoder, and a mask decoder;

[0015] We use low-rank adapter networks to replace several attention network layers of the image encoder of the acquired large visual segmentation model;

[0016] The replaced visual segmentation large model is trained to obtain a trained marine remote sensing ship image segmentation large model;

[0017] The image of the marine remote sensing ship to be detected is input into the trained marine remote sensing ship image segmentation model, and a semantic segmentation prediction map is output.

[0018] As a preferred technical solution, the marine remote sensing ship image to be detected includes a ship target and a background, each marine remote sensing ship image to be detected has a different background, and each marine remote sensing ship image to be detected is provided with a mask true value image corresponding thereto.

[0019] As a preferred technical solution, the visual segmentation model includes an image encoder, a prompt encoder, and a mask decoder. The image of the marine remote sensing ship to be detected is input into the image encoder, and the image encoder outputs image coding features;

[0020] Inputting the sparse hint and the dense hint into a hint encoder, the hint encoder outputting hint encoding features;

[0021] The mask decoder maps image encoding features, cue encoding features and output tokens to dense cues. The mask decoder updates all features using cue-style self-attention and cross-attention mechanisms, upsamples image encoding features, and a multilayer perceptron maps output tokens to a dynamic linear classifier to calculate the dense cue foreground probability of each image position.

[0022] As a preferred technical solution, a low-rank adapter network is used to replace several attention network layers of the image encoder of the acquired large model, specifically including:

[0023] The low-rank adapter network includes a low-rank decomposition network and a hybrid expert network;

[0024] The low-rank decomposition network includes: residual weights, a low-rank encoder, and a low-rank decoder;

[0025] The weight matrix W of the original attention network layer O Applying singular value decomposition, it is expressed as:

[0026] W O =USV T

[0027] Among them, U and V are singular vectors with orthogonal columns, V T is the transpose of V, S represents a diagonal matrix,

[0028] The original attention network layer weight matrix W O It is divided into the main weight matrix and the residual weight matrix;

[0029] Low rank encoder W e and the low-rank decoder W d According to the main weight singular value and vector initialization, the residual weight uses the residual weight singular value S [r:,r:] and vector U [:,r:] , The product of is initialized, which is specifically expressed as:

[0030]

[0031] Among them, W resrepresents the residual weight, r is the rank of low-rank decomposition, r<<min(m,n), m represents the number of image feature input channels, n represents the number of image feature output channels, YesS [r:,r:] The result after the square root, S [:r,:r] represents the main singular value, {U [:,:r] ,V [:,:r]} represents the main singular vector, S [r:,r:] represents the residual singular value, {U [:,r:] ,V [:,r:]} represents the residual singular vector, [:r] represents the first r dimensions, [:,:r] represents the first r columns of a two-dimensional matrix, [:r,:r] represents the first r rows and first r columns of a two-dimensional matrix, [:,r:] represents the last r columns of a two-dimensional matrix, and [r:,r:] represents the last r rows and last r columns of a two-dimensional matrix;

[0032] The hybrid expert network allocates different expert networks for training according to different image features of different marine remote sensing ship images.

[0033] As a preferred technical solution, the hybrid expert network includes multiple expert networks and a gating module. e x is input to the gating module G, and the gating score G(W e x), specifically expressed as:

[0034] G(W e x)=Softmax(KeepTopK(H(W e x),k))

[0035]

[0036] H(W e x) i =(x h ·W g ) i +StandardNormal()·Softplus((x h ·W noise ) i )

[0037] x h =Reshape(GlobalAvePool(W e x),(B,r))

[0038] Among them, GlobalAvePool represents global average pooling, Reshape represents reshaping operation, Softplus represents activation function Softplus, StandardNormal represents standard normal, and image feature We The size of x is B×r×H×W, where r represents the rank of the low-rank decomposition, B is the batch size during training, H and W image features correspond to height and width, and W g represents the trainable gating weight, W noise represents noise, v i are the k values ​​with the largest expert scores, and v is the score of all experts.

[0039] As a preferred technical solution, the image of the marine remote sensing ship to be detected is input into the low-rank adapter network to obtain the output result, which is specifically expressed as:

[0040]

[0041] Among them, h is the output result of the low-rank adapter network, x represents the image features of the marine remote sensing ship to be detected, and E i is the i-th expert network, and n represents the total number of expert networks.

[0042] As a preferred technical solution, the replaced visual segmentation model is trained to obtain a trained marine remote sensing ship image segmentation model, which specifically includes:

[0043] Freeze the prompt encoder to train the low-rank encoder, low-rank decoder and hybrid expert network in the low-rank decomposition network of the low-rank adaptation network;

[0044] The marine remote sensing ship image is input into the image encoder replaced by the low-rank adapter network, and the image coding features are output. The prompt encoder extracts the prompt coding features. The image coding features and the prompt coding features are input into the mask decoder. The mask decoder predicts a binary mask, distinguishes the foreground and background, and outputs a semantic segmentation map of the marine remote sensing ship image.

[0045] The present invention also provides a marine remote sensing ship image segmentation system based on a large visual segmentation model, which is used to implement the above-mentioned marine remote sensing ship image segmentation method based on a large visual segmentation model. The system includes: a test data set construction module, a large visual segmentation model acquisition module, a network replacement module, a training module, and a semantic segmentation prediction map output module;

[0046] The test data set construction module is used to obtain a number of marine remote sensing ship images to be detected as the test data set;

[0047] The visual segmentation large model acquisition module is used to acquire the visual segmentation large model, and the visual segmentation large model includes an image encoder, a prompt encoder, and a mask decoder;

[0048] The network replacement module is used to replace several attention network layers of the image encoder of the obtained visual segmentation large model with a low-rank adapter network;

[0049] The training module is used to train the replaced visual segmentation model to obtain a trained marine remote sensing ship image segmentation model;

[0050] The semantic segmentation prediction map output module is used to input the image of the offshore remote sensing ship to be detected into the trained offshore remote sensing ship image segmentation large model, and output the semantic segmentation prediction map.

[0051] The present invention also provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the above-mentioned marine remote sensing ship image segmentation method based on a large visual segmentation model.

[0052] The present invention also provides a computer device, comprising a processor and a memory for storing a program executable by the processor, wherein when the processor executes the program stored in the memory, the marine remote sensing ship image segmentation method based on the visual segmentation large model as described above is implemented.

[0053] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0054] (1) The present invention uses an end-to-end design based on a large visual segmentation model to reduce manual observation and achieve high efficiency in marine vessel segmentation.

[0055] (2) The present invention introduces a hybrid expert network, which dynamically routes to an expert network in each round of iteration to address the problem of poor multi-scale feature extraction capability caused by different distributions of marine remote sensing ship image data, thereby achieving high adaptability of segmentation.

[0056] (3) The present invention uses a large visual segmentation model with a huge number of parameters and complex design. It has rich prior knowledge and can adapt to complex sea conditions after fine-tuning the model to achieve high segmentation accuracy.

[0057] (4) The present invention introduces a low-rank adapter network, which greatly reduces the training cost caused by fine-tuning the large visual segmentation model and has a low implementation cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flow chart of a method for segmenting marine remote sensing ship images based on a large visual segmentation model of the present invention;

[0059] Figure 2 It is a schematic diagram of the overall architecture of the visual segmentation model of the present invention;

[0060] Figure 3 Schematic diagram of the architecture of the low-rank decomposition network of the present invention;

[0061] Figure 4 A schematic diagram of the architecture of the hybrid expert network of the present invention;

[0062] Figure 5 A schematic diagram of an implementation architecture for end-to-end training of a replaced large visual segmentation model according to the present invention;

[0063] Figure 6 This is a schematic diagram of semantic segmentation prediction of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0065] Example 1

[0066] like Figure 1 As shown, this embodiment provides a method for segmenting marine remote sensing ship images based on a large visual segmentation model, comprising the following steps:

[0067] S1: Acquire several images of marine remote sensing ships to be detected as the data set to be tested;

[0068] In this embodiment, each marine remote sensing ship image to be detected includes a ship target and a background, and each marine remote sensing ship image to be detected has a different background; each marine remote sensing ship image to be detected is provided with a mask true value image corresponding to it one by one;

[0069] S2: Get the large visual segmentation model;

[0070] In this embodiment, the visual segmentation model uses an open source model: the Segmentation Everything Model SAM, which is based on the Vision Transformer. Figure 2 As shown in the figure, it includes three components: an image encoder, a flexible hint encoder and a fast mask decoder. The model parameter size has three types: huge, large and base. The model parameter size is from large to small.

[0071] In this embodiment, the image encoder can be any network that can output a C×H×W image embedding. For scalability and strong pre-training capabilities, SAM uses ViT pre-trained with MAE (Masked autoencoders) and minimizes it to handle high-resolution inputs. Since SAM runs in real time for each prompt, it can afford a higher number of image encoder FLOPs because it is only calculated once per image, not per prompt;

[0072] In this embodiment, the prompt encoder considers two sets of prompts: sparse (points, boxes, text) and dense (masks), and the sparse prompts are mapped to 256-dimensional vector embeddings. A point is represented as the sum of the position encoding of the point and two learned embedded encodings, which respectively indicate whether the point is in the foreground or background. A box is represented by a pair of embedded encodings: the sum of the position encoding of the upper left corner and the learned embedded encoding representing "upper left corner"; the same structure, but using the learned embedded encoding representing "lower right corner". Finally, the text encoder of the visual language model CLIP is used to represent free-form text (in general, any text encoder will do). Dense prompts (i.e., masks) have a spatial correspondence with the image. Dense prompts (i.e., masks) are embedded using convolutions and summed element-wise with the image embedding;

[0073] In this embodiment, the mask decoder effectively maps the image encoding features, the hint encoding features, and the output token to the mask, and the mask decoder uses hint-style self-attention and cross-attention in both directions (hint to image embedding and vice versa) to update all features. After running two blocks, the image encoding features are upsampled, and the MLP maps the output token to a dynamic linear classifier that calculates the mask foreground probability for each image position.

[0074] S3: Use a low-rank adapter network to replace several attention network layers of the image encoder of the acquired visual segmentation large model;

[0075] The original attention network layer of the image encoder of the visual segmentation model to be replaced includes: the query attention weight Q, key attention weight K and value attention weight V of the image encoder of the visual segmentation model, the query attention weight Q is used to calculate the query attention feature vector of the marine remote sensing ship image, the key attention weight K is used to calculate the key attention feature vector of the marine remote sensing ship image, and the value attention weight V is used to calculate the value attention feature vector of the marine remote sensing ship image. The weights of the original attention network layer are recorded as m represents the number of image feature input channels, and n represents the number of image feature output channels;

[0076] In this embodiment, the low-rank adapter network includes a low-rank decomposition network, a hybrid expert network, such as Figure 3 As shown, the low-rank decomposition network includes: residual weight W res , low-rank encoder W e , low-rank decoder W d , the weight matrix W of the original attention network layer O Apply singular value decomposition (SVD), as shown in the formula:

[0077] W O =USV T

[0078] in, is a singular vector with orthogonal columns, V T is the transpose of V, The operation diag(S) converts s into a diagonal matrix S. Represents the singular values ​​in descending order, the current r singular values ​​S [:r] Obviously larger than the other singular values ​​S [r:] When W O The inner rank of is denoted as r, therefore, S together with U and V can be divided into two groups: the main singular values ​​S [:r,:r] and the principal singular vectors -- {U [:,:r] ,V [:,:r]}, and the residual singular value S [r:,r:] and residual singular vectors—{U [:,r:] ,V [:,r:]}, [:r] represents the first r dimensions. [:,:r] represents the first r columns of a two-dimensional matrix, [:r,:r] represents the first r rows and first r columns of a two-dimensional matrix, [:,r:] represents the last r columns of a two-dimensional matrix, and [r:,r:] represents the last r rows and last r columns of a two-dimensional matrix;

[0079] According to the size of the singular value, the present invention converts the original attention network layer weight matrix W O Divided into two parts: the sovereign weight matrix W pri and the residual weight matrix W res , the sovereign weight matrix W pri The rank is low, so we can use and The product of, where r<<min(m,n), r is the rank of the low-rank decomposition, W e and W d Initialized according to the master weight singular value and vector, and trainable, while W res Then use the residual weight singular value S [r:,r:] and vector U [:,r:] , The product of is initialized and frozen, as shown in the following formula, W d W e The integration with residual weights retains the original general semantic segmentation capability of the large image segmentation model:

[0080]

[0081] in, YesS [r:,r:]The result after the square root shows that this embodiment uses a low-rank decomposition network, which can adjust a small number of parameters of the image segmentation model while using a small amount of computing resources to improve the segmentation ability of the image segmentation model in marine remote sensing ship image data.

[0082] like Figure 4 As shown, the hybrid expert network includes multiple expert networks E i and a gating module G that dynamically selects the experts to be activated during the forward pass. Each expert network E i It is a convolutional neural network that specializes in studying specific feature sizes, and the compact gating module learns to dynamically select expert networks based on input data. The hybrid expert network can assign different expert networks to train according to the different image features of different marine remote sensing ship images, so that the large marine image segmentation model can cope with the marine remote sensing ship image segmentation situations in a variety of complex backgrounds;

[0083] In this embodiment, the gate control module G is explained as follows:

[0084] The image features W are input to the gating module G. e The size of x is B×r×H×W. In order to calculate the gating scores assigned to the k experts for the B image feature samples in the batch First, we need to calculate the value of n experts First, global average pooling (denoted as 'GlobalAvePool') is applied, followed by a reshaping operation (denoted as 'Reshape') to obtain Specifically, after 'GlobalAvePool', the size of x becomes B×r×1×1, and then after 'Reshape' it becomes B×r. Subsequently, the trainable gating weights and noise term To calculate H(x):

[0085] x h =Reshape(GlobalAvePool(W e x),(B,r))

[0086] H(W e x) i =(x h ·W g ) i +StandardNormal()·Softplus((x h ·W noise ) i )

[0087] In calculating H(W e x) Then, for each expert Ei Based on its corresponding H(W e x) i value, only the largest k values ​​are retained (labeled as 'KeepTopK', where v i is the score H(W e x) i , v is the score of all experts), and set the remaining values ​​to -∞. After the 'Softmax' operation, these experts whose corresponding gating values ​​are 0 (that is, if G(W e x) j is 0), then there is no need to calculate E i (W e x), which ensures that only the k experts with the highest scores contribute to the final output, and effectively "turns off" other less important experts by setting the gating value of lower scores to 0, thereby achieving efficient computing and resource allocation, and improving the efficiency of marine remote sensing ship image segmentation. This process is a key step in the MoE (Mixture of Experts) architecture, which aims to improve the performance and efficiency of the model by dynamically routing inputs to the most relevant experts. The formula is as follows:

[0088] G(W e x)=Softmax(KeepTopK(H(W e x),k))

[0089]

[0090] After adopting the low-rank adapter network, the image feature x is input into the low-rank adapter network, and the output result of the low-rank adapter network is obtained, which is specifically expressed as:

[0091]

[0092] Among them, h is the output result of the low-rank adapter network, and the input image feature B is the batch size during training, H and W image features correspond to height and width, E i is the i-th expert among all n experts, G is a gated network that only activates the top-k values, and each expert uses deformable convolutions with different padding and dilation rates, thereby enhancing the ability to learn image features at various scales, so that the large model after the low-rank adapter network replacement can adapt to the data distribution and characteristics of different marine remote sensing ships after training;

[0093] S4: training the replaced visual segmentation model on a certain data set to obtain a marine remote sensing ship image segmentation model;

[0094] In order to automate the replacement of the large visual segmentation model and reduce manual operations, end-to-end training was performed. Figure 5 As shown, the prompt encoder is frozen and not trained. For the image encoder, the low-rank encoder, low-rank decoder and hybrid expert network in the low-rank decomposition network of the low-rank adaptation network are trained, and the other parts are frozen to achieve efficient training. The parameters of the mask decoder are fully trained because it is a lightweight module. During training, the marine remote sensing ship image is input into the image encoder replaced by the low-rank adapter network, and the image coding features are output. The prompt encoder automatically inputs constant prompts and extracts the prompt coding features. The image coding features and the prompt coding features are input into the mask decoder together. The mask decoder predicts a binary mask, distinguishes the foreground and the background, and outputs a semantic segmentation map of the marine remote sensing ship image. After the training is completed, a large model for the segmentation of marine remote sensing ship images can be obtained.

[0095] S5: Input the image dataset to be detected into the trained marine remote sensing ship image segmentation model, and output the semantic segmentation prediction map.

[0096] The trained image segmentation model is an end-to-end designed and trained model that can complete the segmentation of marine remote sensing ship images by simply inputting images. Figure 6 As shown, the corresponding semantic segmentation prediction map is obtained.

[0097] The existing methods rely on manual operation, cannot cope with complex marine background conditions, and have poor effects. The present invention introduces a large visual segmentation model to perform marine remote sensing ship image segmentation, and uses the image segmentation large model fine-tuning technology to achieve fast and accurate ship semantic segmentation under the complex background of marine remote sensing ship images, reduce manual recognition and operation, and solve the limitations of existing methods such as high training cost, low monitoring capability, and insufficient adaptability to complex sea conditions.

[0098] Example 2

[0099] This embodiment provides a marine remote sensing ship image segmentation system based on a visual segmentation large model, which is used to implement the marine remote sensing ship image segmentation method based on a visual segmentation large model in the above embodiment 1. The system includes: a test data set construction module, a visual segmentation large model acquisition module, a network replacement module, a training module, and a semantic segmentation prediction map output module;

[0100] In this embodiment, the test data set construction module is used to obtain a number of images of marine remote sensing ships to be detected as the test data set;

[0101] In this embodiment, the visual segmentation large model acquisition module is used to acquire the visual segmentation large model, and the visual segmentation large model includes an image encoder, a hint encoder, and a mask decoder;

[0102] In this embodiment, the network replacement module is used to replace several attention network layers of the image encoder of the obtained visual segmentation large model with a low-rank adapter network;

[0103] In this embodiment, the training module is used to train the replaced visual segmentation model to obtain a trained marine remote sensing ship image segmentation model;

[0104] In this embodiment, the semantic segmentation prediction map output module is used to input the image of the marine remote sensing ship to be detected into the trained marine remote sensing ship image segmentation large model, and output the semantic segmentation prediction map.

[0105] Example 3

[0106] This embodiment provides a storage medium, which may be a ROM, RAM, disk, CD or other storage medium, and the storage medium stores one or more programs. When the program is executed by the processor, the marine remote sensing ship image segmentation method based on the visual segmentation large model of embodiment 1 is implemented.

[0107] Example 4

[0108] This embodiment provides a computing device, which can be a desktop computer, a laptop computer, a smart phone, a PDA handheld terminal, a tablet computer or other terminal device with a display function. The computing device includes a processor and a memory, and the memory stores one or more programs. When the processor executes the program stored in the memory, the marine remote sensing ship image segmentation method based on the visual segmentation large model of Example 1 is implemented.

[0109] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A method for segmenting marine remote sensing ship images based on a large visual segmentation model, characterized in that: The steps include: Acquire several images of marine remote sensing ships to be detected as the data set to be tested; Obtaining a large visual segmentation model, wherein the large visual segmentation model includes an image encoder, a hint encoder, and a mask decoder; We use low-rank adapter networks to replace several attention network layers of the image encoder of the acquired large visual segmentation model; The replaced visual segmentation large model is trained to obtain a trained marine remote sensing ship image segmentation large model; The image of the marine remote sensing ship to be detected is input into the trained marine remote sensing ship image segmentation model, and a semantic segmentation prediction map is output.

2. The method for segmenting marine remote sensing ship images based on a large visual segmentation model according to claim 1 is characterized in that: The marine remote sensing ship image to be detected includes a ship target and a background. Each marine remote sensing ship image to be detected has a different background. Each marine remote sensing ship image to be detected is provided with a mask true value image corresponding to it one by one.

3. The method for segmenting marine remote sensing ship images based on a large visual segmentation model according to claim 1 is characterized in that: The visual segmentation model includes an image encoder, a prompt encoder, and a mask decoder. The image of the offshore remote sensing ship to be detected is input into the image encoder, and the image encoder outputs image coding features. Inputting the sparse hint and the dense hint into a hint encoder, the hint encoder outputting hint encoding features; The mask decoder maps image encoding features, cue encoding features and output tokens to dense cues. The mask decoder updates all features using cue-style self-attention and cross-attention mechanisms, upsamples image encoding features, and a multilayer perceptron maps output tokens to a dynamic linear classifier to calculate the dense cue foreground probability of each image position.

4. The marine remote sensing ship image segmentation method based on the visual segmentation large model according to claim 1 is characterized in that: The low-rank adapter network is used to replace several attention network layers of the image encoder of the acquired large model, including: The low-rank adapter network includes a low-rank decomposition network and a hybrid expert network; The low-rank decomposition network includes: residual weights, a low-rank encoder, and a low-rank decoder; For the original attention network layer weight matrix W O Applying singular value decomposition, it is expressed as: IN O =USV T Among them, U and V are singular vectors with orthogonal columns, V T is the transpose of V, S represents a diagonal matrix, The original attention network layer weight matrix W O It is divided into the main weight matrix and the residual weight matrix; Low rank encoder W e and the low-rank decoder W d According to the main weight singular value and vector initialization, the residual weight uses the residual weight singular value S [r:,r:] and vector U [:,r:] ,vector The product of is initialized, which is specifically expressed as: Among them, W res represents the residual weight, r is the rank of low-rank decomposition, r<<min(m,n), m represents the number of image feature input channels, n represents the number of image feature output channels, YesS [r:,r:] The result after the square root, S [:r,:r] represents the main singular value, {U [:,:r] ,V [:,:r] } represents the main singular vector, S [r:,r:] represents the residual singular value, {U [:,r:] ,V [:,r:] } represents the residual singular vector, [:r] represents the first r dimensions, [:,:r] represents the first r columns of a two-dimensional matrix, [:r,:r] represents the first r rows and first r columns of a two-dimensional matrix, [:,r:] represents the last r columns of a two-dimensional matrix, and [r:,r:] represents the last r rows and last r columns of a two-dimensional matrix; The hybrid expert network allocates different expert networks for training according to different image features of different marine remote sensing ship images.

5. The method for segmenting marine remote sensing ship images based on a large visual segmentation model according to claim 4 is characterized in that: The hybrid expert network includes multiple expert networks and a gating module. e x is input to the gating module G, and the gating score G(W e x), specifically expressed as: G(W e x)=Softmax(KeepTopK(H(W e x),k)) H(W e x) i =(x h ·W g ) i +StandardNormal()·Softplus((x h ·W noise ) i ) x h =Reshape(GlobalAvePool(W e x),(B,r)) Among them, GlobalAvePool represents global average pooling, Reshape represents reshaping operation, Softplus represents activation function Softplus, StandarNormal represents standard normal, and image feature W e The size of x is B×r×H×W, where r represents the rank of the low-rank decomposition, B is the batch size during training, H and W image features correspond to height and width, and W g represents the trainable gating weight, W noise represents noise, v i are the k values ​​with the largest expert scores, and v is the score of all experts.

6. The method for segmenting marine remote sensing ship images based on a large visual segmentation model according to claim 5 is characterized in that: The image of the marine remote sensing ship to be detected is input into the low-rank adapter network to obtain the output result, which is specifically expressed as: Among them, h is the output result of the low-rank adapter network, x represents the image features of the marine remote sensing ship to be detected, and E i is the i-th expert network, and n represents the total number of expert networks.

7. The method for segmenting marine remote sensing ship images based on a large visual segmentation model according to claim 3 is characterized in that: The replaced visual segmentation model is trained to obtain a trained marine remote sensing ship image segmentation model, which specifically includes: Freeze the prompt encoder to train the low-rank encoder, low-rank decoder and hybrid expert network in the low-rank decomposition network of the low-rank adaptation network; The marine remote sensing ship image is input into the image encoder replaced by the low-rank adapter network, and the image coding features are output. The prompt encoder extracts the prompt coding features. The image coding features and the prompt coding features are input into the mask decoder. The mask decoder predicts a binary mask, distinguishes the foreground and background, and outputs a semantic segmentation map of the marine remote sensing ship image.

8. A marine remote sensing ship image segmentation system based on a large visual segmentation model, characterized in that: The method for segmenting marine remote sensing ship images based on a large visual segmentation model according to any one of claims 1 to 7 is used to implement the system, which comprises: a test data set construction module, a large visual segmentation model acquisition module, a network replacement module, a training module, and a semantic segmentation prediction map output module; The test data set construction module is used to obtain a number of marine remote sensing ship images to be detected as the test data set; The visual segmentation large model acquisition module is used to acquire the visual segmentation large model, and the visual segmentation large model includes an image encoder, a prompt encoder, and a mask decoder; The network replacement module is used to replace several attention network layers of the image encoder of the obtained visual segmentation large model with a low-rank adapter network; The training module is used to train the replaced visual segmentation model to obtain a trained marine remote sensing ship image segmentation model; The semantic segmentation prediction map output module is used to input the image of the offshore remote sensing ship to be detected into the trained offshore remote sensing ship image segmentation large model, and output the semantic segmentation prediction map.

9. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the marine remote sensing ship image segmentation method based on a visual segmentation large model as described in any one of claims 1 to 7 is implemented.

10. A computer device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, the marine remote sensing ship image segmentation method based on the visual segmentation large model as described in any one of claims 1 to 7 is implemented.

Images