Water area environment target identification method and system based on SAM model

Abstract

The invention relates to the technical field of image recognition, in particular to a water area environment target recognition method and system based on an SAM model, and the method comprises the following steps: embedding a series of Adapter modules at a specific position of an original SAM model, and generating a W-SAM model for a water area environment; obtaining a water area environment picture, and generating a water area environment data set according to the water area environment picture; according to the water area environment data set, training, testing and adjusting the W-SAM model, and generating a target W-SAM model; and identifying a target in the water area environment by using the target W-SAM model. According to the method, an Adapter mode is selected, efficient fine tuning is carried out by embedding the Adapter on the basis of segmenting all models SAM, and the SAM not only retains original massive rich knowledge, but also can effectively adapt to a water area environment target image.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method and system for identifying targets in a water environment based on a SAM model. Background Art

[0002] In the field of computer vision, large visual models, especially those pre-trained with large-scale datasets, such as the Segment Anything Model (SAM), have demonstrated strong generalization capabilities and efficient performance in a variety of visual tasks. The powerful segmentation capability of the SAM model is due to its unique architectural design, including image encoder, prompt encoder, and mask decoder. The image encoder is based on a Vision Transformer (ViT) structure pre-trained by MAE. The output of the image encoder is a 16-fold downsampled encoding of the input image, which can effectively extract image features; the prompt encoder supports multiple forms of input, enhancing the flexibility of the model, which can be sparse (points, boxes, text) or dense (masks); the mask decoder is a modified Transformer decoder that includes a dynamic mask prediction head. SAM uses a bidirectional cross-attention mechanism, one for prompt-to-image embedding and the other for image-to-prompt embedding, to learn the mapping relationship between prompt and image embedding in each module. After running both modules, SAM upsamples the image embedding and uses a Multilayer Perceptron (MLP) to map the output tokens to a dynamic linear classifier that predicts the object mask for a given image.

[0003] Although SAM has shown significant performance improvements in a wide range of application scenarios, its direct application often fails to achieve ideal recognition results and its performance is significantly limited when faced with the unique complexity and specificity of underwater environments. This is mainly due to the unique challenges of underwater environments, such as imaging mode differences, irregular shapes, blurred boundaries, small target sizes, and low contrast, which are not common in general datasets.

[0004] To overcome this challenge, researchers have explored a variety of strategies to adapt the SAM model to the needs of specific fields. Among them, two main methods have been widely studied: comprehensive fine-tuning and efficient fine-tuning. The comprehensive fine-tuning method adjusts all parameters of the model to adapt to new tasks, but this method easily causes the model to forget old knowledge and may cause overfitting problems. In contrast, the efficient fine-tuning method can achieve optimization for specific tasks while maintaining the model's memory of old tasks. This method not only reduces the consumption of computing resources, but also improves the model's ability to balance new and old tasks. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for identifying water environment targets based on the SAM model in order to solve at least one of the above technical problems.

[0006] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0007] A method for identifying water environment targets based on a SAM model comprises the following steps:

[0008] A series of Adapter modules are embedded in specific positions of the original SAM model to generate a W-SAM model for the water environment;

[0009] Acquire a water environment picture, and generate a water environment data set according to the water environment picture;

[0010] According to the water environment data set, the W-SAM model is trained, tested and adjusted to generate a target W-SAM model;

[0011] The target W-SAM model is used to identify targets in water environments.

[0012] Furthermore, a series of Adapter modules are embedded in specific positions of the original SAM model, including: embedding an Adapter module in the image encoder of the original SAM model, specifically,

[0013] Embed Adapter modules between ViT submodules to inject specific information;

[0014] Embed the Adapter module inside the ViT submodule.

[0015] Furthermore, the Adapter module is embedded in the ViT submodule, specifically:

[0016] The first Adapter module is embedded after the multi-head attention module and before the residual module;

[0017] Embed the second Adapter module on the residual path of the MLP layer after the multi-head attention module;

[0018] After the second Adapter module, a scaling factor is introduced to scale the adaptation information.

[0019] Furthermore, the specific information is: image block embedding and fusion of image high-frequency information.

[0020] Furthermore, a series of Adapter modules are embedded in specific positions of the original SAM model, including: embedding an Adapter module in the image decoder of the original SAM model, specifically,

[0021] The first Adapter module is embedded after the multi-head attention from the cue information to the image embedding, and the residual of the cue embedding is added;

[0022] Embed the second Adapter module on the residual path of the MLP layer after the multi-head attention module;

[0023] The third Adapter module is embedded after the residual connection of the image embedding.

[0024] Furthermore, the water environment picture includes: a real water surface picture taken by visible light and a target picture taken by infrared light.

[0025] Furthermore, the water environment dataset includes a training set, a validation set and a test set in a ratio of 6:2:2.

[0026] Furthermore, the W-SAM model is trained and tested and adjusted, including: freezing and segmenting all pre-trained parameters of the SAM model, and adjusting the parameters of the Adapter module.

[0027] Furthermore, the W-SAM model was trained and tested, including: converting grayscale images to RGB images and applying random flipping and color jittering as data enhancement; the batch size was set to 4 and the initial learning rate was set to 2×10 -4 , using an adaptive moment estimation optimizer with cosine decay learning rate and warmup training; training for 80 epochs on the training set.

[0028] A water environment target recognition system based on the SAM model, comprising:

[0029] Model building module, used to embed a series of Adapter modules in specific positions of the original SAM model to generate a W-SAM model for the water environment;

[0030] A data acquisition module, used to acquire a water environment picture and generate a water environment data set according to the water environment picture;

[0031] A model training module is used to train, test and adjust the W-SAM model according to the water environment data set to generate a target W-SAM model;

[0032] The recognition module uses the target W-SAM model to identify targets in the water environment.

[0033] The beneficial effects of the present invention are:

[0034] The present invention selects the Adapter method and embeds the Adapter to perform efficient fine-tuning on the basis of the Segment Anything Model (SAM), so that the SAM model can not only retain the original massive and rich knowledge, but also effectively adapt to the target image of the water environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flow chart of a method for identifying a water environment target based on a SAM model according to an embodiment of the present invention;

[0036] Figure 2 It is a classic Transformer module;

[0037] Figure 3 An encoder module for W-SAM according to an embodiment of the present invention;

[0038] Figure 4 A decoder module for W-SAM according to an embodiment of the present invention;

[0039] Figure 5 The overall architecture of W-SAM in one embodiment of the present invention;

[0040] Figure 6 This is an example of a water surface visible light image segmentation task. Figure 1 ;

[0041] Figure 7 This is an example of a water surface visible light image segmentation task. Figure 2 ;

[0042] Figure 8 This is an example of a water surface visible light image segmentation task. Figure 3 ;

[0043] Fig. 9 For infrared image segmentation task example Figure 1 ;

[0044] Fig.10 For infrared image segmentation task example Figure 2 ;

[0045] Fig.11 This is a structural diagram of a water environment target recognition system based on the SAM model in one embodiment of the present invention. DETAILED DESCRIPTION

[0046] The content of the present invention will now be discussed with reference to exemplary embodiments. It should be understood that the embodiments discussed are only to enable those skilled in the art to better understand and thus implement the content of the present invention, rather than implying any limitation on the scope of the present invention.

[0047] As used herein, the term “including” and variations thereof are to be interpreted as open-ended terms meaning “including, but not limited to.” The term “based on” is to be interpreted as “based, at least in part, on.” The terms “one embodiment” and “an embodiment” are to be interpreted as “at least one embodiment.”

[0048] The Segment Anything Model (SAM) has been trained on a very large segmentation dataset, including 11 million images and 1 billion masks. SAM has good zero-sample generalization ability in many segmentation tasks and has the ability to segment any target. However, some studies have shown that the performance of SAM may be significantly degraded in certain scenarios (especially in images with irregular shapes, blurred boundaries, small sizes, and low contrast in sensors with different imaging modes), and is attributed to the lack of similar image samples during SAM training. The present invention selects the Adapter method, and embeds the Adapter on the basis of the SAM model for efficient fine-tuning, so that the model can retain the original massive and rich knowledge and effectively adapt to the target images of the aquatic environment.

[0049] Adapter is a bottleneck-structured MLP that uses three neural network layers, downsampling projection, GeLU, and upsampling projection. Specifically, downsampling uses a simple MLP layer to compress a given feature code into a smaller dimension. Upsampling projection uses another MLP layer to restore the compressed feature code to a high-dimensional space and keep it consistent with the original dimension. GeLU is a nonlinear activation function used to produce nonlinear transformations. The input of Adapter can come from the output of the previous neural network layer (Adapter inside the ViT submodule) or the injection of new information (Adapter between ViT submodules). The injection of new information is flexible and diverse, and can be texture, high-frequency information, or manual features.

[0050] Embodiment 1

[0051] Figure 1 A flow chart of a method for identifying a water environment target based on a SAM model according to an embodiment of the present invention; Figure 2 It is a classic Transformer module; Figure 3 An encoder module for W-SAM according to an embodiment of the present invention; Figure 4 A decoder module for W-SAM according to an embodiment of the present invention; Figure 5 FIG. 1 is the overall architecture of W-SAM according to an embodiment of the present invention. Figure 1-5 As shown, according to one embodiment of the present invention, a method for identifying a water environment target based on a SAM model comprises the following steps:

[0052] Step S102: embedding a series of Adapter modules at specific positions of the original SAM model to generate a W-SAM model for the water environment;

[0053] Image segmentation is the process of dividing an image into multiple regions, each corresponding to a different object. Compared with the bounding box or type provided by detection and classification, segmentation can generate a fine-grained description of the target. However, for different physical sensors, such as infrared cameras, the image segmentation task becomes difficult on degraded images due to the high noise, low resolution and complex object shapes of the images.

[0054] Intelligent unmanned platform information perception photoelectric sensors include visible light and infrared sensors. Visible light sensors perform imaging by receiving visible light reflected by objects, while infrared sensors perform imaging by detecting thermal radiation and background radiation emitted by the objects themselves. Due to the large inter-domain differences in image data from different sensors, the usual practice is to train multiple detection and classification models for image data of different modalities and types. However, the main problem with this approach is that it is relatively inefficient and it is difficult to reuse the trained models when the scene or target type changes.

[0055] W-SAM (Water area-SAM) is a method for efficiently fine-tuning large SAM models with limited resources. The core idea is to use low-rank approximation to reduce the dimension of the weight matrix, thereby reducing the scale of trainable parameters in the model to solve the problem that fine-tuning large models requires a lot of computing resources.

[0056] Step S104: obtaining a water environment picture, and generating a water environment data set according to the water environment picture;

[0057] Visible light and infrared images are relatively easy to obtain, and water environment images include visible light images and infrared images. In this embodiment, the visible light data set is real water surface images, covering a variety of typical similar targets in water surface scenes, such as ships, buoys, rail barrages and other targets, totaling 1,800 images, and the infrared data set contains ships, buoys and other targets, totaling 1,200 images.

[0058] Step S106: training, testing and adjusting the W-SAM model according to the water environment data set to generate a target W-SAM model;

[0059] In this embodiment, during the training of the W-SAM model, all pre-trained parameters of the segmentation model SAM are frozen, and the parameters of the Adapter module are adjusted. During the training process, the image encoder parameters of the original SAM model are not trained, and only the Adapter module is adjusted.

[0060] Step S108: Use the target W-SAM model to identify the target in the water environment.

[0061] The trained W-SAM model is used to identify targets in water environments. The target W-SAM model has a better segmentation effect for water environments and can achieve better target recognition results.

[0062] The present invention embeds the Adapter module in the SAM model to construct a W-SAM model for aquatic environments, achieving the purpose of efficiently fine-tuning a large model under limited resources, and effectively solving the problems of low cost-effectiveness and poor model reusability caused by the large differences between image data domains and the need to train multiple models in traditional methods. Through training and testing of specific data sets, the generated W-SAM model of the present invention shows better target recognition and image segmentation effects in aquatic environments, improving the accuracy and efficiency of intelligent unmanned platforms in perceiving aquatic environment information.

[0063] According to an embodiment of the present invention, step S102 includes: embedding an Adapter module in the image encoder of the original SAM model, specifically,

[0064] Embed Adapter modules between ViT submodules to inject specific information;

[0065] Embed the Adapter module inside the ViT submodule.

[0066] Preferably, an Adapter module is embedded in the ViT submodule, specifically:

[0067] The first Adapter module is embedded after the multi-head attention module and before the residual module;

[0068] Embed the second Adapter module on the residual path of the MLP layer after the multi-head attention module;

[0069] After the second Adapter module, a scaling factor is introduced to scale the adaptation information.

[0070] Step S102 also includes: embedding an Adapter module in the image decoder of the original SAM model, specifically,

[0071] Three Adapters are deployed in each Transformer submodule;

[0072] The first Adapter module is embedded after the multi-head attention from the cue information to the image embedding, and the residual of the cue embedding is added;

[0073] Embed the second Adapter module on the residual path of the MLP layer after the multi-head attention module;

[0074] The third Adapter module is embedded after the residual connection of the image embedding.

[0075] In this implementation, W-SAM first freezes the weights of the pre-trained model and adds a series of Adapter modules to the Transformer module. In the W-SAM encoder, the Adapter is embedded between and within the ViT submodules. The Adapter modules embedded between the ViT submodules implement the injection of specific information.

[0076] The Adapter embedded in the W-SAM encoder Transformer submodule contains two specific Adapters. Specifically, the first Adapter is placed after the multi-head attention and before the residual connection. The second Adapter is placed on the residual path of the MLP layer after the multi-head attention. After the second Adapter, a scaling factor s is introduced to scale the adaptation information.

[0077] In the W-SAM decoder, each Transformer submodule deploys three Adapters (such as Figure 4 ). The first Adapter is deployed after the multi-head attention from the cue information to the image embedding and incorporates the residual of the cue embedding. The Adapter is adjusted to integrate the cue information into the module. Specifically, another downsampled projection is used to compress the cue embedding and added to the Adapter's embedding before the GeLU activation. This modification helps the Adapter transform parameters based on the cue information and is more flexible and versatile for different modes and downstream tasks. The second Adapter in the decoder is deployed in exactly the same way as in the encoder to accommodate the MLP-enhanced embedding. The third Adapter is deployed after the residual connection of the image embedding to facilitate the cross-attention mechanism. Another residual connection and layer normalization are connected after the Adapter to output the final result. It should be emphasized that W-SAM only deploys the Adapter in the first of the two modules of the decoder. The second module and the mask prediction head are completely open and can be fully fine-tuned based on the given data.

[0078] The SAM architecture is adjusted to improve its recognition ability for water area target images. Different from the comprehensive fine-tuning strategy, W-SAM (Water area-SAM) freezes all pre-trained parameters of SAM and inserts a series of Adapter modules ( Figure 5 In the W-SAM encoder, Adapter is embedded between ViT submodules ( Figure 5 2 Adapters in the middle and lower part) and inside the ViT submodule ( Figure 3 ). The adapter modules embedded in the ViT submodules realize the injection of specific information. In the W-SAM encoder, the specific “specific information” is selected by image block embedding and the fusion of image high-frequency information.

[0079] According to an embodiment of the present invention, the water environment picture includes: a real water surface picture taken with visible light and a target picture taken with infrared light.

[0080] The water environment dataset includes a training set, a validation set and a test set in a ratio of 6:2:2.

[0081] In this implementation, visible light and infrared images are relatively easy to obtain. The visible light data set is a real water surface image, covering a variety of typical similar targets in water surface scenes, such as ships, buoys, rail forts and other targets, totaling 1,800 images. The infrared data set contains ships, buoys and other targets, totaling 1,200 images. The data set is randomly divided into training set, validation set and test set in a ratio of 6:2:2. The training set provides sufficient learning samples during the model training process; the validation set regularly evaluates the performance of the model to help developers understand the training effect of the model at different stages, and adjust and optimize the model accordingly to ensure that the model will not be overfitted or underfitted during the training process; the test set objectively evaluates the final performance of the model after the model training is completed, providing a reliable basis for the deployment and application of the model.

[0082] The present invention constructs a water environment dataset containing visible light and infrared images and rationally allocates it into training sets, validation sets, and test sets, thereby effectively improving the target recognition capability and robustness of the model in aquatic environments and ensuring the scientificity and accuracy of model training and evaluation.

[0083] According to one embodiment of the present invention, step S106 includes: converting the grayscale image into an RGB image and applying random flipping and color jittering as data enhancement; setting the batch size to 4 and the initial learning rate to 2×10 -4 , using an adaptive moment estimation optimizer with cosine decay learning rate and warmup training; training for 80 epochs on the training set.

[0084] To be consistent with the input of SAM, the grayscale images are converted to RGB images in this implementation, and random flipping and color jittering are introduced as data enhancement. The batch size is set to 4, and an adaptive moment estimation optimizer is used. The initial learning rate of the network is set to 2×10-4, and techniques such as cosine decay learning rate and warm-up training are used in training. All models are trained for 80 rounds on the training set.

[0085] In order to verify the recognition effect, the target W-SAM model generated by the present invention is compared with the SAM model under the same conditions to evaluate the performance of both on visible light and infrared images.

[0086] The loss function uses the cross entropy loss LCE and the Dice loss function LD after Laplace smoothing to supervise the training of the segmentation task. The cross entropy loss function is defined as

[0087]

[0088] In the formula, N represents the number of all pixels in the image; C represents the number of classes; y ij is the true value label, if the sample belongs to this category, it is 1, if not, it is 0; p ij is the probability that the i-th sample belongs to the j-th class.

[0089] The Dice similarity coefficient is usually used to calculate the similarity between two sets, and its value range is [0,1].

[0090]

[0091] In the formula, |X∩Y| represents the common elements of the two sets X and Y, |X| represents the number of elements in the X set, and |Y| represents the number of elements in the Y set.

[0092] Therefore, the Dice loss function can be expressed as

[0093]

[0094] In the formula, X and B represent the masks of the prediction and true value respectively. At the same time, in order to prevent the size of X and Y sets from being 0, resulting in the denominator being 0, after Laplace smoothing, the final Dice loss function like the encoder becomes

[0095]

[0096] Joint loss function L j It is equal to the sum of the cross entropy loss and the Dice loss after Laplace smoothing, which is

[0097] L j =LCE +L D (5)

[0098] The Dice similarity coefficient and pixel accuracy PA are used to evaluate the segmentation accuracy. The definition of Dice is shown in formula (2). Dice is positively correlated with IoU and can be further expressed as

[0099]

[0100] In the formula, TP represents the number of samples whose true value is positive and correctly predicted; FP represents the number of samples whose true value is negative but predicted to be positive; FN represents the number of samples whose true value is negative and correctly predicted.

[0101] The performance of the target W-SAM is evaluated from two aspects: visible light image segmentation effect and infrared image segmentation effect.

[0102] Table 1 Comparison of visible light image segmentation performance of SAM and target W-SAM

[0103]

[0104] Table 1 is a comparison table of the visible light image segmentation performance of SAM and the target W-SAM. Table 1 shows the changes in the model's visible light image segmentation performance before and after using W-SAM, including: (1) using point prompts to provide location information, that is, selecting the target at the center of the image for prompting, represented by SAM-Point; (2) using box prompts to provide location information, that is, selecting the target area on the image for prompting, represented by SAM-Box; (3) after fine-tuning the original SAM image encoder, using box prompts to provide location information, represented by W-SAM.

[0105] As shown in Table 1, the performance of the fine-tuned SAM (i.e., W-SAM) is the best. For three types of targets, namely ships, buoys, and rail forts, the average Dice similarity coefficient of W-SAM is 3.57% and 6.73% higher than that of SAM-Box and SAM-Point, respectively; the average value of PA is 4.0% and 7.45% higher than that of SAM-Box and SAM-Point, respectively. At the same time, the performance of SAM-Box is better than that of SAM-Point, which is consistent with previous studies.

[0106] In addition, we can see that the lowest Dice average value among the three settings is 86.79%, which shows that SAM can be directly applied to the visible light image object segmentation task without additional training and fine-tuning. Considering that the SAM pre-training dataset is a large number of natural light images, the image quality and resolution are very close to those of visible light images. Therefore, it works better on the visible light image segmentation task. Figure 6This is an example of a water surface visible light image segmentation task. Figure 1 , Figure 7 This is an example of a water surface visible light image segmentation task. Figure 2 , Figure 8 This is an example of a water surface visible light image segmentation task. Figure 3 . Figure 6 The target is identified as a ship in the segmentation. Figure 7 The target is segmented and identified as a buoy. Figure 8 The target identified in the middle segmentation is the railroad fortress.

[0107] Table 2 Comparison of infrared image segmentation performance of SAM and target W-SAM

[0108]

[0109]

[0110] Table 2 is a comparison table of the infrared image segmentation performance of SAM and target W-SAM; Table 2 shows the segmentation performance of SAM for infrared target images before and after fine-tuning. As can be seen from Table 2, for infrared images, the performance of SAM has slightly decreased, but the overall trend remains consistent with that of visible light images, and the performance of SAM after fine-tuning is still the best. For ship targets, the average Dice similarity coefficient of W-SAM is 6.26% and 12.49% higher than that of SAM-Box and SAM-Point respectively; the PA average is 5.93% and 12.53% higher than that of SAM-Box and SAM-Point respectively.

[0111] Compared with visible light images, the performance of infrared images using the fine-tuned SAM is more significantly improved. This is because, in general, the resolution of infrared images is lower than that of visible light images, and due to the difference in imaging systems, the target boundaries of infrared images are not as clear as those of visible light images, which causes a certain degree of performance degradation when using the original SAM for image segmentation. For this reason, SAM was fine-tuned for marine infrared data, and the performance was significantly improved. Fig. 9 For infrared image segmentation task example Figure 1 , Fig.10 For infrared image segmentation task example Figure 2 . Fig. 9 The target is identified as a ship in the segmentation. Fig.10 The target identified in the middle segmentation is a buoy.

[0112] It can be seen that W-SAM achieves a better segmentation effect. In addition, since the infrared features of the ship are richer and more clearly distinguished from the background, its Dice similarity coefficient is higher than that of the buoy.

[0113] Using a unified model to implement detection and segmentation tasks in different scenes and different modalities is an important issue in the field of machine vision. The general segmentation model SAM has a strong zero-sample generalization ability and can segment any type of image, providing a new framework for solving the general target segmentation problem. However, due to the obvious domain differences between the visible light image dataset for training SAM and the images acquired by the infrared sensor, there is a large room for improvement in the effect of directly using SAM on these images. To this end, the present invention has significantly improved the model performance of the proposed W-SAM method by fine-tuning the encoder of SAM. The research results provide an important reference for the application of SAM in image segmentation tasks.

[0114] The present invention generates a W-SAM model by fine-tuning the SAM model, which significantly improves the recognition and segmentation performance of water environment targets in visible light and infrared images. In particular, the performance improvement of the fine-tuned model is more significant in infrared images, which verifies the effectiveness and superiority of this method in solving image segmentation tasks in different scenarios and different modalities.

[0115] Embodiment 2

[0116] Fig.11 FIG. 1 is a structural diagram of a water environment target recognition system based on a SAM model according to an embodiment of the present invention. Fig.11 As shown, according to one embodiment of the present invention, a water environment target recognition system based on the SAM model includes:

[0117] A model building module 10 is used to embed a series of Adapter modules at specific positions of the original SAM model to generate a W-SAM model for a water environment;

[0118] The data acquisition module 20 is used to acquire a water environment picture and generate a water environment data set according to the water environment picture;

[0119] The model training module 30 is used to train, test and adjust the W-SAM model according to the water environment data set to generate a target W-SAM model;

[0120] The recognition module 40 uses the target W-SAM model to recognize the target in the water environment.

[0121] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0122] The present invention realizes efficient recognition of water environment targets by constructing a W-SAM system based on the SAM model and combining data acquisition, model training and recognition modules, thereby significantly improving the accuracy and efficiency of target recognition.

[0123] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the present application.

[0124] It should be understood that the size of the serial numbers of each step in the content of the invention and the embodiments of the present invention does not absolutely mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

Claims

1. A method for identifying water environment targets based on the SAM model, characterized in that: The following steps are involved: A series of Adapter modules are embedded in specific positions of the original SAM model to generate a W-SAM model for the water environment; Acquire a water environment picture, and generate a water environment data set according to the water environment picture; According to the water environment data set, the W-SAM model is trained, tested and adjusted to generate a target W-SAM model; The target W-SAM model is used to identify targets in water environments.

2. The water environment target recognition method based on the SAM model according to claim 1 is characterized in that: Embed a series of Adapter modules in specific positions of the original SAM model, including: embedding an Adapter module in the image encoder of the original SAM model, specifically, Embed Adapter modules between ViT submodules to inject specific information; Embed the Adapter module inside the ViT submodule.

3. The water environment target recognition method based on the SAM model according to claim 2 is characterized in that: Embed the Adapter module inside the ViT submodule, specifically: The first Adapter module is embedded after the multi-head attention module and before the residual module; Embed the second Adapter module on the residual path of the MLP layer after the multi-head attention module; After the second Adapter module, a scaling factor is introduced to scale the adaptation information.

4. The water environment target recognition method based on the SAM model according to claim 2 is characterized in that: The specific information is: image block embedding and fusion of image high-frequency information.

5. The water environment target recognition method based on the SAM model according to claim 1 is characterized in that: Embed a series of Adapter modules in specific positions of the original SAM model, including: embedding Adapter modules in the image decoder of the original SAM model, specifically, The first Adapter module is embedded after the multi-head attention from the cue information to the image embedding, and the residual of the cue embedding is added; Embed the second Adapter module on the residual path of the MLP layer after the multi-head attention module; The third Adapter module is embedded after the residual connection of the image embedding.

6. The water environment target recognition method based on the SAM model according to claim 1 is characterized in that: The water environment pictures include: real water surface pictures taken with visible light and target pictures taken with infrared light.

7. The method for identifying water environment targets based on the SAM model according to claim 6 is characterized in that: The water environment dataset includes a training set, a validation set and a test set in a ratio of 6:2:

2.

8. The water environment target recognition method based on the SAM model according to claim 1 is characterized in that: The W-SAM model is trained and tested and adjusted, including: freezing all pre-trained parameters of the segmentation model SAM and adjusting the parameters of the Adapter module.

9. The water environment target recognition method based on the SAM model according to claim 1 is characterized in that: The W-SAM model was trained and tested with adjustments, including: converting grayscale images to RGB images and applying random flipping and color jittering as data augmentation; the batch size was set to 4 and the initial learning rate was set to 2×10 -4 , using an adaptive moment estimation optimizer with cosine decay learning rate and warmup training; training for 80 epochs on the training set.

10. A water environment target recognition system based on the SAM model, characterized in that: include: Model building module, used to embed a series of Adapter modules in specific positions of the original SAM model to generate a W-SAM model for the water environment; A data acquisition module, used to acquire a water environment picture and generate a water environment data set according to the water environment picture; A model training module is used to train, test and adjust the W-SAM model according to the water environment data set to generate a target W-SAM model; The recognition module uses the target W-SAM model to identify targets in the water environment.

Images