An underwater target detection method based on visual-linguistic model knowledge distillation

Abstract

The application discloses an underwater target detection method based on visual-language model knowledge distillation. With the continuous maturity of visual-language large models, their powerful visual feature extraction capability and text alignment capability have been widely applied in the field of deep learning. The method first collects and fuses multiple public underwater datasets, and cleans and preprocesses the data to construct a rich and high-quality training dataset. Secondly, a visual-language model is introduced. In the large model fine-tuning stage, feature extraction and fine-grained fine-tuning are performed through image and text alignment to effectively capture the correlation between images and texts, thereby generating more accurate feature representations. Finally, in the knowledge distillation stage, the method uses a teacher network and a student network for feature migration. The teacher network obtains fine features from the fine-tuned backbone network, and transfers these knowledge to the student network through a mean square error loss function, thereby improving the generalization ability and detection performance of the student network, and finally realizing robust detection of underwater targets. The application fully utilizes the visual-language model and knowledge distillation technology, enhances the underwater target features, enables the model to learn robust target features, and improves the robustness of detection.

Description

Technical Field

[0001] This invention relates to an underwater target detection method based on knowledge distillation of a vision-language model, belonging to the field of computer applications and specifically to underwater target detection methods. Background Technology

[0002] Underwater target detection (UOD) has significant practical applications and potential scientific implications in many key fields, including marine science research, environmental monitoring and protection, and military security. The widespread use and deployment of cameras in Autonomous Underwater Vehicles (AUVs) and Remotely Operated Vehicles (ROVs) facilitates the acquisition of underwater datasets. Therefore, research on deep learning-based underwater target detection (UOD) is of great importance.

[0003] Compared to land-based detection scenarios, underwater optical image detection faces various challenges. Firstly, underwater target detection is challenged by the underwater environment. The first is the haze effect; the absorption and scattering of light by the medium and suspended particles leads to low contrast and a foggy appearance in underwater photography. Movement of image acquisition equipment (such as AUVs and UUVs) also contributes to blurring. Secondly, there is color cast. Water absorbs and reflects light primarily based on wavelength. With increasing depth, longer wavelengths attenuate faster, while shorter wavelengths attenuate more slowly. Underwater images often appear bluish or greenish because shorter wavelengths of blue and green light are absorbed less than other colors. These factors contribute to problems such as unclear target features in underwater scenes. On the other hand, the significant difficulties in collecting and annotating underwater images make it challenging to integrate large underwater datasets with rich domain diversity. Therefore, conducting research on underwater target detection for underwater-related visual tasks to better serve application scenarios is a topic worthy of in-depth study.

[0004] Most common underwater target detection methods are based on conventional target detection models and have not yet effectively utilized large underwater datasets and multimodal large models. For example, application publication number CN115984681A (An underwater target detection method based on YOLOv5) only uses a target detection model; CN116612379A (An underwater target detection method and system based on multi-knowledge distillation) only introduces the conventional distillation method into underwater target detection, which solves the problem of model complexity to some extent, but lacks improvements for the problems of limited underwater data and complex scenarios. Summary of the Invention

[0005] Underwater target detection suffers from a scarcity of datasets due to challenges in annotation and scene complexity. Existing target detection models are designed for terrestrial environments and therefore perform poorly in underwater environments, requiring targeted adjustments and optimizations. With the continuous development of multimodal large language models, multimodal pre-training is increasingly being applied to downstream tasks. The CLIP model, pre-trained on a dataset of 400 million image-text pairs, has demonstrated excellent performance on various downstream tasks, indicating that large models possess superior visual extraction capabilities. Therefore, the purpose of this invention is to address the shortcomings described in the background by providing an underwater target detection method based on knowledge distillation of a vision-language model. Specifically, firstly, a large dataset of existing underwater data needs to be collected to facilitate fine-tuning of the vision-language model, making the model more focused on underwater target features. After collecting and standardizing the data format, fine-tuning is performed on the CLIP model. Then, a custom target detection model is trained on the underwater dataset, utilizing feature knowledge distillation, with the vision-language model acting as the teacher network and the target detection model as the student network. Under the guidance of the teacher network, the student network extracts features that are as close as possible to those extracted by the teacher network. Finally, underwater feature enhancement is achieved by using knowledge distillation from a visual-language model to compensate for the lack of clear features caused by insufficient underwater data.

[0006] To achieve the above objectives, the present invention is implemented using the following technical solutions.

[0007] An underwater target detection method based on knowledge distillation of a vision-language model includes the following steps:

[0008] Step 1, Data Collection and Fusion: Collect multiple publicly available underwater datasets, including but not limited to underwater target detection, classification, and segmentation datasets, and fuse them to construct a rich and high-quality training dataset;

[0009] Step 2, Data Cleaning and Preprocessing: Clean and preprocess the collected underwater data, including label alignment, format standardization, removal of duplicate images and annotation format conversion, to ensure data consistency and quality;

[0010] Step 3, Fine-tuning of the large model: Introduce a visual-language model, use the CLIP method to align images and text, improve feature extraction capabilities through the correlation between vision and language, fine-tune the introduced visual-language large model, extract features through image and text alignment, and generate accurate feature representations;

[0011] Step 4, Teacher Network Construction: Construct the teacher network Model_T, and extract fine-grained feature representations from it using the finely tuned backbone network of the large model;

[0012] Step 5, Knowledge Distillation Strategy: Through the Mean Squared Error (MSE) loss function, the fine features extracted from the teacher network are transferred to the student network for feature transfer.

[0013] Step 6, Student Network Training: Train the student network Model_S by using knowledge distillation to enable it to learn the fine features of the teacher network, thereby improving the generalization ability and detection performance of the student network and ultimately achieving robust detection of underwater targets.

[0014] As a preferred embodiment of the present invention, the specific process of step 1 is as follows:

[0015] 1.1 Obtain the training set D from the publicly available underwater segmentation, classification, and object detection dataset D. train In addition, the corresponding labels for the training set are used to ensure that the data sources are diverse, covering different underwater environments, target types and conditions, so as to increase the diversity and representativeness of the dataset;

[0016] 1.2 For training set D train Perform simple data cleaning and standardize image and label names.

[0017] As a preferred embodiment of the present invention, the specific process of step 2 is as follows:

[0018] 2.1 Tag alignment and formatting consistency:

[0019] Ensure consistent label names and formats across different datasets. Standardize target category labels across all datasets to a unified naming and encoding method to avoid data confusion caused by inconsistent labels. Convert all images and annotation files from all datasets to a unified format. Convert all images to the same resolution and file format (e.g., JPEG), and convert annotation files to unified text descriptions to accommodate visual-text bimodal fine-tuning.

[0020] 2.2 Removing duplicate images:

[0021] Image deduplication algorithms are used to detect and remove duplicate images, ensuring that each image appears only once in the dataset to avoid data redundancy and bias during model training.

[0022] 2.3 Text description annotation generation:

[0023] (1) Generate text description for bounding box:

[0024] Bounding box text description generation leverages bounding box annotations and labels from object detection datasets to generate text descriptions. For object detection data, each image has a corresponding annotation recording the bounding box of each object. A rule-based approach is used to generate five distinct texts to describe the objects in the images. Specifically, the first two texts describe the objects in the image center and surrounding areas, respectively: the first text describes objects in the image center, defining the center area based on the image size, indicating which objects are in the middle of the image; while the second text describes objects not in the center, indicating which objects are around the image. This distinction provides additional background and information about the spatial distribution of objects in the image. The remaining three texts randomly and specifically describe a particular object in the image, such as its quantity. If an object appears more than ten times, a more general term (e.g., "many") is used instead of an exact number to enhance the readability and variability of the text descriptions.

[0025] (2) Mask to bounding box conversion:

[0026] For the segmentation dataset, each image is labeled as a mask. Converting segmentation labels to bounding box labels is a key step in integrating the segmentation dataset into the bounding box-generated text description. To achieve this conversion, the segmentation mask is processed by category, encoding each pixel label to correspond to the target class. Next, the contour points of the connected regions of each class in the mask image are identified. Specifically, we use a boundary tracking algorithm to extract the contours of the mask. Subsequently, we apply the boundary tracking algorithm to extract the topology of the connected components within the binary mask. By sorting the outer boundary contour points of each connected component, the minimum and maximum values ​​on the horizontal and vertical axes are used as the coordinates of the horizontal bounding box of each connected component. Finally, all semantic segmentation annotations are converted from mask to bounding box to bounding box annotations, and then step (1) is performed to obtain the corresponding text description.

[0027] As a preferred embodiment of the present invention, the specific process of step 3 is as follows:

[0028] 3.1 First, the training set D is obtained from steps 1.1 and 1.2. train The corresponding image-text pairs contained therein, where the image is represented as {I1, I2, ..., I... n}, the corresponding text labels are represented as {T1, T2, ..., T}. n};

[0029] 3.2 Then, the text corresponding to each image is input into a vision-language model (e.g., the CLIP model) for large-scale model fine-tuning, image I i Image features F(I) are generated by an image encoder. i ), text label T iText features F(T) are generated using a text encoder. i );

[0030] 3.3 Next, contrastive learning is used to train and the InfoNCE contrastive loss function is used to make the features of matched image-text pairs more similar and the features of unmatched image-text pairs less similar.

[0031]

[0032] sim represents a similarity function (such as cosine similarity), which is a temperature parameter;

[0033] 3.4 Finally, backpropagation and parameter updates are performed. After calculating the parameters, the Adam optimization algorithm is used to update the model parameters. This process is repeated to gradually optimize the model parameters. The model performance is evaluated after each iteration, and the optimal model parameters are selected.

[0034] As a preferred embodiment of the present invention, the specific process of step 5 is as follows:

[0035] 5.1 Initialize Teacher Network Parameters. Randomly initialize the teacher network Model_T, using the training set D obtained after step 2. train’ ;

[0036] 5.2 Calculate the InfoNCE contrastive loss function on the training set D. train' Train the teacher network Model_T until the model converges;

[0037] 5.3 Initialize Student Network Parameters. Perform random initialization on the student network; obtain the characteristics of the fine-tuned teacher and student networks.

[0038] 5.4 Extracting collective knowledge from the teacher network; calculating the distillation loss function (MSE):

[0039]

[0040] As a preferred embodiment of the present invention, the specific process of step 7 is as follows:

[0041] 6.1 Train the student network. Calculate the coordinate error loss L. box Target confidence loss L obj And classification loss L cls These three loss functions are used until the model converges;

[0042] 6.2 Calculate the detection loss L det =αL box +βL obj +γL cls .

[0043]

[0044] Among them, box, box gt ρ represents the center point of the predicted bounding box and the center point of the ground truth bounding box. 2 α represents the Euclidean distance, c represents the diagonal length of the smallest rectangle containing the two boxes, IoU is the intersection-union ratio of the ground truth box and the predicted box, α is the weight function, and v is used to measure the similarity of aspect ratio.

[0045] L obj =-(y log(p)+(1-y)log(1-p))

[0046] Where y is the true label (1 indicates the target exists, 0 indicates the target does not exist), and p is the confidence score of the prediction.

[0047]

[0048] Where C is the number of categories, y i It's a real tag, p i It is the predicted class probability.

[0049] 6.3 Using L det With distillation loss L distill The student target detection network is trained using the sum of its components until the model converges. This results in the final underwater target detection network model.

[0050] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0051] 1. This invention is an underwater target detection method based on knowledge distillation of a vision-language model. By collecting and cleaning public datasets, it solves the problems of label alignment, duplicate image cleaning, file format unification, and annotation format conversion, thereby improving data quality, reducing data noise, and providing higher quality training data for subsequent model training.

[0052] 2. This invention presents an underwater target detection method based on knowledge distillation using a visual-language model. It utilizes a large-scale pre-trained model for fine-tuning, extracting effective features by inputting images from different datasets into the pre-trained model, thereby improving the model's adaptability and generalization ability to different types of data. Furthermore, the use of knowledge distillation technology enables the student network to better learn and imitate the feature representations of the teacher network, improving the model's detection performance and feature representation ability. Attached Figure Description

[0053] Figure 1 This is a flowchart of the underwater target detection method based on knowledge distillation of a vision-language model according to the present invention;

[0054] Figure 2This is a structural diagram of the underwater target detection method based on knowledge distillation of a vision-language model according to the present invention;

[0055] Figure 3 This is a diagram showing the detection effect of underwater target detection according to the present invention. Detailed Implementation

[0056] The following will be combined with the appendix Figures 1-3 The method of the present invention will be described in detail with reference to specific embodiments.

[0057] It is worth noting that embodiments of the present invention are described in detail below, with examples of these embodiments shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0058] With the continuous development of multimodal large language models, multimodal pre-training is increasingly being applied to downstream tasks. The CLIP model, pre-trained on a dataset of 400 million image-text pairs, has demonstrated excellent performance on various downstream tasks, indicating that large models possess superior visual extraction capabilities. Therefore, the purpose of this invention is to address the shortcomings described in the background art by providing an underwater target detection method based on knowledge distillation from a vision-language model. Figure 1 As shown, this invention proposes an underwater target detection method based on knowledge distillation using a vision-language model, comprising the following steps:

[0059] Step 1, Data Collection and Fusion: Collect multiple publicly available underwater datasets, including but not limited to underwater target detection, classification, and segmentation datasets, and fuse them to construct a rich and high-quality training dataset;

[0060] 1.1 Obtain the training set D from the publicly available underwater segmentation, classification, and object detection dataset D. train In addition, the corresponding labels for the training set are used to ensure that the data sources are diverse, covering different underwater environments, target types and conditions, so as to increase the diversity and representativeness of the dataset;

[0061] 1.2 For training set D train Perform simple data cleaning and standardize image and label names.

[0062] Step 2, Data Cleaning and Preprocessing: Clean and preprocess the collected underwater data, including label alignment, format standardization, removal of duplicate images and annotation format conversion, to ensure data consistency and quality;

[0063] 2.1 Tag alignment and formatting consistency:

[0064] Ensure consistent label names and formats across different datasets. Standardize target category labels across all datasets to a unified naming and encoding method to avoid data confusion caused by inconsistent labels. Convert all images and annotation files from all datasets to a unified format. Convert all images to the same resolution and file format (e.g., JPEG), and convert annotation files to unified text descriptions to accommodate visual-text bimodal fine-tuning.

[0065] 2.2 Removing duplicate images:

[0066] Use an image deduplication algorithm to detect and remove duplicate images, ensuring that each image appears only once in the dataset, in order to avoid data redundancy and bias during model training.

[0067] 2.3 Text description annotation generation:

[0068] (1) Generate text description for bounding box:

[0069] Bounding box text description generation leverages bounding box annotations and labels from object detection datasets to generate text descriptions. For object detection data, each image has a corresponding annotation recording the bounding box of each object. A rule-based approach is used to generate five distinct texts to describe the objects in the images. Specifically, the first two texts describe the objects in the image center and surrounding areas, respectively: the first text describes objects in the image center, defining the center area based on the image size, indicating which objects are in the middle of the image; while the second text describes objects not in the center, indicating which objects are around the image. This distinction provides additional background and information about the spatial distribution of objects in the image. The remaining three texts randomly and specifically describe a particular object in the image, such as its quantity. If an object appears more than ten times, a more general term (e.g., "many") is used instead of an exact number to enhance the readability and variability of the text descriptions.

[0070] (2) Mask to bounding box conversion:

[0071] For the segmentation dataset, each image is labeled as a mask. Converting segmentation labels to bounding box labels is a key step in integrating the segmentation dataset into bounding box-generated text descriptions. To achieve this conversion, the segmentation mask is processed by category, and each pixel label is encoded to correspond to the target class. Next, the contour points of the connected regions of each class in the mask image are identified. Specifically, we use a boundary tracking algorithm to extract the contours of the mask. Subsequently, we apply the boundary tracking algorithm to extract the topology of the connected components within the binary mask. By sorting the outer boundary contour points of each connected component, the minimum and maximum values ​​on the horizontal and vertical axes are used as the coordinates of the horizontal bounding box of each connected component. Finally, all semantic segmentation annotations are converted from mask to bounding box to bounding box annotations, and then (1) is performed to obtain the corresponding text descriptions;

[0072] Example of generating a partial dataset, Figure 3 The image shown is generated into a corresponding JSON annotation file according to the rules in step 2, with the following format:

[0073]

[0074]

[0075] Step 3, Fine-tuning of the large model: Introduce a visual-language model, use the CLIP method to align images and text, improve feature extraction capabilities through the correlation between vision and language, fine-tune the introduced visual-language large model, extract features through image and text alignment, and generate accurate feature representations;

[0076] 3.1 First, the training set D is obtained from steps 1.1 and 1.2. train The corresponding image-text pairs contained therein, where the image is represented as {I1,I2,…,I... n The corresponding text labels are represented as {T1,T2,…,T}. n};

[0077] 3.2 Then, the text corresponding to each image is input into a vision-language model (e.g., the CLIP model) for large-scale model fine-tuning, image I i Image features F(I) are generated by an image encoder. i ), text label T i Text features F(T) are generated using a text encoder. i );

[0078] 3.3 Next, contrastive learning is used to train and the InfoNCE contrastive loss function is used to make the features of matched image-text pairs more similar and the features of unmatched image-text pairs less similar.

[0079]

[0080] sim represents a similarity function (such as cosine similarity), which is a temperature parameter;

[0081] 3.4 Finally, backpropagation and parameter updates are performed. After calculating the parameters, the Adam optimization algorithm is used to update the model parameters. This process is repeated to gradually optimize the model parameters. The model performance is evaluated after each iteration, and the optimal model parameters are selected.

[0082] Step 4, Teacher Network Construction: Construct the teacher network Model_T, and extract fine-grained feature representations from it using the finely tuned backbone network of the large model;

[0083] Step 5, Knowledge Distillation Strategy: Through the Mean Squared Error (MSE) loss function, the fine features extracted from the teacher network are transferred to the student network for feature transfer.

[0084] 5.1 Initialize teacher network parameters. Randomly initialize the teacher network Model_T and extract the training set D obtained in step 2. train’ ;

[0085] 5.2 Calculate the InfoNCE contrastive loss function on the training set D. train' Train the teacher network Model_T until the model converges;

[0086] 5.3 Initialize Student Network Parameters. Perform random initialization on the student network; obtain the characteristics of the fine-tuned teacher and student networks.

[0087] 5.4 Extracting collective knowledge from the teacher network; calculating the distillation loss function (MSE):

[0088]

[0089] Step 6, Student Network Training: Train the student network Model_S by using knowledge distillation to enable it to learn the fine features of the teacher network, thereby improving the generalization ability and detection performance of the student network and ultimately achieving robust detection of underwater targets.

[0090] 6.1 Train the student network. Calculate the coordinate error loss L. box Target confidence loss L obj And classification loss L cls These three loss functions are used until the model converges.

[0091] 6.2 Calculate the detection loss L det =αL box +βL obj +γL cls .

[0092]

[0093] Among them, box, box gt ρ represents the center point of the predicted bounding box and the center point of the ground truth bounding box. 2 α represents the Euclidean distance, c represents the diagonal length of the smallest rectangle containing the two boxes, IoU is the intersection-union ratio of the ground truth box and the predicted box, α is the weight function, and v is used to measure the similarity of aspect ratio.

[0094] L obj =-(y log(p)+(1-y)log(1-p))

[0095] Where y is the true label (1 indicates the target exists, 0 indicates the target does not exist), and p is the confidence score of the prediction.

[0096]

[0097] Where C is the number of categories, y i It's a real tag, p i It is the predicted class probability.

[0098] 6.3 Using L det With distillation loss L distill The student target detection network is trained using the sum of its components until the model converges. This results in the final underwater target detection network model.

[0099] This invention presents an underwater target detection method based on knowledge distillation using a vision-language model, combining data fusion, large model fine-tuning, and knowledge distillation techniques. This combination not only improves the model's accuracy and robustness but also reduces its computational complexity, making the invention more efficient and practical in real-world applications.

[0100] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solutions based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. An underwater target detection method based on knowledge distillation of a vision-language model, characterized in that, The steps include the following: Step 1, Data Collection and Fusion: Collect several publicly available underwater datasets, including underwater target detection, classification, and segmentation datasets, and fuse them to construct a rich and high-quality training dataset; Step 2, Data Cleaning and Preprocessing: Clean and preprocess the collected underwater data, including label alignment, format standardization, removal of duplicate images and annotation format conversion, to ensure data consistency and quality; The annotation format conversion in step 2 includes text description annotation generation, which includes: (1) Bounding box text description: For object detection data, text description is generated using the bounding box annotations and labels of the object detection dataset. A rule-based method is used to generate five different texts to describe the objects in the image. The first two texts describe what objects are in the center and around the image, respectively, and the other three texts randomly describe how many of a certain object in the image are there. (2) Mask to bounding box conversion: For the segmentation dataset, the label corresponding to each image is a mask. The contour of the mask is extracted using the boundary tracking algorithm. All semantic segmentation annotations are converted into bounding box annotations through the mask to bounding box conversion. Then, step (1) is executed to obtain the corresponding text description. Step 3, Fine-tuning of the large model: Introduce a visual-language model, use the CLIP method to align images and text, improve feature extraction capabilities through the correlation between vision and language, fine-tune the introduced visual-language large model, extract features through image and text alignment, and generate accurate feature representations; Step 4, Teacher Network Construction: Construct the teacher network Model_T, and extract fine-grained feature representations from it using the finely tuned backbone network of the large model; Step 5, Knowledge Distillation Strategy: Through the mean squared error loss function, the fine features extracted from the teacher network are transferred to the student network for feature transfer. Step 6, Student Network Training: Train the student network Model_S by using knowledge distillation to enable it to learn the fine features of the teacher network, thereby improving the generalization ability and detection performance of the student network and ultimately achieving robust detection of underwater targets. The large model fine-tuning process in step 3 includes adapting to specific tasks and datasets, and optimizing model parameters through multiple rounds of iteration to obtain the best feature representation. The specific process is as follows: First, steps 1 and 2 are used to obtain the corresponding image-text pairs contained in the training set Dtrain, where the image is represented as follows: }, the corresponding text label is represented as }; Then, the text corresponding to each image is input into the vision-language model for large-scale model fine-tuning. Image features are generated using an image encoder. Text tags Text features are generated using a text encoder. ; Next, contrastive learning is used to train the image and text pairs, and the InfoNCE contrastive loss function is applied to make the features of matching image-text pairs more similar and the features of non-matching image-text pairs less similar. The specific formula is as follows: ， sim represents the similarity function, and τ is the temperature parameter; Finally, backpropagation and parameter updates are performed. After calculating the parameters, the Adam optimization algorithm is used to update the model parameters. The above process is repeated to gradually optimize the model parameters. After each iteration, the model performance is evaluated and the best model parameters are selected.

2. The underwater target detection method based on knowledge distillation of a vision-language model according to claim 1, characterized in that, The dataset fusion in step 1 involves integrating image and annotation data from different datasets to construct a unified data format and standard. The specific process is as follows: The training set Dtrain and its corresponding labels are obtained from the publicly available underwater segmentation, classification, and target detection dataset D. The training set Dtrain covers different underwater environments, target types, and conditions. Perform simple data cleaning on the training set Dtrain, and standardize image names and label names.

3. The underwater target detection method based on knowledge distillation of a vision-language model according to claim 1, characterized in that, The cleaning and preprocessing operations in step 2 further include label alignment, format unification, removal of duplicate images, and annotation format conversion. The specific process is as follows: Tag alignment and formatting consistency: Standardize the target category labels in all datasets to a unified naming and encoding method, convert the images and annotation files of all datasets to a unified format, convert all images to the same resolution and file format, and convert the annotation files to a unified text description to adapt to visual-text bimodal fine-tuning; Remove duplicate images: Use an image deduplication algorithm to detect and remove duplicate images, ensuring that each image appears only once in the dataset; Text description annotation generation: (1) Generate text description for bounding box: For object detection data, each image has a corresponding annotation, which records the bounding box of each object. A rule-based method is used to generate five different texts to describe the objects in the image. The first two texts describe what objects are in the center and around the image, respectively. The first text describes the objects in the center of the image, defining the center range based on the image size, indicating which objects are in the middle of the image. The second text describes objects not in the center, indicating which objects are around the image. The remaining three texts will randomly and specifically describe how many of a certain object are in the image. If an object appears more than ten times, a more general term will be used instead of the exact number to enhance the readability and variability of the text descriptions. (2) Mask to bounding box conversion: For the segmentation dataset, the label corresponding to each image is a mask. First, the contour of the mask is extracted using the boundary tracking algorithm. Then, the topology of the connected components within the binary mask is extracted using the boundary tracking algorithm. By sorting the outer boundary contour points of each connected component, the minimum and maximum values ​​on the horizontal and vertical axes are used as the coordinates of the horizontal bounding box of each connected component. Finally, all semantic segmentation labels are converted from mask to bounding box to bounding box labels. Then, step (1) is executed to obtain the corresponding text description.

4. The underwater target detection method based on knowledge distillation of a vision-language model according to claim 1, characterized in that, In step 5, knowledge distillation calculates the feature differences between the teacher network and the student network using the mean squared error loss function, and gradually optimizes the student network to make its feature representation closer to that of the teacher network. The specific process is as follows: Initialize teacher network parameters, for the teacher network Perform random initialization and use the training set Dtrain' obtained after step 2; Calculate the InfoNCE contrastive loss function and train the teacher network on the training set Dtrain'. until the model converges; Initialize the student network parameters using random initialization; obtain the characteristics of the fine-tuned teacher and student networks. , ; Extracting collective knowledge from the teacher network; calculating the distillation loss function: 。 5. The underwater target detection method based on knowledge distillation of a vision-language model according to claim 4, characterized in that, The student online training process in step 6 is as follows: Train the student network and calculate the coordinate error loss. Target confidence loss and classification loss These three loss functions are used until the model converges; Calculate detection loss , ， in, This represents the center point of the predicted bounding box and the center point of the ground truth bounding box. This represents the Euclidean distance, and 'c' represents the diagonal length of the smallest rectangle containing these two boxes. It is the intersection-union ratio of the ground truth bounding boxes and the predicted bounding boxes. It is a weighting function. Used to measure similarity in aspect ratio; ， in, It's a real label. A value of 1 indicates the existence of a target. A value of 0 indicates that there is no target. This is the confidence score of the prediction: ， in, It is the number of categories. It's a real label. It is the predicted class probability; use With distillation loss The summation is used to train the student target detection network until the model converges, ultimately yielding the underwater target detection network model.

Images