Underwater netting damage detection method based on fusion of convolutional neural network and Transform algorithm
By integrating convolutional neural networks and the Transformer algorithm, a lightweight underwater netting damage detection model was constructed, which solved the problems of high training cost and poor robustness of traditional methods. This model achieves efficient and accurate netting damage detection, reduces the dependence on manually labeled data, and improves the robustness and detection efficiency of the system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-29
- Publication Date
- 2026-04-03
AI Technical Summary
Existing underwater net detection methods have high training costs and poor system robustness. Traditional deep learning models have reduced recognition accuracy in new environments, posing safety risks and inaccurate detection problems.
An underwater mesh damage detection method based on the fusion of convolutional neural network and Transformer algorithm is adopted. By customizing identity blocks, hierarchical decomposition blocks, structure preservation blocks and self-supervised contrastive loss function, a lightweight network model is constructed to achieve multi-scale feature extraction and self-supervised learning, thereby reducing training costs and improving robustness.
This system enables efficient and accurate detection of netting damage in various aquatic environments, reducing training costs, improving detection accuracy and robustness, reducing reliance on manually labeled data, and ensuring personnel safety and system stability.
Smart Images
Figure CN121789023A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of netting inspection technology, and in particular to an underwater netting damage detection method based on the fusion of convolutional neural networks and the Transformer algorithm. Background Technology
[0002] Deep-sea aquaculture nets are easily damaged by natural conditions such as wind, waves, currents, and fish movement. Failure to detect damage in time can have serious consequences. At best, it can lead to fish escaping; at worst, it can cause structural damage to the entire aquaculture system, resulting in significant property loss and even personal injury. Fish escape and damage to nets and other facilities will impact the marine ecosystem. Currently, traditional net inspection methods rely on manual inspection. In the complex marine environment, manual inspection presents risks to personnel safety and inaccuracies. With the rise of artificial intelligence (AI) technology, underwater robots use AI algorithms to identify and inspect transmitted net images.
[0003] While this method reduces the risk to personnel safety, these traditional deep learning models rely on large amounts of manually labeled datasets for training to improve recognition accuracy, resulting in high training costs. This fully supervised learning algorithm will cause the model to recognize image information in new environments, leading to a decrease in recognition accuracy and thus reducing the robustness of the entire system. Summary of the Invention
[0004] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide an underwater netting damage detection method based on the fusion of convolutional neural networks and Transformer algorithms, which solves the problems of high training costs and poor system robustness of existing methods.
[0005] To achieve the above objectives, the present invention provides the following solution: A method for detecting underwater netting damage based on the fusion of convolutional neural networks and the Transformer algorithm includes: The image of the underwater netting to be detected is input into a pre-trained underwater netting damage detection model for identification, thereby obtaining the underwater netting damage detection result; the construction process of the underwater netting damage detection model includes: Images of netting in different aquatic environments were collected to obtain an unlabeled image set; An original network model is constructed based on a defined improved convolutional neural network, a lightweight Transformer network, and an asymmetric structure decoder. The outputs of the improved convolutional neural network and the lightweight Transformer network in the original network model are connected to the asymmetric structure decoder via a multi-input adder. The decoder of the improved convolutional neural network embeds a custom identity block, a hierarchical decomposition block, and a structure preservation block. The lightweight Transformer network employs a cascaded two-layer structure. The original network model is iteratively trained using a preset self-supervised contrastive loss function based on the unlabeled image set to obtain the trained underwater netting damage detection model.
[0006] Preferably, the custom identity block includes: a first convolutional layer, a first batch normalization, a first modified linear unit, a second convolutional layer, a second batch normalization, and a first additive layer; The input of the first convolutional layer is connected to the input of the first additive layer.
[0007] Preferably, the hierarchical decomposition block includes: a first branch, a second branch, and a splicing unit; the first branch and the second branch have the same structure; the first branch includes: cascaded average pooling, a third convolutional layer, a third batch normalization, a second corrected linear unit, and a scaling operation; The input terminals of the first branch and the second branch are connected; the output terminals of the first branch and the second branch are respectively connected to the splicing unit.
[0008] Preferably, the structure-preserving block includes: a third branch and a fourth branch; the third branch includes: a fourth convolutional layer in series, a fourth batch normalization, a third corrected linear unit, a fifth convolutional layer, a fifth batch normalization, and a second additive layer; the fourth branch includes: a sixth convolutional layer in series and a sixth batch normalization. The input of the fourth convolutional layer is connected to the input of the sixth convolutional layer; the input of the second additive layer is connected to the output of the sixth batch normalization layer.
[0009] Preferably, the asymmetric structure decoder includes: a third adder in series, a seventh convolutional layer, a seventh batch normalization, a fourth modified linear unit, an eighth convolutional layer, and a normalized exponential function.
[0010] Preferably, the self-supervised contrastive loss function is: ;in, ; ; ; This is the loss value; This represents the number of samples in the current training batch. The number of positive samples; It is a similarity measure for positive samples; The number of negative samples; It serves as a similarity measure for negative samples; , , These are the sample index, positive sample index, and negative sample index, respectively. For similarity functions; For the sample With target category The included angle between them; For the sample With negative categories The included angle between them; For the degree of attention; For angle increments; for and The included angle between them; It is the prototype vector; This is the feature vector of the sample.
[0011] Preferably, the lightweight Transformer network includes: a cascaded T-1 layer and a T-2 layer; the T-1 layer and the T-2 layer have the same architecture; the T-1 layer includes: a cascaded layer normalization, a multi-head self-attention unit, and a fusion unit; The input terminal of the layer normalization is connected to the input terminal of the fusion unit.
[0012] The present invention discloses the following technical effects: This invention provides an underwater netting damage detection method based on the fusion of convolutional neural networks and the Transformer algorithm. By using a custom identity block, it solves the problem of drastic feature map size reduction caused by repeated sampling in traditional methods, enabling the extraction of local features while maintaining the feature map size. Through a hierarchical decomposition block, it addresses the insensitivity of traditional CNNs to small targets due to fixed receptive fields, achieving multi-branch structure and multi-scale parallel feature extraction. Through a structure preservation block, it solves the problem of traditional convolutional neural networks easily losing spatial details of the netting, effectively preserving the shape and boundary information of the netting image. Through a self-supervised contrastive loss function, it solves the problem of decreased recognition accuracy when dealing with unlabeled image information, eliminating the need for re-labeling and training on additional datasets. By using a lightweight Transformer network, it solves the problem of structural redundancy in traditional Transformer encoders, improving overall operating efficiency and reducing training costs. Attached Figure Description
[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0014] Figure 1 This is a schematic diagram of the underwater netting damage detection process based on the fusion of convolutional neural network and Transformer algorithm provided in an embodiment of the present invention; Figure 2 An overall architecture diagram provided for embodiments of the present invention; Figure 3 The overall architecture of the IB algorithm provided in this embodiment of the invention; Figure 4 The overall architecture of the HDB algorithm provided in this embodiment of the invention; Figure 5 The overall architecture of the SPB algorithm provided in this embodiment of the invention; Figure 6 The overall decoder architecture provided in the embodiments of the present invention; Figure 7 The T-1 and T-2 algorithm architectures provided in the embodiments of the present invention; Figure 8 This is a flowchart of the mesh damage detection process provided in an embodiment of the present invention; Figure 9 This is a diagram showing the damage to the mesh fabric provided in an embodiment of the present invention. Figure 10 This is a diagram illustrating the effect of mesh damage detection and identification provided in an embodiment of the present invention. Detailed Implementation
[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0016] The purpose of this invention is to provide an underwater netting damage detection method based on the fusion of convolutional neural networks and the Transformer algorithm, which solves the problems of high training cost and poor system robustness of existing methods.
[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0018] Figure 1 This is a schematic diagram of the underwater netting damage detection process based on the fusion of convolutional neural networks and Transformer algorithms provided in an embodiment of the present invention, as shown below. Figure 1 As shown, this invention provides a method for detecting underwater netting damage based on the fusion of convolutional neural networks and the Transformer algorithm, comprising: Step 100: Input the image of the netting to be detected into the pre-trained underwater netting damage detection model for identification, and obtain the underwater netting damage detection result; the construction process of the underwater netting damage detection model includes: Step 101: Collect images of netting in different aquatic environments to obtain an unlabeled image set; Step 102: Construct the original network model based on the defined improved convolutional neural network, lightweight Transformer network, and asymmetric structure decoder; the outputs of the improved convolutional neural network and the lightweight Transformer network in the original network model are connected to the asymmetric structure decoder through a multi-input adder; the decoder of the improved convolutional neural network embeds a custom identity block, a hierarchical decomposition block, and a structure preservation block; the lightweight Transformer network adopts a two-layer cascaded structure; Step 103: Based on the unlabeled image set, the original network model is iteratively trained using a preset self-supervised contrastive loss function to obtain the trained underwater netting damage detection model.
[0019] Specifically, the custom identity block includes: a first convolutional layer, a first batch normalization, a first modified linear unit, a second convolutional layer, a second batch normalization, and a first additive layer; The input of the first convolutional layer is connected to the input of the first additive layer.
[0020] Furthermore, the hierarchical decomposition block includes: a first branch, a second branch, and a splicing unit; the first branch and the second branch have the same structure; the first branch includes: cascaded average pooling, a third convolutional layer, a third batch normalization, a second corrected linear unit, and a scaling operation; The input terminals of the first branch and the second branch are connected; the output terminals of the first branch and the second branch are respectively connected to the splicing unit.
[0021] Specifically, the structure-preserving block includes: a third branch and a fourth branch; the third branch includes: a fourth convolutional layer in series, a fourth batch normalization, a third corrected linear unit, a fifth convolutional layer, a fifth batch normalization, and a second additive layer; the fourth branch includes: a sixth convolutional layer in series and a sixth batch normalization. The input of the fourth convolutional layer is connected to the input of the sixth convolutional layer; the input of the second additive layer is connected to the output of the sixth batch normalization layer.
[0022] Furthermore, the asymmetric structure decoder includes: a third adder in series, a seventh convolutional layer, a seventh batch normalization, a fourth modified linear unit, an eighth convolutional layer, and a normalized exponential function.
[0023] Specifically, the self-supervised contrastive loss function is: ;in, ; ; ; This is the loss value; This represents the number of samples in the current training batch. The number of positive samples; It is a similarity measure for positive samples; The number of negative samples; It serves as a similarity measure for negative samples; , , These are the sample index, positive sample index, and negative sample index, respectively. For similarity functions; For the sample With target category The included angle between them; For the sample With negative categories The included angle between them; For the degree of attention; For angle increments; for and The included angle between them; It is the prototype vector; This is the feature vector of the sample.
[0024] Furthermore, the lightweight Transformer network includes: a cascaded T-1 layer and a T-2 layer; the T-1 layer and the T-2 layer have the same architecture; the T-1 layer includes: a cascaded layer normalization, a multi-head self-attention unit, and a fusion unit; The input terminal of the layer normalization is connected to the input terminal of the fusion unit.
[0025] refer to Figure 2 Overall Algorithm Architecture Description. Addressing the challenge of identifying damage to aquaculture netting in a marine ranch, this embodiment provides an underwater netting damage detection method based on a fusion of convolutional neural networks and the Transformer algorithm. Figure 2The diagram illustrates the following: CONV (Convolutional Layer): Feature extraction; ADD (Additive Layer): Feature fusion, combining features from different sources; ReLU (Revised Linear Unit): Introducing a non-linear network to provide non-linear modeling capabilities; IB (Custom Identity Block): Maintaining feature consistency and preventing information loss; HDB (Hierarchical Decomposition Block): Decomposing features to different scales to extract more resolution information; OSP (One-Sided Padding): Ensuring the output feature map of a certain layer of the encoder is completely consistent in size with the feature map of the corresponding layer of the decoder; SPB (Structure Preservation Block): Preserving the spatial structure of the feature map and avoiding over-downsampling that leads to loss of detail; BN (Batch Normalization): Standardizing the input of each layer in the neural network; ZP (Zero Padding): Symmetrically padding the input feature map with zero-value pixels around it to prevent loss of edge pixel information; MP (Max Pooling): Gradually reducing the resolution of the feature map, expanding the receptive field, and extracting more abstract features; Softmax (Normalized Exponential Function): Converting the raw data output by the neural network into a probability distribution; A: Represents element-wise addition or feature addition. The red box in the diagram represents the encoder, the green box represents the Transformer algorithm, and the blue box represents the asymmetric decoder. The algorithm's operation involves inputting the image information of the mesh fabric into a convolutional neural network and a Transformer algorithm for data processing. Then, the local features extracted by the convolution are deeply fused with the global context features calculated by the Transformer. Finally, an asymmetric decoder is used to render the fused information into a mesh fabric defect detection image with clear boundaries and accurate categories.
[0026] Specifically, the convolutional neural network algorithm was improved. Decoder: In the decoder, IB (custom identity block), HDB (hierarchical decomposition block), and SPB (structure preservation block) were defined.
[0027] Furthermore, IB (Custom Identity Block). Figure 3 This describes the overall architecture of the IB algorithm. A custom IB (Custom Identity Block) is defined, similar to the identity mapping in a residual network, to maintain the stability of the feature map information, extract local features, and ensure smooth gradient propagation.
[0028] Specifically, HDB (Hierarchical Decomposition Block). Figure 4This is the overall architecture of the HDB algorithm. Legend: Resize: Performs a physical-level size transformation; Concatenation: Merges multi-scale features along the channel dimension; Pooling Factor: Controls the downsampling level, determining how many times the input feature map is reduced; Average Pooling: Performs the actual downsampling operation, converting the feature map from high resolution to low resolution; Resize Factor: Controls the upsampling level, enlarging the low-resolution feature map to a size matching the original input. A custom HDB (Hierarchical Decomposition Block) is defined for multi-scale feature extraction and hierarchical decomposition to improve the model's recognition ability and robustness for underwater aquaculture netting images. Multi-scale feature extraction: HDB is used to generate feature representations at different scales, decomposing features to different scales to extract more resolution information. Hierarchical decomposition: Through the hierarchical decomposition structure, HDB can better separate and represent the set of complete and damaged areas of the netting, improving the model's classification and segmentation accuracy.
[0029] Furthermore, SPB (Structure Retention Block). Figure 5 This describes the overall architecture of the SPB algorithm. A custom SPB (Structure Preservation Block) is defined: it preserves the shape and boundary information of the target, designed to address the problem of blurred boundaries in underwater netting images, thus improving segmentation accuracy. The SPB module is built based on residual learning principles; its role is to fuse convolutional paths that maintain spatial dimensions with an identity mapping shortcut. In the early and middle stages of feature extraction, SPB can act as an anchor point to lock the spatial dimensions of the feature map, ensuring that the geometric information used to describe defects is not diluted or lost, thereby improving the accuracy of the overall segmentation boundaries.
[0030] Specifically, the decoder. Figure 6 This describes the overall architecture of the decoder. The improved algorithm does not use a traditional symmetric decoder structure, but instead employs an asymmetric structure, enabling the decoder to better compute the asymmetric information fused from the encoder features and Transformer features.
[0031] Preferably, this embodiment introduces a self-supervised learning mechanism. This mechanism can utilize experience learned from existing datasets to adapt to netting damage detection in different aquatic environments, and can achieve cross-dataset netting damage detection without the need for re-labeling additional datasets for training. In traditional self-supervised learning mechanisms, the loss function... Its function is to directly compare the model's predicted output with manually labeled information in the output space (pixel level), measuring the difference between the model's prediction and the true value, and providing direction and basis for model optimization. The contrastive loss function in this algorithm... This transforms feature representations from a standard space to an angular space. In this angular space, the model reduces the angles between features of similar samples and increases the angles between features of dissimilar samples. This algorithm allows the model to be trained without relying on any manual annotations, using experience to guide it. It concentrates similar feature data in the angular space while separating feature data of different classes. In other words, this function acts as an unsupervised clustering guide, enabling the model to learn how to distinguish between damaged and intact mesh, thus achieving a self-supervised learning effect. The mathematical expression is: Extract the feature vectors of samples from the target domain (new dataset). Pull towards the prototype vector of its corresponding predicted category. And move away from other category prototypes, that is, map the features of the target domain to the category prototypes (experiences) learned in the source domain. , The number of positive and negative samples. It calculates the similarity with all positive samples; It calculates the similarity with all negative samples. This represents the number of samples in a training batch. : Represents the feature vector With category prototype vector The angle between two objects. It is the core indicator for measuring the similarity between two objects in angular space: the smaller the angle, the larger the cosine value, indicating that the two objects are more similar. Similarity function, which measures the similarity between two vectors, where It is an angular increment that makes the angle requirements between positive sample pairs more stringent. This can improve feature discrimination performance. Parameters It is the degree to which the model pays attention to samples that are not easy to identify.
[0032] when A small loss value indicates that the features extracted by the model have formed a good separation in the angular space, with similar features among samples of the same class and significant differences in features among samples of different classes. The identified image is similar to images from experience and belongs to the same category. Conversely, a large loss value indicates that the identification did not separate the sample features well and requires iterative identification. In each iteration, the model uses a contrastive loss function. Training constraints were imposed to ensure that the model could maintain high accuracy and robustness in detecting mesh damage even with training on a small dataset.
[0033] refer to Figure 7 This embodiment simplifies the Transformer algorithm. Compared to the traditional Transformer algorithm, it does not use the standard 8-layer or 12-layer structure, but simplifies it to 2 layers, with T-1 and T-2 tightly integrated in a serial manner. T-1 and T-2 have the same architecture. Specifically: Layer Normalization: Each feature vector is independently normalized across all channel dimensions; Multi-Head Self-Attention (MHSA): Used to establish global relationships between all image patches; Fusion: The output of the sub-layer (MHSA or FFN) is added element-wise to the sub-layer input; Feed-Forward Network (FFN): Performs independent, non-linear transformations on the features at each location.
[0034] Specifically, the impact of reducing the number of Transformer layers on the mesh fabric damage identification results was investigated using the same dataset. An ablation experiment was conducted (only the number of Transformer layers in the mesh fabric damage identification algorithm was changed, while other conditions remained unchanged). The experimental results for different numbers of Transformer layers are shown in Table 1.
[0035] Table 1
[0036] The results show that the 2-layer algorithm achieves the best performance compared to the 8-layer and 5-layer Transformer algorithms, which greatly reduces the overall complexity of calculating mesh damage. Since the study only focuses on identifying mesh damage, the 2-layer structure, tightly integrated sequentially, is sufficient to function as an attention module. Too many layers would not only increase the computational burden but also lead to overfitting.
[0037] Furthermore, the effectiveness and process of damage detection were identified. The improved fusion algorithm and traditional recognition algorithms YOLOv10, SETR, and SegFormer were compared. An experiment was conducted using three sets of unknown data (without manual annotation) of mesh fabric to test the recognition rate of the mesh fabric. The experimental results of the improved fusion algorithm and traditional mesh fabric recognition algorithm are shown in Table 2.
[0038] Table 2
[0039] The experimental data above shows that, in the three unknown mesh fabric datasets, the improved fusion algorithm proposed in this embodiment achieves excellent performance in the three core metrics of mAP, μIoU, and μDC, demonstrating higher recognition accuracy when dealing with unknown mesh fabric data. This means that in actual mesh fabric damage detection, the method in this embodiment has better accuracy and robustness compared to traditional recognition methods.
[0040] Specifically, this section describes the process and effectiveness of damage detection. (Reference) Figure 8 Under the guidance of the information transmission module, the raw image information of the netting acquired by the image acquisition module on the ROV underwater robot is transmitted to the algorithm processing module for image information analysis to detect whether the netting is broken or damaged. This processed information is then transmitted to the information storage module and a remote terminal. The system automatically detects and analyzes the netting damage throughout the entire process. This underwater netting damage detection method, based on the fusion of convolutional neural networks and Transformer algorithms, improves netting detection efficiency while effectively reducing the consumption of manpower and resources in netting damage detection, thus ensuring personnel safety and improving the safety and efficiency of deep-sea aquaculture systems. (Reference for recognition effect) Figure 9 and Figure 10 Damaged parts of the mesh are shown in red, while intact parts are shown in black.
[0041] The beneficial effects of this invention are as follows: (1) Multi-scale feature extraction to reduce the loss of detailed information. In traditional mesh fabric damage recognition algorithms, convolutional neural networks are usually stacked convolutions in a single path, and repeated sampling (pooling or large stride convolution) will cause the feature map size to shrink drastically, which will lose the precise boundary and shape information of the object. Traditional CNNs have the disadvantages of fixed receptive fields, insensitivity to small targets, easy loss of spatial detail information, and single feature extraction level. To address these problems, this invention proposes a multi-modal design for the encoder part of the convolutional neural network: custom IB (custom identity block), HDB (hierarchical decomposition block), SPB (structure preservation block), etc., to make up for these shortcomings. The custom IB module can keep the size of the feature map unchanged and extract local features. The custom HDB module can realize multi-scale receptive fields, improve the sensitivity to defects of different sizes, and realize multi-branch structure and multi-scale parallel extraction of features. In the face of the problem that traditional convolutional neural networks are prone to losing the spatial details of mesh fabric, the custom SPB module in the algorithm can reduce the loss of spatial information of mesh fabric feature maps, improve the accuracy of segmentation boundaries, and effectively preserve the shape and boundary information of mesh fabric images.
[0042] (2) Reduced model training cost and self-learning algorithm. Currently, existing artificial intelligence recognition algorithm models are all fully supervised learning, which requires a large amount of labeled data for training. This method significantly increases the model training cost, and the recognition accuracy will decrease when faced with unlabeled image information. The core logic of the loss function in traditional recognition algorithms is to directly compare the model's predicted output with the manually labeled information in the output space (pixel level). The goal of this type of loss function is to make the predicted image infinitely close to the labeled image to achieve the recognition purpose, which requires a large amount of labeled data. However, the contrastive loss function in this invention gets rid of the dependence on pixel-level labeling. The model brings similar features closer and pushes away dissimilar features in the non-pixel level angular space, realizing a self-supervised learning mechanism. This mechanism can use the experience learned in the existing dataset to adapt to the identification and detection of net damage in different aquatic environments without the need to re-label additional datasets for training, thus reducing the need for labeled samples.
[0043] (3) This invention employs a fusion of multi-scale convolution and Transformer algorithms for detecting netting damage. The multi-head attention mechanism in the Transformer model makes the model more comprehensive in observing information about the netting, resulting in more accurate recognition. In the field of image recognition, the Vision Transformer has at least 12 layers of Transformer encoders. In the improved fusion algorithm, the segmentation of netting damage relies more on the local features extracted by the convolutional neural network. The Transformer mainly plays an auxiliary enhancement role here, rather than a dominant role, so it does not require too many layers. The Transformer algorithm is designed to be lightweight. While ensuring the overall accuracy of the algorithm, the overall efficiency of the algorithm is improved and the training cost is reduced by reducing the number of structural layers. This underwater netting damage detection method based on the fusion of convolutional neural network and Transformer algorithm not only extracts features at multiple scales, reducing the loss of detailed information, but also reduces the learning and training cost, improving the accuracy and robustness of detection.
[0044] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0045] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for detecting underwater netting damage based on the fusion of convolutional neural networks and the Transformer algorithm, characterized in that, include: The image of the netting to be detected is input into a pre-trained underwater netting damage detection model for identification, and the underwater netting damage detection result is obtained. The construction process of the underwater netting damage detection model includes: Images of netting in different aquatic environments were collected to obtain an unlabeled image set; An original network model is constructed based on a defined improved convolutional neural network, a lightweight Transformer network, and an asymmetric structure decoder. The outputs of the improved convolutional neural network and the lightweight Transformer network in the original network model are connected to the asymmetric structure decoder via a multi-input adder. The decoder of the improved convolutional neural network embeds a custom identity block, a hierarchical decomposition block, and a structure preservation block. The lightweight Transformer network employs a cascaded two-layer structure. The original network model is iteratively trained using a preset self-supervised contrastive loss function based on the unlabeled image set to obtain the trained underwater netting damage detection model.
2. The underwater netting damage detection method based on the fusion of convolutional neural network and Transformer algorithm according to claim 1, characterized in that, The custom identity block includes: a first convolutional layer, a first batch normalization, a first modified linear unit, a second convolutional layer, a second batch normalization, and a first additive layer; The input of the first convolutional layer is connected to the input of the first additive layer.
3. The underwater netting damage detection method based on the fusion of convolutional neural network and Transformer algorithm according to claim 1, characterized in that, The hierarchical decomposition block includes: a first branch, a second branch, and a splicing unit; the first branch and the second branch have the same structure; the first branch includes: cascaded average pooling, a third convolutional layer, a third batch normalization, a second corrected linear unit, and a scaling operation; The input terminals of the first branch and the second branch are connected; the output terminals of the first branch and the second branch are respectively connected to the splicing unit.
4. The underwater netting damage detection method based on the fusion of convolutional neural network and Transformer algorithm according to claim 1, characterized in that, The structure preservation block includes: a third branch and a fourth branch; the third branch includes: a fourth convolutional layer in series, a fourth batch normalization, a third corrected linear unit, a fifth convolutional layer, a fifth batch normalization, and a second additive layer; the fourth branch includes: a sixth convolutional layer in series and a sixth batch normalization. The input of the fourth convolutional layer is connected to the input of the sixth convolutional layer; the input of the second additive layer is connected to the output of the sixth batch normalization layer.
5. The underwater netting damage detection method based on the fusion of convolutional neural network and Transformer algorithm according to claim 1, characterized in that, The asymmetric structure decoder includes: a third adder in series, a seventh convolutional layer, a seventh batch normalization, a fourth modified linear unit, an eighth convolutional layer, and a normalized exponential function.
6. The underwater netting damage detection method based on the fusion of convolutional neural network and Transformer algorithm according to claim 1, characterized in that, The self-supervised contrastive loss function is: ;in, ; ; ; This is the loss value; This represents the number of samples in the current training batch. The number of positive samples; It is a similarity measure for positive samples; The number of negative samples; It serves as a similarity measure for negative samples; , , These are the sample index, positive sample index, and negative sample index, respectively. For similarity functions; For the sample With target category The included angle between them; For the sample With negative categories The included angle between them; For the degree of attention; For angle increments; for and The included angle between them; It is the prototype vector; This is the feature vector of the sample.
7. The underwater netting damage detection method based on the fusion of convolutional neural network and Transformer algorithm according to claim 1, characterized in that, The lightweight Transformer network includes: a cascaded T-1 layer and a T-2 layer; the T-1 layer and the T-2 layer have the same architecture; the T-1 layer includes: a cascaded layer normalization, a multi-head self-attention, and a fusion unit; The input terminal of the layer normalization is connected to the input terminal of the fusion unit.