Underwater crack segmentation identification method based on double-teacher knowledge distillation
By combining a dual-teacher knowledge distillation transfer learning framework with ground and underwater models, the problem of scarce underwater crack detection data was solved, the robustness and accuracy of underwater crack segmentation were improved, and efficient crack identification was achieved.
Patent Information
- Application Number
- CN202511469395.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-15
- Publication Date
- 2026-02-24
AI Technical Summary
Existing underwater crack detection methods suffer from problems such as scarce data, low segmentation accuracy, and high cost, making it difficult to effectively identify cracks in complex underwater environments.
A dual-teacher knowledge distillation transfer learning framework is adopted, which combines a ground crack segmentation model and an underwater saliency segmentation model. A dynamically adjusted knowledge distillation loss function is designed. By fusing knowledge from the ground and underwater teacher models, the robustness and segmentation ability of the underwater crack segmentation model are improved.
The robustness and segmentation accuracy of the underwater crack segmentation model have been improved, achieving higher recognition accuracy and generalization ability, while reducing labor costs.
Smart Images
Figure CN121564334A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image perception technology, specifically relating to an underwater crack segmentation and recognition method based on dual-teacher knowledge distillation. Background Technology
[0002] In water resource management, the construction of large-scale water conservancy facilities (such as dams and offshore platforms) is crucial for the collection and utilization of water resources. Long-term use and the effects of natural forces cause structural deformation and even cracks in stress concentration areas, threatening their safety and functionality. While surface cracks can be detected and located using efficient technologies such as drones, underwater cracks are difficult to identify due to poor imaging quality and complex environments, posing a significant technical challenge. Therefore, developing efficient underwater crack detection methods is of great importance for ensuring the long-term stable operation of infrastructure.
[0003] Traditional manual inspection methods suffer from significant limitations such as low efficiency, high cost, and strong subjectivity. In recent years, with the development of autonomous underwater vehicle (AUV) technology, its application in underwater structure crack detection has gradually become a research hotspot. By introducing computer vision technology, AUVs can achieve high-precision underwater image acquisition, and machine learning algorithms can be used to analyze and process the images, thereby realizing automatic crack detection and quantitative damage assessment.
[0004] Collecting underwater crack datasets is extremely difficult. Furthermore, due to the complex structure and numerous branches of cracks, pixel-by-pixel segmentation and annotation of the collected crack images consumes a significant amount of manpower. Therefore, there are virtually no publicly available underwater crack datasets. Currently, most researchers improve the accuracy of underwater crack segmentation through image augmentation or transfer learning. However, how to train a network model with high generalization ability and strong segmentation capabilities using limited data resources remains an important research problem. Summary of the Invention
[0005] The purpose of this invention is to propose an underwater crack segmentation transfer learning framework with dual-teacher knowledge distillation, thereby addressing the problem of scarce available research data in the background.
[0006] The technical solution adopted in this invention is:
[0007] A method for underwater crack segmentation and recognition based on dual-teacher knowledge distillation is characterized by: simultaneously performing segmentation prediction on an underwater crack image using a surface crack segmentation model and an underwater saliency segmentation model to obtain the prediction distributions of the two teachers; comparing the two prediction distributions with the true values to obtain the confidence scores of the two teachers regarding the foreground and background in the crack segmentation, where the foreground corresponds to the crack portion of the crack segmentation prediction image and the background corresponds to the underwater background portion; and extracting the confidence scores of the two teachers regarding the knowledge corresponding to the student model, i.e., the foreground confidence score in the segmentation prediction image of the first teacher. Corresponding to the knowledge of cracks, the background confidence in the segmentation prediction graph of the second teacher. Based on the corresponding underwater background knowledge, the knowledge of the two teacher models can be fused. After obtaining the corresponding foreground and background confidence scores, normalizing both yields the dual-teacher soft loss factor. The value of .
[0008] Compared with the prior art, the present invention has the following advantages:
[0009] This invention proposes a transfer learning framework based on dual-teacher distillation for underwater crack segmentation tasks. A surface crack teacher model and an underwater saliency segmentation model transfer surface crack knowledge and underwater background knowledge to the underwater crack segmentation model, respectively. A dynamically adjusted knowledge distillation loss function based on dual-teacher confidence is designed. Each teacher obtains a segmentation confidence score by comparing their generated segmentation prediction map with the ground truth label. This confidence score controls the knowledge output of the teacher model to the student model. Ablation experiments are conducted on the designed underwater crack segmentation student model WCrack-Unet, demonstrating that the model using this method exhibits higher robustness and segmentation ability during recognition. Attached Figure Description
[0010] Figure 1 It is a transfer learning framework based on dual-teacher knowledge distillation for underwater fracture segmentation tasks.
[0011] Figure 2 This is the underwater crack segmentation student model network architecture designed in this invention.
[0012] Figure 3 It is a dynamic adjustment strategy for the dual-teaching soft loss factor based on confidence level. Detailed Implementation
[0013] To better understand the purpose, structure, and function of this invention, the invention will be described in further detail below with reference to the accompanying drawings.
[0014] Please see Figure 1 A transfer learning framework based on dual-teacher knowledge distillation for underwater crack segmentation tasks includes at least the following steps:
[0015] Step 1: Train the ground crack segmentation model. Input the underwater crack optical image into the ground crack segmentation model to obtain an underwater crack segmentation image with ground crack knowledge.
[0016] Step 2: The underwater saliency segmentation model outputs the segmentation results of the underwater crack image after learning from a dataset rich in underwater background knowledge.
[0017] Step 3: Combining the advantages of the segmentation prediction map with the ground crack knowledge from Step 1 and the underwater background knowledge from Step 2, the student model is guided to improve the underwater crack segmentation accuracy through knowledge distillation.
[0018] See Figure 2 In step 3, this invention designs a student model suitable for underwater crack segmentation (this model is used to identify underwater cracks). This model is an improvement on the Unet-FCN surface crack segmentation model, and improves the model's sensitivity to key features by introducing an SE attention mechanism to mine the relationships between channels. The SE attention mechanism first extracts the data from the convolutional layer... The feature map X of size X is convolved with a 2D spatial kernel to obtain C outputs, which are then reassembled to obtain a feature map U of uniform size. Among them, This can be considered a standard convolution operator, namely:
[0019]
[0020] The obtained feature map U is then subjected to global average pooling, compressing the global spatial information into a single channel descriptor. The feature map U of size can be embedded with global information to obtain a feature vector Z containing contextual information. The size of this feature vector is... Each element in the feature vector Z corresponds to the global statistical information of one channel, alleviating the channel dependency problem. The specific calculation formula is as follows:
[0021]
[0022] Where Zc represents the C-th element in the compressed feature vector Z. The input feature map is then used to fully capture the inter-channel dependencies through excitation. During this process, the network learns the non-linear relationships between channels and generates corresponding weights for each channel. To meet these criteria, a gating mechanism is constructed using two fully connected (FC) layers. The first FC layer performs dimensionality reduction, compressing the number of channels from C to... The ReLU function is applied for activation, where r represents the compression ratio. The second FC layer restores the number of channels to the input value C by increasing the dimensionality, and then normalizes it using the sigmoid function, as shown in the following formula:
[0023]
[0024] in, It is a dimension-reduced weight matrix. It is the ReLU activation function. It is a dimension-reduced weight matrix. It uses the Sigmoid activation function. Finally, the weights of each channel are multiplied channel-by-channel with the input feature map to recalibrate the importance of each channel. The enhanced feature map is then output to the next layer of the network. The specific recalibration formula is as follows:
[0025]
[0026] in, This represents the weight of the c-th channel. The c-th channel represents the normalized original feature map.
[0027] Furthermore, the knowledge guidance provided by the teacher model to the student model during knowledge distillation is reflected in the design of the loss function. The loss function in knowledge distillation comprises two parts: a hard loss representing the difference between the student model and the true label, and a soft loss representing the difference between the student model and the teacher model. The two loss parts are weighted and summed to form the total loss function. This invention will detail the design scheme of the loss function based on dual-teacher knowledge distillation and the implementation process of dynamically adjusting the weights designed according to the two loss functions.
[0028] The hard loss function represents the difference between the student model and the true label. Considering that the pixels representing cracks account for a small percentage of the image and the ratio of positive to negative classes is severely imbalanced, this invention adopts the Focal Loss function. Focal Loss focuses more on hard and minority samples in the image, increasing the model's attention to cracks. The specific formula for the Focal Loss function is as follows:
[0029]
[0030] in, This represents the model's predicted probability of the true class. γ is the focusing parameter, controlling the rate of weight decay between easy and difficult samples. As a modulation factor controlling the loss contribution of easy and difficult samples, the larger γ is, the stronger the suppression of easy samples, making the model pay more attention to difficult samples. As a balancing factor to regulate the class balance of positive and negative samples, this invention takes... γ equals 1, and γ equals 2.
[0031] The soft loss function represents the difference between the student model and the teacher model, which is reflected by the KL divergence. The KL divergence represents the distribution of predicted probabilities generated by the student model. Soft label probability distribution generated by the teacher model The higher the KL divergence value, the greater the difference between the two distributions; conversely, a lower value indicates a closer similarity between the two distributions, meaning the student model's predictions are closer to the high-precision teacher model's predictions. The formula for the soft loss function is as follows:
[0032]
[0033] Here, T is a temperature coefficient representing the degree of soft label softening. A higher T indicates a smoother, softer soft label, which is more tolerant of small errors in the student model compared to soft labels with lower T values. This helps prevent the student model from overfitting to noise or subtle differences in the teacher model during training, thus improving the model's robustness. The soft loss function is calculated by multiplying T after calculating the KL divergence. 2 This is to maintain gradient consistency. Since the derivative of the softmax function introduces 1 / T, the gradient of the loss function with respect to the probability distribution will include a term of 1 / T. 2 The gradient can be restored to a reasonable magnitude. In this invention, the value of T is 3.
[0034] See Figure 3 Unlike the soft loss definition used in single-teacher knowledge distillation, the student model needs to consider the effects of both teachers on the student model when learning from a dual-teacher model. The specific formula is as follows:
[0035]
[0036] in, and These represent the individual KL divergence values between the two teacher and student models, As a dual-teacher soft loss factor, it controls the impact of different teachers on the student model. A larger value means the student model is more biased towards learning from the first teacher's prediction. In this paper... The larger the value, the more the student model for underwater fracture segmentation will be influenced by the teacher model for terrestrial fracture segmentation. Compared to the second-ranked underwater saliency segmentation teacher model, the generated prediction map will be closer to the prediction map of the terrestrial fracture segmentation model.
[0037] This invention draws on the concept of confidence level in statistics to construct a two-factor soft loss factor based on confidence level. The dynamic adjustment strategy is shown in the attached diagram. Figure 3 As shown, by simultaneously performing segmentation predictions on an underwater crack image using both a ground crack segmentation model and an underwater saliency segmentation model, we can obtain the prediction distributions of two teachers. Comparing the two prediction distributions with the actual values yields the confidence scores of the two teachers regarding the foreground and background in the crack segmentation. The foreground corresponds to the crack portion in the crack segmentation prediction image, and the background corresponds to the underwater background portion. Therefore, we extract the confidence scores of the two teachers regarding the knowledge corresponding to the student model, i.e., the foreground confidence score in the segmentation prediction image of the first teacher. Corresponding to the knowledge of cracks, the background confidence in the segmentation prediction graph of the second teacher. Based on the corresponding underwater background knowledge, the model knowledge of the two teachers can be integrated.
[0038] After obtaining the corresponding foreground and background confidence scores, normalizing both yields the dual-teaching soft loss factor. The value of is given by the following formula:
[0039]
[0040] During crack segmentation, the network model generates and outputs a predicted segmentation map. By comparing the predicted map pixel-by-pixel with the manually labeled ground truth (GT) map, different pixels can be classified into four categories: True Positive (TP), samples correctly predicted as positive; True Negative (TN), samples correctly predicted as negative; False Positive (FP), samples incorrectly predicted as positive; and False Negative (FN), samples incorrectly predicted as negative. The number of pixels corresponding to each of the four categories in the predicted segmentation map can be statistically determined. These four metrics, represented as a matrix, form the Confusion Matrix, as shown in Table 1.
[0041] Table 1 Confusion Matrix
[0042]
[0043] Mean Intersection over Union (mIoU) is the most crucial evaluation metric in image segmentation, used to measure the accuracy of a model in pixel-level classification. Its core idea is to calculate the overlap ratio between the predicted and ground truth regions and then average the results for all categories. The formula is as follows:
[0044]
[0045] Where N represents the number of categories in the image segmentation, The corresponding category is The interaction ratio at any given time, and the specific formula for calculating the interaction ratio is as follows:
[0046]
[0047] In this equation, TP, FP, and FN represent the values of the corresponding categories in the table. Observing the above formula, we can see that mIoU calculates the intersection and union ratio of the predicted and ground truth regions. TP is the intersection of the two sets, and TP, FP, and FN are the union of the two sets. mIoU intuitively reflects the degree of overlap between the predicted and ground truth regions and is a core evaluation metric in semantic segmentation.
[0048] Furthermore, semantic segmentation and underwater saliency segmentation of crack images both fall under binary classification tasks. In the semantic segmentation task of crack images, most cracks, due to their morphological features, occupy no more than 20% of the entire image. This means that positive examples (pixels labeled as cracks) account for less than 20% of the entire image, which is a serious class imbalance problem. To address this issue, the F1 score has emerged as an evaluation metric. The F1 score is influenced by precision and recall, and is defined as the harmonic mean of precision and recall, comprehensively reflecting the model's predictive ability for the small proportion of positive samples. The formula is as follows:
[0049]
[0050] Where P and R represent the precision and recall of the positive class, respectively, as shown in the following formula:
[0051]
[0052] Precision represents the proportion of actual positive examples among those predicted as positive by the model; precision reflects the model's accuracy. Recall represents the proportion of correctly predicted positive examples among all positive samples; recall reflects the model's comprehensiveness. The F1 score is obtained by calculating the harmonic mean of precision and recall. The harmonic mean differs from the arithmetic mean; it is more sensitive to low values. Only when both precision and recall are high will the F1 score approach 1. This allows the F1 score to effectively balance the costs of both type I and type II errors (FP and FN).
[0053] In this set of examples, the model's ability to segment underwater cracks is reflected by two metrics: mIoU and F1 score. A dataset of 300 images of underwater structural cracks was obtained by using an AUV equipped with an optical camera to photograph underwater cracks in dam bridges. The model was validated on this dataset, and the resulting data is shown in Table 2.
[0054] Table 2 Comparison of ablation experimental indices based on the benchmark model
[0055]
[0056] Observing Table 2, we can see that in the first group of experiments, the baseline crack segmentation model achieved an mIoU value of 82.67% and an F1 score of 79.00% after underwater crack segmentation learning. The second group of experiments shows the performance of the improved underwater crack segmentation model on the underwater crack dataset. Compared to the baseline model, the two metrics were improved by approximately 0.8% and 1.3%, respectively. The third group of experiments, based on the second group of experiments, adopted a dual-teacher knowledge distillation transfer learning framework. The fourth group of experiments, based on the third group of experiments, adjusted the knowledge distillation loss function control strategy, resulting in improvements in both metrics, ultimately achieving an mIoU value of 84.59% and an F1 score of 81.88%, which are improvements of approximately 2.0% and 3.0% respectively compared to the first group of baseline models.
[0057] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.
Claims
1. An underwater crack segmentation and identification method based on dual-teacher knowledge distillation, characterized in that: By simultaneously performing segmentation predictions on an underwater crack image using both a surface crack segmentation model and an underwater saliency segmentation model, the predicted distributions for two teachers are obtained. Comparing these two predicted distributions with the actual values yields the confidence scores of the two teachers regarding the foreground and background in the crack segmentation. The foreground corresponds to the crack portion in the predicted segmentation image, and the background corresponds to the underwater background portion. The confidence scores of the two teachers regarding the knowledge corresponding to the student model are then extracted; specifically, the foreground confidence score in the first teacher's segmentation prediction image. Corresponding to the knowledge of cracks, the background confidence in the segmentation prediction graph of the second teacher. Based on the corresponding underwater background knowledge, the knowledge of the two teacher models can be fused. After obtaining the corresponding foreground and background confidence scores, normalizing both yields the dual-teacher soft loss factor. The value of .
2. The underwater crack segmentation and identification method based on dual-teacher knowledge distillation according to claim 1, characterized in that: The dual-teaching soft loss factor The specific formula is as follows: 。 3. The underwater crack segmentation and identification method based on dual-teacher knowledge distillation according to claim 1, characterized in that: The specific steps include the following: Step 1: Train the ground crack segmentation model. Input the underwater crack optical image into the ground crack segmentation model to obtain an underwater crack segmentation image with ground crack knowledge. Step 2: After learning from a dataset rich in underwater background knowledge, the underwater saliency segmentation model outputs the segmentation results of the underwater crack image; Step 3: Combining the advantages of the segmentation prediction map with the ground crack knowledge from Step 1 and the underwater background knowledge from Step 2, the student model is guided to improve the underwater crack segmentation accuracy through knowledge distillation.
4. The underwater crack segmentation and identification method based on dual-teacher knowledge distillation according to claim 3, characterized in that: In step 3, the student model suitable for underwater crack segmentation tasks adopts Unet-FCN and introduces the SE attention mechanism. The SE attention mechanism first extracts the data from the convolutional layer. The feature map X of size X is convolved with a 2D spatial kernel to obtain C outputs, which are then reassembled to obtain a feature map U of uniform size, where, This is considered a standard convolution operator, namely: The obtained feature map U is then subjected to global average pooling, compressing the global spatial information into a single channel descriptor. The feature map U of size can be embedded with global information to obtain a feature vector Z containing contextual information. The size of this feature vector is... Each element in the feature vector Z corresponds to the global statistical information of one channel, alleviating the channel dependency problem. The specific calculation formula is as follows: Where Zc represents the C-th element in the compressed feature vector Z. The input feature map is then used to comprehensively capture the dependencies between channels through activation. During this process, the network learns the non-linear relationships between channels and generates corresponding weights for each channel. To meet these criteria, a gating mechanism is constructed using two fully connected layers. The first fully connected layer performs dimensionality reduction, compressing the number of channels from C to... The ReLU function is applied for activation, where r represents the compression ratio. The second fully connected layer restores the number of channels to the input value C by increasing the dimensionality, and then normalizes it using the sigmoid function, as shown in the following formula: in, It is a dimension-reduced weight matrix. It is the ReLU activation function. It is a dimension-reduced weight matrix. It uses the Sigmoid activation function. Finally, the weights of each channel are multiplied channel-by-channel with the input feature map to recalibrate the importance of each channel. The enhanced feature map is then output to the next layer of the network. The specific recalibration formula is as follows: in, This represents the weight of the c-th channel. The c-th channel represents the normalized original feature map.
5. The underwater crack segmentation and identification method based on dual-teacher knowledge distillation according to claim 4, characterized in that: In step 3, during knowledge distillation, the hard loss function, specifically formulated as follows: in, γ represents the model's predicted probability of the true class, and is the focusing parameter. It acts as a balancing factor to regulate the class balance between positive and negative samples.
6. The underwater crack segmentation and identification method based on dual-teacher knowledge distillation according to claim 5, characterized in that: In step 3, during knowledge distillation, the soft loss function, specifically the knowledge guidance provided by the teacher model to the student model, is formulated as follows: Where T is the temperature coefficient, representing the degree of softening of the soft label.
7. The underwater crack segmentation and identification method based on dual-teacher knowledge distillation according to claim 6, characterized in that: In step 3, the specific formula for the student model learning from the dual-teacher model is as follows: in, and These represent the individual KL divergence values between the two teacher and student models, As a dual-teacher soft loss factor, it controls the impact of different teachers on the student model.