An AI-assisted image data labeling method based on a few samples of a metal surface of a power plant

Abstract

An AI-assisted image data labeling method based on few samples of metal surfaces in power plants, the steps are as follows: obtaining the original image data of the metal surface of the power plant, preprocessing the image to remove noise and interference information, and obtaining a clear metal surface image; according to the extracted feature information, a multi-prototype decoder is constructed, the image features are divided into multiple prototypes through a dynamic clustering algorithm, and the diversity features of the metal surface defects are captured; in the model training process, data enhancement technology is used to rotate, scale, flip and other operations on the original image to generate diversified training samples and improve the generalization ability of the model; according to the detection result, the labeling information of the metal surface defects is generated, the labeling result is compared with the original image, and it is judged whether the detection result meets the preset accuracy requirement; if the detection result does not meet the preset accuracy, the model parameters are adjusted, and the training and detection are performed again until the defect detection result meeting the accuracy requirement is obtained.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to an AI-assisted image data annotation method for metal surfaces with few samples. Background Technology

[0002] In the inspection of metal surfaces in power plants, traditional methods rely on visual inspection by professionals. This approach is not only time-consuming and labor-intensive but also susceptible to subjective factors. With the development of computer vision and deep learning, automated inspection methods have been gradually introduced, but these methods typically require a large amount of labeled data as a training basis. However, data acquisition for metal surfaces in power plants is costly, and the labeling process is complex, resulting in a scarcity of labeled data. How to achieve accurate detection of metal surface defects with limited samples has become a pressing technical problem.

[0003] Existing technologies such as TagLab, while accelerating the manual annotation process, have limited generalization capabilities when dealing with complex and varied metal surface features due to their convolutional neural network (CNN)-based models. This is particularly true when handling special cases such as open curves and internal holes, where existing tools fall short of expectations. Furthermore, TagLab has limitations when processing large images, failing to adapt to larger orthophotos, which further restricts its application in the complex environments of power plants. Summary of the Invention

[0004] The technical problem this invention aims to solve is to provide an AI-assisted image data annotation method for power plant metal surfaces with limited sample data. By introducing a Transformer structure and utilizing its attention mechanism, more effective information exchange can be achieved between different regions, thereby improving segmentation accuracy. Simultaneously, the multi-prototype decoder can flexibly capture the diversity within the image, dynamically clustering image features into different prototypes to better adapt to different parts and variations of the metal surface object. This improvement not only enhances the model's generalization ability but also reduces the dependence on large amounts of labeled data, enabling accurate detection and annotation of metal surface defects even with limited sample data.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A method for AI-assisted image data annotation based on a small number of samples from the metal surface of a power plant, comprising the following steps:

[0007] S1: Acquire raw image data of the metal surface of the power plant, preprocess the image to remove noise and interference information, and obtain a clear image of the metal surface.

[0008] S1.1: Obtain the raw image data of the metal surface of the power plant, and use the Gaussian filtering algorithm to smooth the image and remove high-frequency noise from the image.

[0009] S1.2: The smoothed image is further processed using a median filtering algorithm to eliminate isolated noise points and interference information in the image.

[0010] S1.3: Binarize the filtered image according to the preset threshold to separate the metal surface features from the background area.

[0011] S1.4: Perform morphological opening operations on the binarized image to remove small interference points in the image and retain the main features of the metal surface.

[0012] S1.5: The Canny edge detection algorithm is used to extract the edges of the processed image to obtain the contour information of the metal surface.

[0013] S1.6: Based on the edge extraction results, the metal surface is segmented using a region growing algorithm to obtain independent metal surface regions.

[0014] S1.7: Perform histogram equalization on the segmented metal surface region to enhance image contrast and obtain a clear metal surface image.

[0015] S2: In the preprocessed image, a Transformer-based attention mechanism module is used to extract features from different regions of the image to obtain global and local feature information.

[0016] S2.1: Preferably, the Transformer module is used to process the preprocessed image to obtain different regions in the image.

[0017] S2.2: Feature extraction is performed on the acquired image region through an attention mechanism to obtain region feature information.

[0018] S2.3: Based on the regional feature information, a global feature extraction method is used to obtain the global features of the image.

[0019] S2.4: For global features, a local feature extraction method is used to obtain local features of the image.

[0020] S2.5: Fusion of global and local features to obtain comprehensive feature information of the image.

[0021] S2.6: Based on the comprehensive feature information, identify the key regions in the image and determine the key features of the image.

[0022] S2.7: Use machine learning algorithms to classify the key features of the image and obtain the final classification result of the image.

[0023] S3: Based on the extracted feature information, a multi-prototype decoder is constructed, and the image features are divided into multiple prototypes through a dynamic clustering algorithm to capture the diverse features of metal surface defects.

[0024] S3.1: Preferably, an image of a metal surface defect is acquired, and image features are extracted.

[0025] S3.2: The image features are divided into multiple prototypes using a dynamic clustering algorithm.

[0026] S3.3: Construct a multi-prototype decoder by inputting the segmented features into the decoder.

[0027] S3.4: If a feature belongs to a certain prototype, then determine the defect type of that prototype.

[0028] S3.5: Generate defect classification results based on the determined defect types.

[0029] S3.6: Output the diversity characteristics of metal surface defects based on the classification results.

[0030] S3.7: Update the prototype partitioning of the multi-prototype decoder based on the diversity characteristics of the output.

[0031] S4: Input the segmented prototype features into the few-shot learning model, and fine-tune the model using the pre-trained network parameters and a small amount of labeled data to obtain a defect detection model adapted to few-shot scenarios.

[0032] S4.1: Preferably, sample images from the target dataset are obtained, and feature extraction operations are performed on the sample images to obtain prototype features.

[0033] S4.2: Perform a partitioning operation on the prototype features according to the preset partitioning rules to generate a training feature set.

[0034] S4.3: Obtain network parameters from the pre-trained model and input the training feature set into the few-shot learning model.

[0035] S4.4: Load network parameters into a few-shot learning model and perform model fine-tuning operations using a small amount of labeled data.

[0036] S4.5: Use the gradient descent algorithm to optimize model parameters and update network weights.

[0037] S4.6: Evaluate model performance using cross-validation to determine whether the model accuracy has reached a preset threshold.

[0038] S4.7: If the model accuracy meets the requirements, the final few-sample defect detection model will be output.

[0039] S5: During model training, data augmentation techniques are used to rotate, scale, and flip the original image to generate diverse training samples and improve the model's generalization ability.

[0040] S5.1: Preferably, the original image dataset is obtained, and preprocessing operations are performed on the image features.

[0041] S5.2: Use rotation operations to transform the angle of the image to generate training samples with different viewpoints.

[0042] S5.3: Adjust the image size through scaling operations to obtain training data for multi-scale features.

[0043] S5.4: Use the flip operation to transform the image horizontally or vertically, increasing sample diversity.

[0044] S5.5: Construct an enhanced training sample set based on the preprocessed image data.

[0045] S5.6: Use convolutional neural networks to extract image features and train a deep model.

[0046] S5.7: If the model performs poorly on the validation set, adjust the data augmentation strategy and optimize the model parameters.

[0047] S6: To address the problem of processing large images, an image segmentation strategy is adopted, which divides the large image into multiple small blocks, inputs them into the model for processing, and then stitches the results together to obtain the defect detection results of the overall image.

[0048] S6.1: Preferably, the original large-size image is obtained, and the image is divided into multiple small-size image blocks using an image block segmentation strategy.

[0049] S6.2: Each small image patch is used as input and fed into a pre-built convolutional neural network model for feature extraction.

[0050] S6.3: If there are abnormal feature values ​​in the output feature map of the convolutional neural network, then the region is determined to have a defect.

[0051] S6.4: Generate the corresponding defect marker map based on the defect detection results of each small image block.

[0052] S6.5: Use an image stitching algorithm to seamlessly stitch together the defect marker images of all small image blocks.

[0053] S6.6: If inconsistent detection results occur in the boundary areas during the stitching process, a weighted average method is used for smoothing.

[0054] S6.7: Finally, obtain the complete large-size image of the defect detection result, and output the defect location and type information.

[0055] S7: When dealing with special scenarios such as unclosed curves and internal holes, an edge detection algorithm is introduced. Combined with the defect area output by the model, edge correction and hole filling are performed to improve the accuracy of the detection results.

[0056] S7.1: Preferably, edge information of unclosed curves is extracted from the image using an edge detection algorithm to obtain initial edge data.

[0057] S7.2: Based on the initial edge data and the defect area output by the model, determine the break location of the unclosed curve.

[0058] S7.3: Perform interpolation calculations between fracture locations to generate continuous edge data and complete edge correction.

[0059] S7.4: For the internal hole region, a region growing algorithm is used to expand from the edge of the defect region inward to obtain the hole boundary.

[0060] S7.5: Fill pixels inside the hole boundary, and combine the color information of the surrounding area to achieve hole filling.

[0061] S7.6: The corrected edge data is fused with the filled hole area to generate a complete detection result image.

[0062] S7.7: Based on the complete detection result image, output the final defect area range to complete the detection process.

[0063] S8: Based on the detection results, generate annotation information for metal surface defects, compare the annotation results with the original image, and determine whether the detection results meet the preset accuracy requirements.

[0064] S8.1: Preferably, an image processing algorithm is used to obtain an image of the metal surface, and an edge detection algorithm is used to identify defect areas in the image.

[0065] S8.2: Generate defect annotation information based on the recognition results, and perform pixel-level comparison and analysis between the annotation information and the original image.

[0066] S8.3: If the comparison result matches the preset accuracy threshold, then the detection result is determined to meet the requirements.

[0067] S8.4: If it does not match, adjust the detection parameters and reprocess.

[0068] S8.5: Analyze the shape, size, and location features of the defect area using feature extraction algorithms to construct a defect classification model.

[0069] S8.6: Automatically identify defect types using a classification model and generate defect type labeling information.

[0070] S8.7: Calculate the area ratio of the defect region based on the defect type labeling information to determine the degree of defect.

[0071] S8.8: Match the defect severity with preset standards and output the final test results and accuracy assessment report.

[0072] S9: If the detection result does not meet the preset accuracy, adjust the model parameters, retrain and re-detect until the defect detection result that meets the accuracy requirements is obtained.

[0073] S9.1: Preferably, the defect detection accuracy value output by the detection model is obtained, and the accuracy value is compared with the preset accuracy threshold. If the accuracy value is less than the preset value, the parameter adjustment mechanism is activated.

[0074] S9.2: Use a grid search algorithm to iterate and optimize the model parameter values ​​to obtain the optimal parameter combination.

[0075] S9.3: Retrain the model based on the adjusted parameter values ​​to obtain the updated detection model.

[0076] S9.4: Use the updated model to detect defect samples and obtain new detection accuracy values.

[0077] S9.5: Compare the new detection accuracy value with the preset value. If the accuracy requirement is met, the process is terminated. If not, the parameter adjustment and model training process is repeated.

[0078] S9.6: Use cross-validation during training to evaluate model performance and determine the optimal model version.

[0079] S9.7: The final output is a defect detection model that meets the accuracy requirements.

[0080] The present invention can achieve the following beneficial effects:

[0081] This method first preprocesses the original image, extracts features using the Transformer attention mechanism, and captures the diversity of defects through a multi-prototype decoder. In a few-shot learning model, fine-tuning is performed using pre-trained parameters and a small amount of labeled data to achieve accurate detection of metal surface defects. To address the problem of large image processing, a block-based strategy is adopted, and an edge detection algorithm is introduced to handle special scenarios. Data augmentation techniques are used to improve the model's generalization ability, and iterative optimization ensures that the detection results reach the preset accuracy. This invention effectively solves the problems of scarce samples, large image sizes, and diverse defect types in power plant metal surface defect detection, improving detection efficiency and accuracy, and providing strong support for power plant equipment maintenance. Attached Figure Description

[0082] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0083] Figure 1 This is a flowchart of an AI-assisted image data annotation method for metal surfaces with few samples according to the present invention.

[0084] Figure 2 This is a flowchart illustrating the processing flow of the Transformer module of the present invention. Detailed Implementation

[0085] Preferred solutions include Figures 1 to 2 As shown, an AI-assisted image data annotation method based on a small number of samples from the metal surface of a power plant is described, with the following specific steps:

[0086] S1. Obtain the raw image data of the metal surface of the power plant, preprocess the image to remove noise and interference information, and obtain a clear image of the metal surface.

[0087] Raw image data of the power plant's metal surface was acquired. A Gaussian filter was used to smooth the image, removing high-frequency noise. A median filter was then applied to further process the smoothed image, eliminating isolated noise points and interference. The filtered image was binarized according to a preset threshold to separate the metal surface features from the background. Morphological opening was performed on the binarized image to remove small interference points while preserving the main features of the metal surface. The Canny edge detection algorithm was used to extract edges from the processed image, obtaining the contour information of the metal surface. Based on the edge extraction results, a region growing algorithm was used to segment the metal surface, obtaining independent metal surface regions. Histogram equalization was then applied to the segmented metal surface regions to enhance image contrast and obtain a clear metal surface image. The image preprocessing parameters are shown in Table 1.

[0088] Table 1 Image Preprocessing Parameters

[0089]

[0090] Specifically, raw image data of the power plant's metal surface can be acquired using a high-resolution industrial camera with a resolution of 12 megapixels and a frame rate of 30fps to ensure image detail capture. During image acquisition, uniform illumination technology is employed, with the light source color temperature set to 5000K to reduce the impact of shadows and reflections on image quality. Raw images typically contain noise and interference, thus requiring preprocessing. First, a Gaussian filter algorithm is used to smooth the image, with a filter kernel size of 5x5 and a standard deviation of 5, to remove high-frequency noise. Next, a median filter algorithm is used to further remove salt-and-pepper noise, with a filter window size of 3x3, ensuring effective suppression of fine particle noise in the image. To enhance image contrast, a histogram equalization algorithm is used to expand the image's grayscale levels between 0 and 255, making the texture and defects of the metal surface more clearly visible. Finally, the Canny edge detection algorithm is used to extract edge information from the metal surface, with a low threshold set to 50 and a high threshold set to 150, ensuring the integrity and continuity of edge information. The preprocessed images can clearly show the texture and defects of the metal surface, providing a high-quality data foundation for subsequent image analysis and defect detection.

[0091] S2. In the preprocessed image, a Transformer-based attention mechanism module is used to extract features from different regions of the image to obtain global and local feature information.

[0092] The Transformer module is used to process the preprocessed image and obtain different regions within it. An attention mechanism is then used to extract features from these regions, yielding regional feature information. Based on this regional feature information, a global feature extraction method is employed to obtain the global features of the image. For the global features, a local feature extraction method is then used to obtain the local features of the image. The global and local features are then fused to obtain the comprehensive feature information of the image. Based on this comprehensive feature information, key regions in the image are identified, determining the key features. A machine learning algorithm is then used to classify the key features of the image, obtaining the final classification result. The feature extraction parameters are shown in Table 2.

[0093] Table 2: Feature Extraction Parameter Table

[0094]

[0095] Specifically, in the preprocessed image, bilinear interpolation is first used to adjust the image resolution to 224×224 pixels to meet the input requirements of the subsequent Transformer model. Next, a convolutional neural network (CNN) is used to extract low-level features from the image, employing a 3×3 convolution kernel with a stride of 1 and padding of 1 to generate a feature map. Subsequently, the feature map is divided into 16×16 pixel blocks, each containing an 8×8 feature vector, which serves as the input sequence for the Transformer. In the Transformer's attention mechanism module, a multi-head self-attention mechanism is used, with 8 heads and each head having a dimension of 64. Attention weights for different regions in the image are obtained by calculating the dot product of the query, key, and value; the weight matrix has a dimension of 16×16×8. Through weighted summation, the global feature representation of each pixel block is obtained, while a local attention mechanism is used to extract local features from the 8×8 feature vectors within each pixel block. Finally, global and local feature information are fused, and a fully connected layer is used for classification. The output dimension of the fully connected layer is 1000, corresponding to the number of categories in the ImageNet dataset. The entire process optimizes the model parameters using the backpropagation algorithm, with a learning rate set to 0.01. The Adam optimizer is used for gradient updates, and the number of iterations is 100, to achieve accurate extraction and classification of image features.

[0096] S3. Based on the extracted feature information, construct a multi-prototype decoder and divide the image features into multiple prototypes through a dynamic clustering algorithm to capture the diverse features of metal surface defects.

[0097] Images of metal surface defects are acquired, and image features are extracted. A dynamic clustering algorithm is used to divide the image features into multiple prototypes. A multi-prototype decoder is constructed, and the segmented features are input into the decoder. If a feature belongs to a certain prototype, the defect type of that prototype is determined. Based on the determined defect type, a defect classification result is generated. The diversity features of metal surface defects are output based on the classification result. The prototype division of the multi-prototype decoder is updated based on the output diversity features. The construction parameters of the multi-prototype decoder are shown in Table 3.

[0098] Table 3: Parameters for Building a Multi-Prototype Decoder

[0099]

[0100] Specifically, in constructing the multi-prototype decoder, the first step is to extract features from the input metal surface defect image. A convolutional neural network (CNN) is used as the feature extractor, such as the ResNet-50 model. The input image size is 224×224 pixels. After multiple convolution and pooling operations, a 512-dimensional feature vector is obtained. Next, these feature vectors are divided using a dynamic clustering algorithm. The K-means clustering algorithm is selected, with 5 cluster centers and 100 iterations. Euclidean distance is used as the similarity measure to divide the feature vectors into 5 prototype clusters, each representing a type of defect feature. To capture the diverse features of metal surface defects, each prototype cluster is further analyzed, calculating the mean and variance of the feature vectors within the cluster. For example, the mean of the first prototype cluster is [12, 34, 56, ...], and the variance is [01, 02, 03, ...], which describes the feature distribution of the cluster. Subsequently, the feature information of each prototype cluster is input into a multi-prototype decoder. The decoder employs a fully connected neural network with 512 input layer nodes, 256 hidden layer nodes, and 10 output layer nodes, corresponding to 10 defect categories. Through training, the decoder is enabled to accurately identify and classify different types of metal surface defects. For example, during training, a cross-entropy loss function is used with a learning rate of 0.01 and a batch size of 32. After 1000 iterations, the model achieves an accuracy of 96% on the test set. Ultimately, this multi-prototype decoder effectively captures the diverse features of metal surface defects and achieves high-precision defect classification.

[0101] S4. Input the segmented prototype features into the few-shot learning model, and fine-tune the model using the pre-trained network parameters and a small amount of labeled data to obtain a defect detection model adapted to few-shot scenarios.

[0102] Sample images from the target dataset are acquired, and feature extraction is performed on these images to obtain prototype features. Based on a pre-defined partitioning rule, the prototype features are partitioned to generate a training feature set. Network parameters are obtained from the pre-trained model, and the training feature set is input into the few-shot learning model. The network parameters are loaded into the few-shot learning model, and model fine-tuning is performed using a small amount of labeled data. The gradient descent algorithm is used to optimize the model parameters and update the network weights. The model performance is evaluated using cross-validation to determine if the model accuracy reaches a preset threshold. If the model accuracy meets the requirements, the final few-shot defect detection model is output. The fine-tuning parameters of the few-shot learning model are shown in Table 4.

[0103] Table 4. Parameters for Fine-tuning the Few-Shot Learning Model

[0104]

[0105] Specifically, when the segmented prototype features are input into the few-shot learning model, image features are first extracted using a pre-trained ResNet-50 network trained on the ImageNet dataset, outputting a feature dimension of 2048. Next, the Prototypical Network algorithm is used to calculate the prototype features for each category. Specifically, for each category, the feature vectors of five labeled samples are taken, and their mean is calculated as the prototype feature for that category. For example, for category A, the mean of the five 2048-dimensional feature vectors is calculated to obtain a 2048-dimensional prototype feature vector. Then, fine-tuning is performed using a small amount of labeled data, employing the cross-entropy loss function, a learning rate of 0.01, 100 iterations, and selecting Adam as the optimizer. During fine-tuning, the category to which a query sample belongs is determined by calculating the Euclidean distance between the query sample and the prototype features of each category. For example, for query sample Q, the distances to the prototype features of categories A, B, and C are calculated to be 2, 5, and 8 respectively, thus classifying it as category A. Ultimately, the fine-tuned model was able to adapt to scenarios with few samples, achieving an accuracy of 85% in defect detection tasks with only 10 labeled samples.

[0106] S5. During model training, data augmentation techniques are used to rotate, scale, and flip the original images to generate diverse training samples and improve the model's generalization ability.

[0107] Obtain the original image dataset and perform preprocessing operations on image features. Rotate the images to transform their angles, generating training samples with different viewpoints. Adjust the image size using scaling to obtain training data with multi-scale features. Flip the images horizontally or vertically to increase sample diversity. Construct an augmented training sample set based on the preprocessed image data. Use a convolutional neural network to extract image features and train a deep model. If the model performs poorly on the validation set, adjust the data augmentation strategy and optimize the model parameters.

[0108] Specifically, during model training, data augmentation techniques significantly enhance the model's generalization ability through a series of image transformation operations. Taking rotation as an example, the original image can be rotated at random angles, typically ranging from -30 to 30 degrees. This randomness helps the model learn features from different viewpoints. Scaling, using bilinear interpolation, adjusts the image size to 8 to 2 times the original size, simulating observation effects at different distances and enhancing the model's adaptability to scale changes. Flipping operations include horizontal and vertical flipping. Horizontal flipping is achieved through matrix transposition, while vertical flipping is achieved through mirroring along the y-axis. These two operations generate mirror-symmetric training samples, effectively improving the model's ability to recognize symmetrical features. Furthermore, color dithering adjusts the image's brightness, contrast, and saturation. Brightness is adjusted from 8 to 2, while contrast and saturation are adjusted from 9 to 1, simulating image changes under different lighting conditions and further enhancing the model's robustness in complex environments. Through the comprehensive application of these data augmentation techniques, the diversity of training samples was significantly improved, and the accuracy of the model on the test set increased from 85% to 92%, verifying the effectiveness of data augmentation techniques in improving the generalization ability of the model.

[0109] S6. To address the problem of processing large images, an image segmentation strategy is adopted. The large image is divided into multiple small blocks, which are then input into the model for processing. The results are then stitched together to obtain the defect detection results of the overall image.

[0110] The process begins by acquiring the original large image and segmenting it into multiple smaller image patches using an image segmentation strategy. Each small image patch is then input into a pre-built convolutional neural network (CNN) model for feature extraction. If abnormal feature values ​​are found in the output feature map of the CNN, the region is determined to have a defect. Based on the defect detection results of each small image patch, a corresponding defect marker map is generated. An image stitching algorithm is then used to seamlessly stitch together the defect marker maps of all small image patches. If inconsistencies in the detection results of boundary regions occur during the stitching process, a weighted average method is used for smoothing. Finally, the complete large image with defect detection results is obtained, outputting the defect location and type information.

[0111] Specifically, in large-scale image processing, the 4096×4096 pixel image is first segmented into 256×256 pixel blocks, with a 10-pixel overlap between each block to ensure the integrity of edge information. A sliding window algorithm is used with a window stride of 246 pixels to ensure that all blocks cover the entire image without omission. After segmentation, each block is input into a pre-trained convolutional neural network model for processing. The model uses the ResNet-50 architecture and is fine-tuned for defect detection tasks through transfer learning. The loss function is cross-entropy loss, the optimizer is Adam, and the learning rate is set to 0.01. The model extracts features and classifies each block, outputting a defect probability map. After processing, the defect probability maps of all blocks are stitched together, and a weighted average method is used to fuse overlapping areas. The weights decrease linearly according to the distance of pixels from the overlap boundary to ensure a smooth transition in the stitched image. The final defect detection results for the overall image are as follows: the accuracy of the defect area reaches 95%, the recall rate is 90%, the F1 score is 92%, and the defect location error is less than 5 pixels, which meets the high precision requirements of industrial inspection.

[0112] S7. When dealing with special scenarios such as unclosed curves and internal holes, an edge detection algorithm is introduced. Combined with the defect area output by the model, edge correction and hole filling are performed to improve the accuracy of the detection results.

[0113] Edge detection algorithms are used to extract edge information of unclosed curves from images, obtaining initial edge data. Based on this initial edge data and the defect area output by the model, the break location of the unclosed curve is determined. Interpolation calculations are performed between the break locations to generate continuous edge data, completing edge correction. For internal hole regions, a region growing algorithm is used to expand inward from the defect region edge to obtain the hole boundary. Pixel filling is performed inside the hole boundary, combining the color information of the surrounding area to achieve hole filling. The corrected edge data is fused with the filled hole region to generate a complete detection result image. Based on the complete detection result image, the final defect region range is output, completing the detection process.

[0114] Specifically, when handling special scenarios such as unclosed curves and internal holes, the Canny edge detection algorithm is first used for image preprocessing, with high and low thresholds set to 100 and 200 respectively to ensure edge continuity and accuracy. Using the edge information extracted by the Canny algorithm, combined with the defect region output by the model, the unclosed curve is corrected using morphological closing operations. A 3×3 rectangular structuring element is used for dilation and erosion operations to ensure smooth and closed edges. For internal holes, a region-growing-based hole-filling algorithm is used, with a seed pixel value of 128 and a growth threshold of 50, to gradually fill the hole region. Gaussian filtering is introduced for noise reduction during edge correction and hole filling, with a σ value of 5 to reduce the impact of noise on the detection results. Finally, the accuracy of the detection results is evaluated by calculating the area and perimeter of the corrected region, ensuring that the area error is controlled within 5% and the perimeter error within 3%, thereby improving the overall detection accuracy and reliability.

[0115] S8. Based on the detection results, generate annotation information for metal surface defects, compare the annotation results with the original image, and determine whether the detection results meet the preset accuracy requirements.

[0116] Image processing algorithms are used to acquire images of the metal surface, and edge detection algorithms are used to identify defect areas in the images. Defect annotation information is generated based on the identification results, and this annotation information is compared with the original image at the pixel level. If the comparison result matches a preset accuracy threshold, the detection result is considered satisfactory; otherwise, the detection parameters are adjusted and reprocessed. Feature extraction algorithms are used to analyze the shape, size, and location features of the defect areas to construct a defect classification model. The classification model is used to automatically identify defect types and generate defect type annotation information. Based on the defect type annotation information, the area ratio of the defect region is calculated to determine the degree of defect. The defect degree is matched with preset standards, and the final detection result and accuracy evaluation report are output.

[0117] Specifically, in metal surface defect detection, images of the metal surface are first acquired using a high-resolution industrial camera with a pixel resolution of 5000×3000. The images are processed using the YOLOv5 deep learning-based algorithm. During training, the algorithm used 100,000 labeled images containing defects such as cracks, scratches, and dents, with a training set to test set ratio of 8:2. During model training, the initial learning rate was set to 0.01. After 100 epochs of training, the model achieved a mean accuracy (mAP) of 93% on the test set. During detection, the algorithm identified a linear crack with a length of 17 mm and a width of 3 mm, and two circular dents with diameters of 1 mm and 5 mm, respectively. The system then compared the detection results with preset accuracy standards. The crack length detection error was ±2 mm, and the dent diameter detection error was ±1 mm, both meeting the ±5 mm accuracy standard required for industrial inspection. To further verify the inspection results, the system uses image registration technology to compare the labeled results with the original image. The SIFT feature point matching algorithm is used to calculate the registration error, and the results show that the feature point matching accuracy reaches 97%, indicating that the labeled location highly matches the actual defect location. Finally, the system generates a detailed defect report based on the inspection results, including defect type, location coordinates, and dimensional parameters, and stores this data in a database to provide data support for subsequent quality analysis and process improvement.

[0118] S9. If the detection result does not meet the preset accuracy, adjust the model parameters and retrain and detect until the defect detection result that meets the accuracy requirements is obtained.

[0119] The process involves obtaining the defect detection accuracy value output by the detection model and comparing it with a preset accuracy threshold. If the accuracy value is lower than the preset threshold, a parameter adjustment mechanism is initiated. A grid search algorithm is used to traverse and optimize the model parameters to obtain the optimal parameter combination. The model is then retrained based on the adjusted parameter values ​​to obtain an updated detection model. The updated model is used to detect defect samples to obtain a new detection accuracy value. This new detection accuracy value is compared with the preset threshold. If the accuracy requirement is met, the process terminates; otherwise, the parameter adjustment and model training process is repeated. During training, cross-validation is used to evaluate model performance and determine the optimal model version. Finally, a defect detection model that meets the accuracy requirements is output.

[0120] Specifically, in the defect detection process, an initial image dataset is first acquired, for example, containing 1000 labeled images of industrial parts, of which 500 are normal samples and 500 are defect samples. A convolutional neural network (CNN) is used as the base model, with an initial learning rate of 0.01 and a batch size of 32, trained using the cross-entropy loss function. During training, the model achieves an accuracy of 92% on the training set, but only 85% on the validation set, failing to meet the preset 90% accuracy requirement. Therefore, the model parameters are adjusted: the learning rate is reduced to 0.005, the batch size is increased to 64, and residual connections are introduced into the network to optimize feature extraction. After retraining, the model's accuracy on the validation set improves to 88%, but still does not meet the target. Further analysis reveals a class imbalance problem in the dataset, with a small number of defect samples. Therefore, data augmentation techniques, such as rotation, scaling, and flipping, are used to expand the defect samples to 800, and the Focal Loss function is introduced to alleviate the class imbalance problem. After retraining, the model achieved an accuracy of 91% on the validation set, close to the preset target. Finally, by adjusting the model's regularization parameters, reducing the Dropout rate from 5 to 3, and adding a global average pooling layer, the model's performance was further optimized. Ultimately, the model's accuracy on the validation set stabilized at 93%, successfully meeting the accuracy requirements for defect detection.

[0121] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A method for labeling small-sample AI-assisted image data of metal surfaces, characterized in that, The method includes: Raw image data of the metal surface of a power plant is acquired. The images are preprocessed to remove noise and interference, resulting in clear images of the metal surface. In the preprocessed images, a Transformer-based attention mechanism is used to extract features from different regions, obtaining global and local feature information. Based on the extracted features, a multi-prototype decoder is constructed, using a dynamic clustering algorithm to divide the image features into multiple prototypes, capturing the diverse features of metal surface defects. The segmented prototype features are input into a few-shot learning model, and fine-tuned using pre-trained network parameters and a small amount of labeled data to obtain a defect detection model adapted to few-shot scenarios. During model training, data augmentation techniques are employed to rotate and shrink the original images. The model employs drop and flip operations to generate diverse training samples, enhancing its generalization ability. For large image processing, an image segmentation strategy is used, dividing the large image into multiple smaller blocks, which are then fed into the model for processing. The results are then stitched together to obtain the overall image's defect detection result. When processing unclosed curves and internal holes, an edge detection algorithm is introduced, combined with the defect area output by the model, to perform edge correction and hole filling, improving the accuracy of the detection results. Based on the detection results, annotation information for metal surface defects is generated. The annotation results are compared with the original image to determine if the detection results meet the preset accuracy requirements. If the detection results do not meet the preset accuracy, the model parameters are adjusted, and training and detection are repeated until defect detection results that meet the accuracy requirements are obtained.

2. The method according to claim 1, characterized in that, The process of acquiring raw image data of the power plant's metal surface, preprocessing the image to remove noise and interference, and obtaining a clear image of the metal surface includes: The raw image data of the metal surface of the power plant was acquired, and the image was smoothed by Gaussian filtering algorithm to remove high-frequency noise from the image; The smoothed image is further processed using a median filtering algorithm to eliminate isolated noise points and interference information in the image; The filtered image is binarized according to a preset threshold to separate the metal surface features from the background area. Morphological opening operations are performed on the binarized image to remove small interference points while preserving the main features of the metal surface. The Canny edge detection algorithm is used to extract edges from the processed image to obtain the contour information of the metal surface; Based on the edge extraction results, the metal surface is segmented using a region growing algorithm to obtain independent metal surface regions; Histogram equalization is performed on the segmented metal surface regions to enhance image contrast and obtain a clear metal surface image.

3. The method according to claim 1, characterized in that, In the preprocessed image, a Transformer-based attention mechanism module is used to extract features from different regions of the image, obtaining global and local feature information, including: The Transformer module is used to process the preprocessed image and obtain different regions in the image. Feature extraction is performed on the acquired image regions using an attention mechanism to obtain region feature information; Based on regional feature information, a global feature extraction method is used to obtain the global features of the image; For global features, a local feature extraction method is used to obtain local features of the image; Global and local features are fused to obtain comprehensive feature information of the image; Based on comprehensive feature information, identify key regions in the image and determine the key features of the image; Machine learning algorithms are used to classify key features of images to obtain the final classification results.

4. The method according to claim 1, characterized in that, The process involves constructing a multi-prototype decoder based on the extracted feature information, and using a dynamic clustering algorithm to divide the image features into multiple prototypes to capture the diverse features of metal surface defects, including: Acquire images of metal surface defects and extract image features; The image features are divided into multiple prototypes using a dynamic clustering algorithm; Construct a multi-prototype decoder and input the segmented features into the decoder; If a feature belongs to a certain prototype, then the defect type of that prototype is determined; Based on the determined defect type, generate defect classification results; Based on the classification results, the diversity characteristics of metal surface defects are output; The prototype partitioning of the multi-prototype decoder is updated based on the diversity characteristics of the output.

5. The method according to claim 1, characterized in that, The process involves inputting the segmented prototype features into a few-shot learning model, fine-tuning the pre-trained network parameters using a small amount of labeled data, and obtaining a defect detection model adapted to few-shot scenarios, including: Obtain sample images from the target dataset, perform feature extraction on the sample images, and obtain prototype features; According to the preset partitioning rules, the prototype features are partitioned to generate a training feature set; Obtain network parameters from the pre-trained model and input the training feature set into the few-shot learning model; Load network parameters into a few-shot learning model and perform model fine-tuning operations using a small amount of labeled data; The gradient descent algorithm is used to optimize the model parameters and update the network weights. The model performance is evaluated using cross-validation to determine whether the model accuracy has reached a preset threshold. If the model accuracy meets the requirements, the final few-sample defect detection model will be output.

6. The method according to claim 1, characterized in that, During model training, data augmentation techniques are employed to rotate, scale, and flip the original image, generating diverse training samples to improve the model's generalization ability. This includes: Obtain the original image dataset and perform preprocessing operations on image features; The image is rotated to transform its angle, generating training samples with different viewpoints; By adjusting the image size through scaling operations, training data for multi-scale features can be obtained; Use the flip operation to transform images horizontally or vertically, increasing sample diversity; Based on the preprocessed image data, construct an enhanced training sample set; Image features are extracted using a convolutional neural network, and a deep model is trained. If the model performs poorly on the validation set, adjust the data augmentation strategy and optimize the model parameters.

7. The method according to claim 1, characterized in that, The proposed method for processing large images employs an image segmentation strategy, dividing the large image into multiple smaller blocks, which are then input into the model for processing. The results are then stitched together to obtain the overall image defect detection results, including: The original large-size image is acquired, and the image is divided into multiple small-size image blocks using an image block segmentation strategy. Each small image patch is used as input and fed into a pre-built convolutional neural network model for feature extraction. If there are abnormal feature values ​​in the output feature map of the convolutional neural network, then the region is determined to have a defect. Based on the defect detection results of each small image patch, a corresponding defect marker map is generated; An image stitching algorithm is used to seamlessly stitch together the defect marker images of all small image blocks; If inconsistent detection results occur in the boundary areas during the stitching process, a weighted average method is used for smoothing. Finally, a complete large-size image of the defect detection results is obtained, and the defect location and type information are output.

8. The method according to claim 1, characterized in that, When processing unclosed curves and internal holes, an edge detection algorithm is introduced. Combined with the defect area output by the model, edge correction and hole filling are performed to improve the accuracy of the detection results, including: Initial edge data is obtained by extracting edge information of unclosed curves from an image using an edge detection algorithm. Based on the initial edge data and the defect area output by the model, the break location of the unclosed curve is determined. Interpolation calculations are performed between fracture locations to generate continuous edge data, thus completing edge correction. For the internal cavity region, a region growing algorithm is used to expand from the edge of the defect region inward to obtain the cavity boundary; Pixel filling is performed inside the hole boundary, and the hole filling is achieved by combining the color information of the surrounding area; The corrected edge data is fused with the filled hole areas to generate a complete detection result image; Based on the complete detection result image, the final defect area range is output, completing the detection process.

9. The method according to claim 1, characterized in that, The step of generating annotation information for metal surface defects based on the detection results, comparing the annotation results with the original image, and determining whether the detection results meet the preset accuracy requirements includes: Image processing algorithms are used to acquire images of metal surfaces, and edge detection algorithms are used to identify defect areas in the images. Defect annotation information is generated based on the identification results, and the annotation information is compared and analyzed at the pixel level with the original image. If the comparison result matches the preset accuracy threshold, then the detection result is determined to meet the requirements; If the results are inconsistent, adjust the detection parameters and reprocess. A defect classification model is constructed by analyzing the shape, size, and location of defective regions using feature extraction algorithms. The classification model is used to automatically identify defect types and generate defect type labeling information; Based on the defect type labeling information, calculate the area ratio of the defective region and determine the degree of defect; The defect severity is matched with preset standards, and the final test results and accuracy assessment report are output.

10. The method according to claim 1, characterized in that, If the detection result does not reach the preset accuracy, the model parameters are adjusted, and training and detection are repeated until a defect detection result that meets the accuracy requirements is obtained, including: Obtain the defect detection accuracy value output by the detection model, compare the accuracy value with the preset accuracy threshold, and if the accuracy value is less than the preset value, activate the parameter adjustment mechanism. A grid search algorithm is used to iterate and optimize the model parameter values ​​to obtain the optimal parameter combination. The model is retrained based on the adjusted parameter values ​​to obtain the updated detection model; The updated model is used to detect defect samples to obtain new detection accuracy values; The new detection accuracy value is compared with the preset value. If the accuracy requirement is met, the process is terminated. If not, the parameter adjustment and model training process is repeated. Cross-validation is used during training to evaluate model performance and determine the optimal model version. The final output is a defect detection model that meets the accuracy requirements.

Images