Intelligent Diagnostic System and Method for Micro-defects in Porcelain Insulators Based on Scanning Electron Microscopy Detection

Abstract

The application discloses a porcelain insulator micro-defect intelligent diagnosis system and method based on scanning electron microscope detection, and belongs to the technical field of insulator detection. The system comprises an algorithm selection module, a defect detection module and a physical size calculation module. The algorithm selection module selects a corresponding defect detection algorithm according to the contrast and magnification of the scanning electron microscope picture of the porcelain insulator, and automatically selects the optimal defect detection algorithm. The defect detection module detects the hole and quartz particle in the scanning electron microscope picture of the porcelain insulator according to the defect detection algorithm determined by the algorithm selection module, and obtains the pixel size information. The physical size calculation module obtains the actual physical size by extracting the scale information in the scanning electron microscope picture of the porcelain insulator. The application realizes the automation of the porcelain insulator micro-defect detection, solves the problems of low artificial recognition efficiency and low accuracy, and is suitable for different equipment and different magnification detection scenes.

Description

Technical Field

[0001] This invention belongs to the field of insulator testing technology, and relates to an intelligent diagnostic system and method for micro-defects in porcelain insulators based on scanning electron microscopy. Background Technology

[0002] Currently, the diagnosis of micro-defects in porcelain insulators based on scanning electron microscopy (SEM) mainly relies on manual identification of the SEM results by technicians, which is labor-intensive and inefficient. The main indicators reflecting micro-defects in porcelain insulators are porosity and large quartz particles. When the porosity of a porcelain component reaches 10%, its strength decreases by 50% compared to when the porosity is 0. For SEM images of porcelain insulators, porosity can be approximately reflected by the proportion of pores in the microscope image. Excessively large quartz particles can easily cause annular cracks in the surrounding glassy phase during firing, adversely affecting the performance of the porcelain body. Quartz particles larger than 30 micrometers are considered large quartz particles that detrimental to the insulator's performance.

[0003] When manually identifying defects, the large number and irregular shapes of holes in scanning electron microscope (SEM) images of porcelain insulators make manual calculation of hole proportions extremely difficult. For indicators involving large quartz particles, manually identifying and calculating the size of each porcelain insulator image using SEM is also time-consuming and labor-intensive. Therefore, this invention provides an intelligent diagnostic system and method for micro-defects in porcelain insulators based on SEM detection, enabling automated detection of holes and quartz particles in SEM images of porcelain insulators. Summary of the Invention

[0004] This system is suitable for detecting micro-defects on the surface of porcelain insulators using scanning electron microscopy (SEM). This method solves the problems of low efficiency and low accuracy in manual identification of defects in SEM images of porcelain insulators, and realizes automated identification of SEM results, greatly reducing the manual intervention process.

[0005] This invention is achieved through the following technical solution: An intelligent diagnostic system for micro-defects in porcelain insulators based on scanning electron microscopy, comprising:

[0006] The algorithm selection module is configured as follows:

[0007] Category 1: If the contrast of the scanning electron microscope image of the porcelain insulator is higher than the preset contrast threshold and the magnification is lower than the preset magnification threshold, then the hole detection algorithm based on image brightness features is used to detect hole defects, and the quartz particle detection algorithm based on the target detection model is used to detect quartz particles.

[0008] Category 2: Based on the contrast of the scanning electron microscope image of porcelain insulator being lower than the preset contrast threshold and the magnification being lower than the preset magnification threshold: use the hole detection algorithm based on the instance segmentation model to detect hole defects, and use the quartz particle detection algorithm based on the target detection model to detect quartz particles.

[0009] Category 3: If the contrast of the scanning electron microscope image of the porcelain insulator is higher than the preset contrast threshold and the magnification is higher than the preset magnification threshold, then the hole detection algorithm based on image brightness features is used to detect hole defects, and the quartz particle detection algorithm based on the target detection model is used to detect quartz particles.

[0010] Category 4: Based on the fact that the contrast of the scanning electron microscope image of porcelain insulator is lower than the preset contrast threshold and the magnification is higher than the preset magnification threshold, a hole detection algorithm based on the instance segmentation model is used to detect hole defects, and a quartz particle detection algorithm based on the target detection model is used to detect quartz particles.

[0011] The defect detection module is used to detect holes and quartz particles in the scanning electron microscope images of porcelain insulators according to the defect detection algorithm determined by the algorithm selection module, and to obtain their pixel size information.

[0012] The physical size calculation module is used to extract the scale bar annotation information from the scanning electron microscope image of the porcelain insulator and convert the pixel size of the defect into the actual physical size.

[0013] Furthermore, in the algorithm selection module, the formula for calculating the contrast ratio Q is:

[0014] ,in, This represents the number of pixels in a scanning electron microscope image of a porcelain insulator. This represents the brightness value of the pixel in the hole. Q is the sum of the brightness values ​​of all pixels in the image; when Q is less than 2, it is a low-contrast image, and when Q is greater than 2, it is a high-contrast image; low magnification and high magnification are divided by a magnification factor of 500x.

[0015] Furthermore, the hole detection algorithm based on image brightness features is applicable to high-contrast images, including image subject extraction, hole segmentation threshold calculation, and hole instance extraction; wherein image subject extraction determines the boundary between the subject and the labeled portion by finding rows with an average brightness of 0; the hole segmentation threshold is calculated using the histogram. Figure 1 The maximum value of the first derivative corresponds to 1 / 3 of the brightness value; hole instance extraction is achieved by generating a hole mask map, removing individuals smaller than the hole segmentation threshold, and performing maximum connectivity search in the region.

[0016] Furthermore, the hole detection algorithm based on the instance segmentation model is suitable for low-contrast images. It is trained a second time based on the pre-trained instance segmentation model, and the small target sliding window detection technique is used depending on the magnification. When the scanning electron microscope image of the porcelain insulator is at a low magnification, the sliding window detection is enabled.

[0017] Furthermore, the quartz particle detection algorithm based on the target detection model selects different target detection models according to the contrast and determines whether to enable small target sliding window detection according to the magnification; sliding window detection is enabled at low magnification and not enabled at high magnification.

[0018] Furthermore, the physical size calculation module calculates the physical size of a single pixel by: extracting the image annotation portion, detecting the number of pixels W of the scale, recognizing the size value V corresponding to the scale through OCR, and calculating the size of a single pixel F = V / W; the physical size of the defect is calculated by combining the pixel size and the size F of a single pixel.

[0019] This invention also provides an intelligent diagnostic method for micro-defects in porcelain insulators based on scanning electron microscopy, comprising the following steps:

[0020] The scanning electron microscope image of the porcelain insulator is input into the algorithm selection module. The algorithm selection module is based on a deep learning classification model and determines the corresponding defect detection algorithm according to the contrast and magnification.

[0021] The defect detection module calls a predetermined defect detection algorithm to detect the pixel size information of holes and quartz particles in the scanning electron microscope image of the porcelain insulator;

[0022] The physical size calculation module extracts the scale information from the scanning electron microscope image of the porcelain insulator and converts the pixel size of the defect into the actual physical size.

[0023] Furthermore, in the algorithm selection module, the image classification includes four categories: high contrast and low magnification, low contrast and low magnification, high contrast and high magnification, and low contrast and high magnification; corresponding hole detection algorithms and quartz particle detection algorithms are selected for different categories.

[0024] Furthermore, in the defect detection module, hole detection uses an algorithm based on image brightness features for high-contrast images and an algorithm based on instance segmentation models for low-contrast images; quartz particle detection selects different target detection models based on contrast and determines whether to enable small target sliding window detection based on magnification.

[0025] Furthermore, in the physical size calculation process, the area of ​​the hole is calculated by the number of pixels contained in the hole and the area of ​​a single pixel F²; the physical length and width of the quartz particle are calculated by its pixel length and width and the size of a single pixel F.

[0026] The intelligent diagnostic system and method for micro-defects in porcelain insulators based on scanning electron microscopy provided by this invention have the following significant advantages compared with the prior art:

[0027] This invention solves the inefficiency of traditional manual identification of holes and quartz particles in SEM images of porcelain insulators. Through an automated inspection process, technicians are no longer required to calculate the hole ratio or quartz particle size from each SEM image of a porcelain insulator, greatly shortening the inspection cycle and reducing manual labor intensity. It is suitable for large-scale batch inspection scenarios.

[0028] For scanning electron microscope (SEM) images of porcelain insulators from different devices, at different magnifications, and with varying brightness characteristics, an algorithm selection module automatically matches the optimal detection algorithm (such as an algorithm based on image brightness features or an instance segmentation model), avoiding misjudgments caused by subjective experience differences in manual identification. Simultaneously, through optimization using techniques such as histogram analysis, etching operations, and sliding window detection, shadow interference and isolated noise pixels are effectively removed, and features of pores and quartz particles are accurately extracted, improving the consistency and reliability of defect identification.

[0029] The system automatically classifies the contrast (high / low) and magnification (above / below 500x) of scanning electron microscope images of porcelain insulators based on the YOLOv11 classification model. It calls appropriate detection algorithms and parameters (such as small target sliding window technology) for different categories, which can cover the detection needs of different scanning electron microscope equipment and different microscales. It does not require separate debugging for specific scenarios, thus enhancing the versatility and adaptability of the system.

[0030] The physical size calculation module extracts scale information from the scanning electron microscope images of porcelain insulators and converts the pixel size of defects into actual physical sizes (such as hole area, length and width of quartz particles). This provides a quantitative basis for the performance evaluation of porcelain insulators (such as porosity calculation and large quartz particle identification), avoiding errors from manual conversion and ensuring the scientific and practical nature of the test data.

[0031] By employing techniques such as pre-trained model secondary training, improved erosion operation (10 times more efficient than traditional methods), and adaptive sliding window strategy, we can reduce unnecessary computational overhead while ensuring the detection accuracy of small targets (such as tiny holes at low magnification), thus achieving an efficient balance between detection speed and accuracy. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the intelligent diagnostic system for micro-defects in porcelain insulators according to the present invention.

[0033] Figure 2 The same pixel brightness map is used for the surface shadows and holes of the porcelain insulator.

[0034] Figure 3 The image shows the histogram curve of a scanning electron microscope image of a porcelain insulator and the corresponding first derivative curve.

[0035] Figure 4 This is a hole mask diagram.

[0036] Figure 5 The flowchart for generating the hole mask image.

[0037] Figure 6 for Figure 4 A magnified view of a portion of the image.

[0038] Figure 7 for Figure 6 The effect after corrosion.

[0039] Figure 8 Comparison of the hole effect in scanning electron microscope images of porcelain insulators at different magnifications.

[0040] Figure 9 This is a flowchart of the ruler extraction process. Detailed Implementation

[0041] The invention will now be explained in further detail with reference to the accompanying drawings.

[0042] like Figure 1 As shown, the intelligent diagnostic system for micro-defects in porcelain insulators based on scanning electron microscopy includes:

[0043] The algorithm selection module inputs the scanning electron microscope (SEM) images of porcelain insulators into the algorithm selection module for identification. The algorithm selection module selects different defect detection algorithms based on the different features of the SEM images of porcelain insulators, such as content and brightness.

[0044] After receiving the defect detection algorithm type determined by the algorithm selection module, the defect detection module calls the corresponding defect detection algorithm to perform defect detection on the scanning electron microscope image of the porcelain insulator and obtain the pixel size information of different defects.

[0045] After completing defect detection, the physical size calculation module sends the pixel size information of the defect and the scanning electron microscope image of the porcelain insulator into the physical size calculation module to calculate the actual physical size information of the defect.

[0046] The algorithm selection module, based on the YOLOv11 classification and detection model, enables automatic classification of electron microscopy images of porcelain insulator surfaces under different shapes and magnifications. The module comprises two steps: image feature selection and algorithm selection.

[0047] First, scanning electron microscope (SEM) images of porcelain insulators acquired by different devices were analyzed and screened to identify key features affecting the identification method. Through analysis of typical porcelain insulator SEM images, this system extracted two image feature indicators—contrast Q and SEM magnification—to group the porcelain insulator SEM images.

[0048] The contrast ratio Q of scanning electron microscope (SEM) images of porcelain insulators affects the accuracy of hole detection. Different contrast ratios Q require different hole detection algorithms. The formula for calculating the contrast ratio Q is as follows:

[0049] ;

[0050] in, This represents the number of pixels in a scanning electron microscope image of a porcelain insulator. This represents the brightness value of the pixel in the hole. This is the sum of the brightness values ​​of all pixels in the scanning electron microscope image of the porcelain insulator.

[0051] After calculation, SEM images of porcelain insulators with a brightness ratio of less than 2 between the hole area and the porcelain insulator surface are classified as low-contrast images. Images with a brightness ratio greater than 2 between the hole area and the porcelain insulator surface are classified as high-contrast images.

[0052] The magnification of a scanning electron microscope (SEM) indicates the microscopic scale of the current SEM image of a ceramic insulator. Different quartz particle detection methods will be used for different microscopic scales. Based on the analysis of typical samples, the pixel size distribution of quartz particles in the image at different magnifications is shown in Table 1.

[0053] Table 1. Pixel size distribution of quartz particles at different magnifications

[0054]

[0055] As shown in the table above, samples with magnification of 500x or higher contain quartz particles with relatively small pixels (less than 50×50 pixels). Therefore, this system uses 500x magnification as a dividing line to extract two magnification indices: 500x and above, and below 500x, as the selection criteria for the defect detection algorithm.

[0056] If the extraction of the aforementioned image feature indicators uses general image feature extraction methods, there are problems such as implementation difficulties and low efficiency. Image feature extraction methods first require preliminary identification of the hole regions, followed by statistical analysis of the brightness of these regions. Next, the overall brightness of the scanning electron microscope (SEM) image of the porcelain insulator is analyzed to obtain the parameters needed for contrast ratio (Q). Then, the magnification annotation information in the SEM image of the porcelain insulator needs to be extracted to obtain the SEM magnification. The overall calculation process is quite cumbersome.

[0057] The image classification algorithm based on YOLOv11 can distinguish the magnification and contrast of porcelain insulator scanning electron microscope images with only one recognition, which is highly efficient.

[0058] Compared to image feature extraction-based methods, the YOLOv11-based image classification algorithm only needs to perform one detection to obtain the final calculated classification.

[0059] Based on image feature metrics, an algorithm for classifying and grouping images into four categories was constructed. The specific categories and their differences are shown in Table 2.

[0060] Table 2 Classification of System Construction Tasks

[0061]

[0062] After training with YOLOv11 for classification, the training task achieved a very high accuracy.

[0063] By constructing four classification task categories, different defect detection algorithms and their parameters were selected for each category. The specific defect detection algorithm selection strategy is as follows:

[0064] Category 1 (High Contrast, Low Magnification): A hole detection algorithm based on image brightness features is used to detect hole defects, while a quartz particle detection algorithm based on a target detection model (target detection model for the high contrast group) is used to detect quartz particles. When using the quartz particle detection algorithm, the small target sliding window detection technique needs to be enabled.

[0065] Category 2 (low contrast, low magnification): Hole defects are detected using an instance segmentation model-based hole detection algorithm, while quartz particles are detected using a target detection model-based quartz particle detection algorithm (target detection model for low contrast group). When calling the above two detection algorithms, the small target sliding window detection technology needs to be enabled.

[0066] Category 3 (High Contrast, High Magnification): Hole defects are detected using an image brightness feature-based hole detection algorithm. Quartz particles are also detected using a target detection model-based quartz particle detection algorithm (target detection model for the high contrast group).

[0067] Category 4 (low contrast, high magnification): Hole defects are detected using an instance segmentation model-based hole detection algorithm, while quartz particles are detected using a target detection model-based quartz particle detection algorithm (target detection model for low contrast group).

[0068] The defect detection module provides detection algorithms for abnormal holes and quartz particles at different contrasts and magnifications. The detection algorithms are as follows:

[0069] The hole detection algorithm based on image brightness features is suitable for hole detection under high contrast: the hole is extracted by the difference in brightness between the hole and the ceramic part in the scanning electron microscope image of the ceramic insulator.

[0070] The hole detection algorithm based on the instance segmentation model is suitable for hole detection under low contrast: the instance segmentation model is the YOLOv11 semantic segmentation model, which segments the holes. At the same time, considering the morphological differences caused by the difference in magnification of the scanning electron microscope images of porcelain insulators, the hole detection algorithm based on the instance segmentation model will decide whether to use the sliding window detection mechanism according to the magnification.

[0071] The quartz particle detection algorithm based on the target detection model is suitable for detecting quartz particles: it uses the YOLOv11 target detection model to detect quartz particles, while also considering the differences in appearance caused by differences in contrast and magnification. The quartz particle detection algorithm based on the target detection model will select different target detection models according to different contrast, and decide whether to enable sliding window detection according to the magnification.

[0072] The hole detection algorithm based on image brightness features is suitable for image detection in high-contrast images, where hole areas appear black and non-hole areas are brighter. The algorithm consists of three parts: extraction of the main image portion, calculation of the hole segmentation threshold, and extraction of hole instances.

[0073] Image Subject Extraction: Since the scanning electron microscope (SEM) image of a porcelain insulator contains annotation information (a black area with white text), the black background can interfere with the extraction of holes. Therefore, we prioritize extracting only the portion containing the surface features of the porcelain insulator. The SEM image of a porcelain insulator contains both the subject and the annotation portions. First, we extract the subject, which refers to the area showing the surface image of the insulator. The boundary between the subject and the annotation portions has several rows of completely black areas. Therefore, by extracting the average brightness feature of each row in the SEM image, we find the row with an average brightness value of 0 (the boundary is completely black with a brightness value of 0), which represents the boundary between the subject and the annotation portions. Therefore, the process for extracting the subject is as follows:

[0074] Calculate the average brightness value for each row. The formula for calculating the average brightness value of each row is: , where w is the number of pixels in the row. Let be the pixel brightness value in the i-th row and j-th column of the scanning electron microscope image of the porcelain insulator, where i is the row index and j is the column index; read the average brightness value of each row from top to bottom. Find the first row number Z where the average brightness is not 0; extract rows 0 to Z from the scanning electron microscope image of the porcelain insulator, which constitutes the main part of the scanning electron microscope image of the porcelain insulator.

[0075] Hole segmentation threshold calculation: Hole segmentation threshold extraction mainly involves finding the critical point in brightness that distinguishes holes from the surface of the porcelain insulator in a scanning electron microscope image. This brightness threshold allows for quick differentiation between pixels representing holes and pixels representing the porcelain insulator surface. Hole segmentation threshold calculation, which primarily involves finding this critical point in brightness, includes three steps:

[0076] 1. Remove shadows from the surface of porcelain insulators to avoid misjudging shadowed areas as holes;

[0077] 2. Analyze the distribution of pixel counts for each brightness level in the scanning electron microscope images of the insulators using a histogram based on brightness.

[0078] 3. By using histogram statistics, find the threshold for separating the brightness of the hole and the surface of the porcelain insulator.

[0079] Shadow removal on the surface of porcelain insulators refers to the process of identifying holes through brightness differences. First, it's necessary to find the critical point in brightness between the holes and the porcelain insulator surface in a scanning electron microscope (SEM) image. Generally, holes have lower brightness, while the porcelain insulator surface has higher brightness. Theoretically, a specific critical point can be found as the boundary between the hole pixels and the porcelain insulator surface pixels. Then, in actual SEM images of insulators, at the pixel scale, the porcelain insulator surface will also have darker pixels representing its surface unevenness and shadows. For example... Figure 2 As shown, the black pixels within the two red circles at the top actually represent the shadows on the surface of the porcelain insulator, while the red circles at the bottom represent the hole areas. Therefore, these shadows need to be removed before extracting the hole segmentation threshold. This is achieved by using max pooling. The max pooling operation steps are as follows: traverse the scanning electron microscope image of the porcelain insulator pixel by pixel, reading the position of each pixel; for each pixel, access the brightness values ​​of its eight neighboring pixels, and take the maximum value as the brightness of the current pixel; repeat the above operation until all pixels have been traversed. After the above operation, the shadow parts in the scanning electron microscope image of the insulator will be removed.

[0080] Histogram statistics based on brightness refer to: performing histogram statistics on the scanning electron microscope images of porcelain insulators after the first step of shadow removal, and finding the position with the largest first derivative in the histogram. The process is as follows:

[0081] (1) Iterate through each pixel, read its brightness value, and put it into set M.

[0082] (2) Sort the brightness values ​​in M ​​from low to high, and count the number of times each value appears, n, to obtain the distribution of each brightness value. The set N of brightness distributions is the brightness distribution map of the image (i.e., the histogram of image brightness).

[0083] (3) The relationship between set N and brightness L is regarded as a mapping of discrete functions. Calculate the derivative of the discrete function f(L) to obtain Find L within the range of 0-255. maximum value The corresponding brightness value when it occurs .

[0084] (4) The final discrete function and its derivative The distribution is as follows Figure 3 As shown, brightness L is the horizontal axis, the left vertical axis represents the number of pixels, and the right vertical axis is the derivative value. The blue curve is... The red curve is .

[0085] Generally, there is a significant difference in brightness between holes and the surface of porcelain insulators, which is reflected in the histogram as a sudden change in the rate of brightness change. Based on the analysis of a large number of scanning electron microscope images of porcelain insulators, a hole segmentation threshold can be summarized. ,Pick What happened 1 / 3, that is Its segmentation effect is the best.

[0086] Hole Instance Extraction: After distinguishing between hole pixels and surface pixels of the ceramic insulator, it is also necessary to count which hole pixels constitute a single hole. These hole clusters are treated as a single hole instance for subsequent calculation of the physical dimensions (area, length, width, number, etc.) of the holes.

[0087] After obtaining the hole segmentation threshold Next, the morphology of each hole in the scanning electron microscope image of the porcelain insulator needs to be extracted to facilitate subsequent calculation of hole size information (such as area, length, width, etc.). The specific processing steps are as follows:

[0088] 1. Generate a hole mask image: Based on the hole segmentation threshold, generate a hole mask image that is the same size as the original image, with the brightness of the hole position being 255 and the brightness of other positions being 0.

[0089] 2. Small Individual Removal from Hole Mask Image: Remove isolated white pixels (misidentified hole pixels) from the hole mask image.

[0090] 3. Hole Instance Extraction: The local maximum connectivity search algorithm is used to extract individual hole instances.

[0091] Reference Figure 5 The process of generating the hole mask image is as follows:

[0092] (1) Input the original image, read the size information of the original image, and obtain the pixel size (h, w) of the image in the horizontal and vertical directions. h is the number of pixels in the column and w is the number of pixels in the row.

[0093] (2) Based on the above dimensions, create an image mk that is entirely black, i.e., an image with a brightness value of 0.

[0094] (3) Traverse each pixel of the original image from top to bottom and from left to right. .

[0095] (4) Read pixels brightness value .

[0096] (5) If the pixel The brightness value is less than the hole segmentation threshold. Then the image will be mk The pixel at that location is set to white.

[0097] (6) Repeat steps 3-5 until all pixels have been traversed. The final result is a hole mask image, as shown below. Figure 4 As shown.

[0098] After obtaining the hole mask image, examining its details still reveals that some parts of the ceramic piece are identified as holes. This is due to the previously calculated hole segmentation threshold. This is an empirical coefficient obtained through trial and error using multiple images, not a precise value. Therefore, some porcelain pieces may still be identified as having holes. However, these pixels are mostly scattered and small, such as... Figure 6 As shown. Therefore, before extracting hole instances, these small pixels need to be removed. This invention uses an erosion operation to filter out these pixels. Unlike traditional erosion algorithms, this invention combines the removal requirements with a more efficient algorithm. The specific steps are as follows:

[0099] 1. Traverse the hole mask image Q, and record the coordinates (x, y) of all pixels with a brightness of 255 in the mask image, forming a set. .in These are pixel coordinates. This represents the width of the hole mask image, where d is the width index. is the height of the hole mask image, and 'e' is the height index.

[0100] 2. Traversing a collection pixel coordinates Detect the coordinates of its 8 neighboring pixels. Are they all in the set? middle.

[0101] 3. If any adjacent pixel coordinate is not in the set In the middle, the coordinates and the remainder in the set All adjacent elements in the set are put into the set. middle.

[0102] 4. Repeat step 2 until all elements have been traversed.

[0103] 5. Traversing a collection ,for For each element in the image, find the position corresponding to the hole mask Q, and set its color to 0. The final hole mask will look like this. Figure 7 As shown. Compared to traditional erosion operations, the method of this invention is 10 times faster in terms of convolution performance.

[0104] After processing the hole mask image, each hole instance needs to be extracted individually for subsequent hole size calculations. This invention employs a maximum connected component search algorithm to extract hole instances. The specific steps are as follows:

[0105] 1. Traverse each pixel in the hole mask image Q, find all non-zero pixels, record their positions, and form a set. ,in These are pixel coordinates.

[0106] 2. Take any pixel coordinate from set S. and create a new set .

[0107] 3. Add to collection In, and will Remove from set S.

[0108] 4. For pixel coordinates Check the coordinates of its 8 adjacent pixels. Check if the element is in set S. If it is, remove the element from set S and add it to set S. middle.

[0109] 5. For sets Repeat step 4 for each element in the set until no more new elements are added. In the middle. At this time, the set The element recorded is a hole instance.

[0110] 6. Repeat step 2 until set S contains no elements. For each set created during the iteration... This represents the coordinates of all pixels contained in a single hole instance on this image, and the corresponding set of hole instances is... , where t is the number of holes in the hole mask image, and u is the hole number.

[0111] Because the edges of holes are relatively clear (brighter) and their morphological features are more obvious in low-magnification images, hole detection can be performed using an instance segmentation model. The steps of the hole detection algorithm based on the instance segmentation model include:

[0112] 1. Secondary training using a pre-trained model can greatly reduce training time.

[0113] Since instance segmentation is a complex task, training an instance segmentation model from scratch requires massive amounts of high-quality datasets and a long training run to complete a single instance segmentation task. Therefore, to accelerate this process, this embodiment performs secondary training based on the pre-trained model YOLO11s-seg, which can greatly improve training efficiency.

[0114] Pre-trained models are machine learning models that have been pre-trained on large-scale datasets, typically based on deep learning techniques. These models learn common features or patterns from massive public datasets, forming a basic understanding of the data. Pre-trained models for segmentation tasks usually possess the various convolutional features and detection head structures required for segmentation. Therefore, based on pre-trained segmentation models, secondary training often only requires fine-tuning with a small amount of domain-specific datasets to be competent for segmentation tasks in that domain.

[0115] YOLO11s-seg is an instance segmentation model built on the public dataset COCO. Based on the pre-trained model YOLO11s-seg, secondary training was conducted using samples of holes on the surface of porcelain insulators. After only 300 rounds of training, the relevant metrics improved by 60%.

[0116] 2. Adaptive detection based on different magnifications: By combining the characteristics of the magnification of the scanning electron microscope images of porcelain insulators, it is determined whether to use the small target sliding window detection measure, thereby achieving a balance between detection accuracy and speed.

[0117] Because electron microscope images of porcelain insulator surfaces are relatively large (typically 2000x1000 pixels or larger), while the default YOLO 11s-seg model is 640x640 pixels, images are scaled to 640x640 when using this model for detection. For small holes (less than 30x30 pixels), scaling will result in the loss of their appearance features, leading to missed detections. This is especially true for images at low magnification (such as...). Figure 8 (As shown). Therefore, this system has made the following adjustments based on the original YOLO11s-seg model:

[0118] The model size was adjusted to 1088x1088, which ensures both the detection accuracy for low-magnification images and the detection speed for high-magnification images.

[0119] The small target sliding window detection technique is employed for detection. Small targets typically refer to objects that occupy a small pixel area in an image (e.g., less than 50×50 pixels), have blurred details, and are easily disturbed by the background (e.g., vehicles in drone images, ships in satellite images). Because small targets lack sufficient texture and structural information, deep learning features are difficult to extract effectively. The core of the small sliding window detection technique is to locate tiny targets in complex scenes through dense sampling.

[0120] This invention employs an adaptive detection technique based on different magnification levels. Since small object detection reduces detection speed (by more than 20 times compared to typical segmentation tasks), this invention uses the algorithm selection module described earlier to detect the image magnification level and dynamically determine whether to enable small object recognition. First, a YOLOv11-based image classification algorithm is called to determine the image classification type, which then determines whether to enable small window detection. If the classification result is category 1 or category 2 (i.e., magnified images), then sliding window detection is enabled, and object detection is performed.

[0121] 3. Hole Instance Extraction: Since each hole instance detected by instance segmentation is a binary mask image, it is necessary to further convert the binary mask image into a set of hole coordinates for subsequent hole size calculation.

[0122] Each hole instance detected by the instance segmentation model is represented by a binary mask. The size of the binary mask is the same as the original image size, and the positions of all pixels with a brightness value of 1 in the binary mask are the positions of the holes in the original image. Therefore, after detection based on the instance segmentation model, the following steps are needed to extract the hole instances:

[0123] Suppose that after detecting a single image through instance segmentation, a set of binary mask images is obtained. ,in For the v-th binary mask image, The number of binary mask images; create a collection to record instances of holes. For each binary mask image Record the coordinates of pixels with a brightness of 1 to form a set. ,in It refers to the coordinates of pixels in a binary mask whose brightness value is not 0. The width of the binary mask image. The row index of the binary mask image. The height of the binary mask image. The column index of the binary mask image; Place into the collection of hole instances middle.

[0124] This invention employs target detection to identify quartz particles in scanning electron microscope images of porcelain insulators. Furthermore, due to the significant morphological variations among quartz particles, various methods for expanding the sample size are considered. The specific steps for constructing the quartz particle detection algorithm based on the target detection model include the following:

[0125] 1. Sample expansion based on public datasets: Since quartz particles have similar characteristics to rocks, this system selects a rock dataset from public datasets and mixes it with the current quartz particle dataset to improve its generalization ability.

[0126] 2. Data Sample Expansion Based on AICG: Currently, large models have strong generalization capabilities; therefore, this system employs AICG automatic data expansion technology to increase sample diversity. This system uses the data expansion tool from the Tongyi Qianwen model to expand the number of original samples.

[0127] 3. Target detection of quartz particles based on different classification characteristics.

[0128] The scanning electron microscope (SEM) images of ceramic insulators are classified using a YOLOv11-based image classification algorithm to determine their categories. Different categories require different target detection models and strategies. For example, the main difference between categories 1 and 2, and between categories 3 and 4, is the contrast ratio Q. Quartz particles exhibit significantly different morphological appearances under different contrast ratios Q. If a single model is used to detect these two appearances, it would require a large size of the model and a high degree of precision in distributing the sample data for each morphology. Therefore, this system employs two small-sized models to detect each morphology separately, which is beneficial for model training and improving detection accuracy.

[0129] Furthermore, the main difference between categories 1 and 3, and between categories 2 and 4, lies in the different magnification levels. In the low-magnification SEM images of ceramic insulators, the number of pixels for quartz particles is relatively small. Therefore, a small target detection technique based on a sliding window is needed for detection, rather than a high-magnification array.

[0130] In summary, adaptive detection of quartz particles is achieved based on different contrast ratios (Q) and magnifications. An image classification algorithm based on YOLOv11 is used to detect the contrast level and magnification of scanning electron microscope images of ceramic insulators. Then, different models are selected for target detection based on the contrast level. Simultaneously, the use of small target detection techniques is determined based on the magnification.

[0131] The quartz particle information detected by the quartz particle target detection model is mainly the bounding box information of the quartz particles in the image. The bounding box information includes two sets of parameters: the pixel coordinates (X,Y) of the center point of the quartz particle, and the pixel width and pixel height of the quartz particle.

[0132] This system primarily identifies the size of large quartz particles; therefore, it is also necessary to identify the largest quartz particles in the scanning electron microscope images of porcelain insulators. The main process is as follows:

[0133] For any scanning electron microscope image of a porcelain insulator The detected aggregate of large quartz particles is , where s is the number of quartz particles detected in the scanning electron microscope image of the ceramic insulator. Number the quartz particles. For the first The x-axis of each quartz grain For the first The ordinate of each quartz grain. For the first The width of each quartz grain For the first The height of each quartz grain.

[0134] Read For each element in the array, calculate its diagonal length. Find the length of the diagonal. The longest element is the bounding box of the largest quartz grain in the scanning electron microscope image of the porcelain insulator.

[0135] The physical size calculation module primarily calculates the physical size information of pores and quartz particles. Defects detected by the algorithm only have pixel dimensions (such as pixel length and pixel width). The physical size calculation module extracts scale bar annotation information from the scanning electron microscope images of the porcelain insulator to provide a basis for subsequent calculation of defect size information. Therefore, this module mainly consists of two parts:

[0136] Physical size calculation of a single pixel in the image: Using the scale information in the image, the physical size represented by each pixel in the scanning electron microscope image of the porcelain insulator is calculated.

[0137] The size of the defect is calculated by statistically analyzing the length, width, and area of ​​the defect based on the size of each pixel.

[0138] The steps for calculating the physical size of a single pixel in an image are as follows:

[0139] 1. Extract the labeled portions from the scanning electron microscope images of porcelain insulators: calculate the average brightness value for each row. The formula for calculating the average brightness value of each row is: , where w is the number of pixels in the row. The pixel brightness value is in the i-th row and j-th column of the image; the average brightness value of each row is read from top to bottom. Find the first non-zero row number Z; extract the part from row Z to the last row of the image. This part is the labeled part of the image, denoted as labeled part image B.

[0140] 2. Extract the number of pixels W (i.e., pixel width) of the ruler in the labeled part.

[0141] 3. Extract the numerical value V of the dimension information from the labeled section.

[0142] 4. Calculate the size of a single pixel: F = V / W.

[0143] The ruler in the image is characterized by a continuous white line. Therefore, this feature needs to be utilized for extraction. The detection process is as follows: Figure 9 As shown, the specific testing steps are as follows:

[0144] 1. Input the labeled image B, and create a variable. This is used to record the number of the longest consecutive white pixels in the horizontal direction in the labeled image B. Initially... .

[0145] 2. Create a variable Used to record the starting X coordinate (i.e., column number) of the longest consecutive white pixel, initially .

[0146] 3. Iterate through the pixels of the labeled portion of image B line by line.

[0147] 4. For the nth row, create a variable for that row. This is used to record the longest consecutive white pixel count in the row, initially... .

[0148] 5. Read the pixel P in the nth row and ith column from left to right. i The brightness of P i Is the brightness 255?

[0149] 6. If P i If the brightness is 255, then Return to step 4.

[0150] 7. Otherwise, check if T is greater than W. If yes, execute the following steps in sequence: And assign the value of T to W (W=T), and finally reset T to 0 (T=0). If not, return to step 4.

[0151] 8. Repeat steps 4-7 until all rows have been traversed. At this point, W is the number of pixels on the ruler in the image. G is the X coordinate of the starting point of the ruler.

[0152] Since the size information is located below the ruler, the size information of the ruler can be obtained by extracting the image below the ruler and performing OCR recognition. The steps are as follows: Extract columns G to (G+W) of the labeled image B; obtain the text image R containing the size information; use the OCR recognition algorithm on image R to extract the text and obtain the size value V.

[0153] The area of ​​a hole defect can be calculated by multiplying the number of pixels contained within the defect by F. 2 We obtain the coordinates. Specifically, for a single hole instance, since its coordinate set was obtained previously. Area of ​​the hole For the length and width of a quartz grain, the physical dimensions of the scale can be obtained using the number of pixels W in the horizontal direction and the number of pixels H in the vertical direction. And the physical dimension width to the scale .

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described in the present invention. Therefore, although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention. All technical solutions and improvements that do not depart from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A porcelain insulator micro-defect intelligent diagnosis system based on scanning electron microscope detection, characterized in that, The application relates to a ceramic insulator defect detection method and device. The algorithm selection module is configured to: Category 1: if the contrast of the scanning electron microscope picture of the ceramic insulator is higher than a preset contrast threshold value and the scanning electron microscope magnification is lower than a preset scanning electron microscope magnification threshold value, a hole detection algorithm based on image brightness features is used to detect hole defects, and a quartz particle detection algorithm based on a target detection model is used to detect quartz particles; Category 2: if the contrast of the scanning electron microscope picture of the ceramic insulator is lower than the preset contrast threshold value and the scanning electron microscope magnification is lower than the preset scanning electron microscope magnification threshold value, a hole detection algorithm based on an instance segmentation model is used to detect hole defects, and the quartz particle detection algorithm based on the target detection model is used to detect quartz particles; Category 3: if the contrast of the scanning electron microscope picture of the ceramic insulator is higher than the preset contrast threshold value and the scanning electron microscope magnification is higher than the preset scanning electron microscope magnification threshold value, the hole detection algorithm based on the image brightness features is used to detect hole defects, and the quartz particle detection algorithm based on the target detection model is used to detect quartz particles; Category 4: if the contrast of the scanning electron microscope picture of the ceramic insulator is lower than the preset contrast threshold value and the scanning electron microscope magnification is higher than the preset scanning electron microscope magnification threshold value, the hole detection algorithm based on the instance segmentation model is used to detect hole defects, and the quartz particle detection algorithm based on the target detection model is used to detect quartz particles; The hole detection algorithm based on the image brightness features comprises picture main body part extraction, hole segmentation threshold value calculation and hole instance extraction. The calculation formula of contrast Q in the algorithm selection module is: wherein, represents the pixel of the hole in the scanning electron microscope picture of the porcelain insulator, is the brightness value of the hole pixel, is the sum of the brightness values of all pixels in the picture; the scanning electron microscope magnification represents the microscale of the current scanning electron microscope picture of the porcelain insulator; The hole detection algorithm based on the instance segmentation model is based on a pre-trained instance segmentation model YOLO11s-seg and is secondarily trained, and whether small target sliding window detection technology is adopted is determined according to the scanning electron microscope magnification. The quartz particle detection algorithm based on the target detection model selects different target detection models according to the contrast and determines whether to start small target sliding window detection according to the scanning electron microscope magnification. The defect detection module is used for detecting holes and quartz particles in the scanning electron microscope picture of the ceramic insulator according to the defect detection algorithm determined by the algorithm selection module and obtaining pixel size information of the holes and the quartz particles. The physical size calculation module is used for extracting scale mark information in the scanning electron microscope picture of the ceramic insulator and converting the pixel size of the defects into actual physical size. In the algorithm selection module, when the contrast Q is less than 2, the picture is a low-contrast picture, and when the contrast Q is greater than 2, the picture is a high-contrast picture; 500 times of scanning electron microscope magnification is taken as a boundary to divide low scanning electron microscope magnification and high scanning electron microscope magnification.

2. The porcelain insulator micro-defect intelligent diagnosis system according to claim 1, characterized in that, The hole detection algorithm based on the image brightness features is suitable for high-contrast pictures.

3. The porcelain insulator micro-defect intelligent diagnosis system according to claim 1, characterized in that, The picture main body part extraction is achieved by finding a row with an average brightness of 0 to determine the boundary between the main body and the mark part; the hole segmentation threshold value is 1 / 3 of the brightness value corresponding to the maximum value of the first derivative of a histogram; The hole instance extraction is achieved by generating a hole mask picture, eliminating individuals and regions smaller than the hole segmentation threshold value and realizing region maximum connection search. When the scanning electron microscope picture of the ceramic insulator is low scanning electron microscope magnification, the hole detection algorithm based on the instance segmentation model starts sliding window detection.

4. The micro-defect intelligent diagnosis system for porcelain insulator according to claim 1, characterized in that, ​ 5. The micro-defect intelligent diagnosis system for porcelain insulator according to claim 1, characterized in that, The quartz particle detection algorithm based on the target detection model opens the sliding window detection at a low scanning electron microscope magnification and does not open the sliding window detection at a high scanning electron microscope magnification.

6. The micro-defect intelligent diagnosis system for porcelain insulator according to claim 1, characterized in that, The step of calculating the physical size of a single pixel by the physical size calculation module includes: extracting a picture label part, detecting the number of pixels W of a scale, identifying the size value V corresponding to the scale through OCR, and calculating the size F of a single pixel as F=V / W; and the physical size of a defect is calculated through the pixel size and F.

7. A porcelain insulator micro-defect intelligent diagnosis method based on scanning electron microscope detection, characterized in that, The method comprises the following steps: The scanning electron microscope picture of the porcelain insulator is input into the algorithm selection module, the algorithm selection module determines the corresponding defect detection algorithm according to the contrast and the scanning electron microscope magnification based on the classification detection model of deep learning; Category 1: based on the scanning electron microscope picture of the porcelain insulator, if the contrast is higher than a preset contrast threshold and the scanning electron microscope magnification is lower than a preset scanning electron microscope magnification threshold, the hole detection algorithm based on the image brightness feature is used to detect the hole defect, and the quartz particle detection algorithm based on the target detection model is used to detect the quartz particle; Category 2: based on the scanning electron microscope picture of the porcelain insulator, if the contrast is lower than the preset contrast threshold and the scanning electron microscope magnification is lower than the preset scanning electron microscope magnification threshold, the hole detection algorithm based on the instance segmentation model is used to detect the hole defect, and the quartz particle detection algorithm based on the target detection model is used to detect the quartz particle; Category 3: based on the scanning electron microscope picture of the porcelain insulator, if the contrast is higher than the preset contrast threshold and the scanning electron microscope magnification is higher than the preset scanning electron microscope magnification threshold, the hole detection algorithm based on the image brightness feature is used to detect the hole defect, and the quartz particle detection algorithm based on the target detection model is used to detect the quartz particle; Category 4: based on the scanning electron microscope picture of the porcelain insulator, if the contrast is lower than the preset contrast threshold and the scanning electron microscope magnification is higher than the preset scanning electron microscope magnification threshold, the hole detection algorithm based on the instance segmentation model is used to detect the hole defect, and the quartz particle detection algorithm based on the target detection model is used to detect the quartz particle; The calculation formula of contrast Q in the algorithm selection module is: wherein, represents the pixel of the hole in the scanning electron microscope picture of the porcelain insulator, is the brightness value of the hole pixel, is the sum of the brightness values of all pixels in the picture; the scanning electron microscope magnification represents the microscale of the current scanning electron microscope picture of the porcelain insulator; The hole detection algorithm based on the image brightness feature comprises the steps of extracting a picture main part, calculating a hole segmentation threshold value, and extracting a hole instance; The hole detection algorithm based on the instance segmentation model is based on the pre-trained instance segmentation model YOLO11s-seg and is retrained, and whether to use the small target sliding window detection technology is determined according to the scanning electron microscope magnification; The quartz particle detection algorithm based on the target detection model selects different target detection models according to the contrast and determines whether to open the small target sliding window detection according to the scanning electron microscope magnification; The defect detection module detects the hole and the quartz particle in the scanning electron microscope picture of the porcelain insulator according to the defect detection algorithm determined by the algorithm selection module, and obtains the pixel size information of the hole and the quartz particle; The physical size calculation module extracts the scale label information in the scanning electron microscope picture of the porcelain insulator, and converts the pixel size of the defect into the actual physical size.

8. The method according to claim 7, characterized in that, The hole detection algorithm based on the image brightness feature is suitable for high-contrast pictures. Wherein the picture main body part extraction determines the demarcation line between the main body and the label part by finding the row with average brightness of 0; the hole segmentation threshold is 1 / 3 of the brightness value corresponding to the maximum value of the first derivative of the histogram; The hole instance extraction is realized by generating a hole mask image, eliminating individuals smaller than the hole segmentation threshold, and region maximum connected search.

9. The method according to claim 7, wherein the method further comprises: determining the micro-defects of the porcelain insulator according to the obtained image data. When the scanning electron microscope picture of the porcelain insulator is a low scanning electron microscope magnification, the hole detection algorithm based on the instance segmentation model opens the sliding window detection.

10. The method of claim 7, wherein the method further comprises: The step of calculating the physical size of a single pixel by the physical size calculation module includes: extracting a picture label part, detecting the pixel number W of a scale, identifying the size value V corresponding to the scale by OCR, and calculating the size F of a single pixel = V / W; the physical size of the defect is calculated by the pixel size and F.

Images