Power transmission line strain clamp defect intelligent classification method and system based on two-dimensional image

Abstract

The invention discloses an intelligent classification method and system for defects of a strain clamp of a power transmission line based on a two-dimensional image, and belongs to the field of defect classification of the strain clamp, and the method comprises the steps: carrying out the preprocessing of the two-dimensional image of the strain clamp, extracting related features, and constructing a defect region candidate set; the method comprises the following steps: firstly, acquiring edge information by using an edge detection algorithm, and calculating curvature and roughness to generate a morphological feature vector; then, training a support vector machine model based on the defect sample set, and classifying the first candidate set to obtain a first defect classification result; thirdly, density feature correction is carried out on wire clamp images of different materials, gray gradient features are extracted, a second candidate set is obtained, and the second candidate set is input into the trained SVM model for second defect classification; and finally, integrating the first classification result and the second classification result to generate a final defect classification result. According to the method, morphological characteristics and material characteristics are effectively combined, and the accuracy and robustness of defect detection are improved.

Description

Technical Field

[0001] The present invention belongs to the field of strain clamp defect classification, and in particular relates to a method and system for intelligently classifying transmission line strain clamp defects based on two-dimensional images. Background Art

[0002] Intelligent defect classification of transmission line tension clamps relies on 2D image analysis, but in practice, there is a unique technical challenge: how to distinguish defects with similar density but significantly different morphology. Some defects, such as early corrosion and certain types of wear, may show similar pixel value distribution and grayscale gradient characteristics on 2D images, that is, similar image density. However, their actual morphology may be significantly different. Corrosion may appear as rough patches, while wear may appear as smooth depressions or grooves.

[0003] Due to the complex environment in which the tension clamp is located, factors such as light, rain, and dirt will further interfere with the image quality, making the density features of the defective area more blurred. For example, a thin layer of dust may change the grayscale value of the corroded area, making it more difficult to distinguish from the worn area. In addition, the material and surface treatment process of the tension clamp will also affect the image performance of the defect. Different materials have different reflectivities to light, resulting in different density features on clamps of different materials even for the same defect. This further increases the difficulty of fine classification of defects based on image density. Therefore, how to establish a more effective distinction mechanism between defects with similar density features is a key technical bottleneck in realizing the intelligent classification of transmission line tension clamp defects. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method for intelligent classification of transmission line tension clamp defects based on two-dimensional images, comprising:

[0005] Acquire a two-dimensional image of the tension clamp, and pre-process the two-dimensional image of the tension clamp to obtain a first image;

[0006] Extracting relevant features of the first image, and constructing a first candidate set of defect areas based on the relevant features;

[0007] Extracting edge information of the first candidate set by an edge detection algorithm, and calculating the curvature and roughness of the edge to obtain a morphological feature vector;

[0008] Acquire a defect sample set, train a support vector machine model based on the defect sample set, and obtain a trained support vector machine model;

[0009] Inputting the first candidate set into the trained support vector machine model for calculation to obtain a first defect classification result;

[0010] Density feature correction is performed on the wire clip images of different materials to obtain a second image, and pixel value distribution and grayscale gradient features of the second image are extracted to obtain a second candidate set;

[0011] Inputting the second candidate set and the morphological feature vector into the trained support vector machine model for calculation to obtain a second defect classification result;

[0012] The first defect classification result and the second defect classification result are integrated to generate a final classification result.

[0013] Preferably, the process of obtaining the first image includes: acquiring a two-dimensional image of the tension clamp to obtain an original image, performing image grayscale, normalization and denoising operations on the original image to obtain the first image.

[0014] Preferably, the process of constructing a first candidate set of defect areas based on the relevant features includes:

[0015] For each pixel in the first image, counting the frequency of occurrence of its pixel value to obtain a pixel value distribution histogram;

[0016] The Sobel operator is used to calculate the horizontal and vertical gradients of each pixel in the first image to obtain a gradient image;

[0017] The pixel value distribution histogram and the gradient image are merged, and if the pixel gradient value is greater than a preset threshold and the pixel value is located in an abnormal interval of the pixel value distribution histogram, the pixel is determined to be a candidate pixel;

[0018] Adopting the depth-first search algorithm, adjacent candidate pixels are connected into connected regions to obtain multiple connected region sets;

[0019] The pixel density of each connected region is calculated. If the pixel density of the connected region is greater than a preset density threshold, the region is determined to be a defect candidate region, forming the first candidate set.

[0020] Preferably, the process of obtaining the morphological feature vector includes:

[0021] Calculate the fluctuation degree of the grayscale value of the edge pixel points of the first candidate set to calculate the edge roughness, and calculate the curvature to obtain the curvature value;

[0022] The obtained edge curvature and edge roughness values ​​are combined into a morphological feature vector.

[0023] Preferably, the process of obtaining the trained support vector machine model includes:

[0024] Construct a sample library containing various defect types such as early corrosion and wear, where each sample contains density features and morphological feature vectors;

[0025] A sample feature vector is obtained from a sample library, and the sample feature vector is input into a support vector machine model for training. During the training process, if the density feature vectors of two samples are similar, the difference between the morphological feature vectors is trained to obtain the trained support vector machine model.

[0026] Preferably, the process of obtaining the first defect classification result includes:

[0027] Inputting the first candidate set into the trained support vector machine model to calculate the probability value of the candidate area being corroded or worn;

[0028] Based on the probability value; determine the category of the candidate area, wherein, if the corrosion probability is greater than a preset threshold, the candidate area is marked as a corrosion defect, if the wear probability is greater than the preset threshold and the corrosion probability is less than the wear probability, the candidate area is marked as a wear defect, and the first defect classification result is generated.

[0029] Preferably, the process of obtaining the second candidate set includes:

[0030] Obtain material properties and surface treatment process parameters corresponding to each wire clamp from the wire clamp database;

[0031] An industrial camera is used to shoot wire clamps of different materials and surface treatment processes to obtain a wire clamp image dataset;

[0032] The wire clamp image is preprocessed by using grayscale and histogram equalization methods to obtain a preprocessed wire clamp image;

[0033] The density features of the preprocessed wire clamp image are extracted by gray-level co-occurrence matrix algorithm to obtain the texture information of the wire clamp image.

[0034] According to the material and surface treatment process information of the wire clamp, a mapping relationship model between the material reflectivity and density characteristics is established;

[0035] According to the reflectivity compensation model, the density characteristics of the wire clamp image are corrected to compensate for the reflectivity differences caused by the material and surface treatment process, and the corrected density characteristics are obtained;

[0036] The wire clip image is reconstructed according to the corrected density features to obtain the second candidate set.

[0037] Preferably, the process of obtaining the second defect classification result includes:

[0038] Obtaining pixel value distribution of the second image: calculating the number of occurrences of each pixel value of the second image, constructing a pixel value distribution histogram, and obtaining pixel value distribution characteristics;

[0039] Obtaining the grayscale gradient features of the second image: using the Sobel operator to calculate the horizontal and vertical gradients of the second image, obtaining the gradient amplitude and direction information, and forming the grayscale gradient features;

[0040] The second defect classification result is obtained based on the pixel value distribution feature and the grayscale gradient feature.

[0041] In order to solve the above technical problems, the present invention also provides a transmission line tension clamp defect intelligent classification system based on two-dimensional images, comprising:

[0042] A first image processing module, used for acquiring a two-dimensional image of the tension clamp, and preprocessing the two-dimensional image of the tension clamp to obtain a first image;

[0043] A first candidate processing module, configured to extract relevant features of the first image and construct a first candidate set of defective areas based on the relevant features;

[0044] A morphological feature vector construction module, used to extract edge information of the first candidate set by using an edge detection algorithm, and calculate the curvature and roughness of the edge to obtain a morphological feature vector;

[0045] A model building module, used to obtain a defect sample set, train a support vector machine model based on the defect sample set, and obtain a trained support vector machine model;

[0046] A first defect classification result generating module, used for inputting the first candidate set into the trained support vector machine model for calculation to obtain a first defect classification result;

[0047] A second candidate processing module is used to perform density feature correction on the wire clip images of different materials to obtain a second image, extract pixel value distribution and grayscale gradient features of the second image, and obtain a second candidate set;

[0048] A second image processing module, used for inputting the second candidate set and the morphological feature vector into the trained support vector machine model for calculation to obtain a second defect classification result;

[0049] A fusion module is used to integrate the first defect classification result and the second defect classification result to generate a final classification result.

[0050] Compared with the prior art, the present invention has the following advantages and technical effects:

[0051] The present invention first preprocesses the acquired two-dimensional image of the tension clamp to eliminate the influence of light and rain dirt, and extracts density features such as pixel value distribution and grayscale gradient to construct a preliminary defect candidate set. Since corrosion and wear defects may be similar in density features, the present invention further extracts morphological features such as edge curvature and roughness of the candidate area, and combines a pre-built database containing corrosion and wear samples to train a support vector machine model to achieve defect differentiation based on morphological features. Taking into account the differences in density features of wire clamps of different materials, the present invention further performs density feature correction based on the material information of the wire clamp, extracts density features again and combines them with morphological features, and uses the trained support vector machine model to perform more accurate defect classification. The present invention significantly improves the accuracy of corrosion and wear defect identification of tension clamps through the fusion of density and morphological features and the correction of material differences. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0053] Figure 1 A schematic diagram of a method flow of an embodiment of the present invention;

[0054] Figure 2 A detection schematic diagram of an embodiment of the present invention;

[0055] Figure 3 is another detection schematic diagram of an embodiment of the present invention;

[0056] Figure 4 Constructing a flow chart for the first candidate set of an embodiment of the present invention;

[0057] Figure 5 Constructing a flow chart for the second candidate set of an embodiment of the present invention;

[0058] Figure 6 Schematic diagram of system modules according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0060] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0061] Embodiment 1

[0062] like Figure 1 As shown, this embodiment provides a method for intelligently classifying defects of power transmission line tension clamps based on two-dimensional images, including:

[0063] Step S101, obtaining a two-dimensional image of the tension clamp, preprocessing the image to eliminate the influence of changes in lighting conditions and interference from rain and dirt, including operations such as image grayscale, normalization and denoising, to obtain a first image.

[0064] A two-dimensional image of the tension clamp is obtained to obtain an original image. The original image is converted into a grayscale image to obtain a first image. If the image pixel values ​​are unevenly distributed, histogram equalization is performed to enhance the image contrast and obtain a second image. The second image is normalized to scale the pixel values ​​to a specific range, such as between 0 and 1, to obtain a third image. If the image has salt and pepper noise, a median filter is used to remove the noise to obtain a fourth image. If the image has Gaussian noise, a Gaussian filter is used to remove the noise to obtain a fifth image. According to the characteristics of rainwater and dirt, morphological operations, such as opening or closing operations, are used to remove the interference of rainwater and dirt in the image to obtain a sixth image. An edge detection algorithm, such as the Canny operator, is used to extract edge information in the sixth image to obtain an edge image. Based on the edge image and a pre-established tension clamp template, a template matching algorithm, such as a normalized cross-correlation algorithm, is used to locate the position of the tension clamp in the edge image to obtain a matching result. Whether the tension clamp has defects is determined based on the matching result. If the matching degree is lower than the preset threshold, it is judged that the tension clamp has defects and the defect information is output; if the matching degree is higher than the preset threshold, it is judged that the tension clamp is in normal condition and the normal information is output.

[0065] Furthermore, a two-dimensional image of the tension clamp is obtained, that is, a photo of the tension clamp is taken with a camera. The original image may be in color or grayscale. For example, a color photo of a tension clamp on an outdoor pole tower is taken with a camera, which is the original image. This photo contains background information such as the tension clamp itself, the pole tower, the conductor, and the sky. Converting the original image to a grayscale image can simplify the image data and reduce the amount of calculation. Color images usually contain three color channels: red, green, and blue, while grayscale images have only one channel, and the value of each pixel represents the brightness of the pixel. For example, in the original color image, the tension clamp part may be red, the pole tower is gray, and the sky is blue. After conversion to a grayscale image, these parts all become different grayscale values. If the pixel values ​​of the grayscale image are unevenly distributed, for example, the image is dark or bright as a whole, it will cause the loss of detail information. At this time, histogram equalization is required. Histogram equalization can stretch the pixel value range of the image to make the light and dark contrast of the image clearer. Assume that in the original grayscale image, most of the pixel values ​​are concentrated in the darker area, making the details of the tension clamp unclear. After histogram equalization, the pixel values ​​will be redistributed, making the details of the tension clamp more obvious. Normalizing the grayscale image can scale the pixel values ​​to a specific range, such as between 0 and 1. This can facilitate subsequent image processing algorithms, such as some algorithms that have requirements on the range of input data. Assume that the pixel value range of the grayscale image after histogram equalization is 0 to 255. After normalizing the image to between 0 and 1, each pixel value will be divided by 255. Salt and pepper noise may exist in the image, which appears as randomly appearing black and white points. For example, some pixels in the image become pure black or pure white points due to errors in the camera sensor or transmission process. These noises will affect subsequent image processing and need to be removed using median filtering. Median filtering replaces the pixel value with the median value of the pixels around the pixel point, which can effectively remove salt and pepper noise. Gaussian noise may also exist in the image, which appears as a blurred image. For example, due to insufficient light or camera shake, each pixel value of the image will be subject to a random interference, causing the image to look blurry. Gaussian filtering is to convolve the image with a Gaussian kernel, which can effectively remove Gaussian noise. Due to the influence of the outdoor environment, there may be interference from rain and dirt in the tension clamp image. For example, rain will form water droplets on the surface of the tension clamp, and dirt will form spots on the surface, which will affect the subsequent defect detection. Morphological operations, such as opening operations, can remove small objects in the image, such as raindrops or dirt. Opening operations are operations that first erode and then expand, which can remove objects smaller than the structural element in the image while retaining the overall shape of the image. Edge detection can extract the contour information of objects in the image. For example, the edge of the tension clamp can be detected using the Canny operator. The Canny operator first performs a Gaussian filter on the image, then calculates the gradient of the image, and finally obtains the edge image through non-maximum suppression and double threshold detection.Template matching can be used to locate the position of a specific object in an image. For example, a template image of a tension clamp can be pre-established, and then the normalized cross-correlation algorithm can be used to search for the position in the edge image that is most similar to the template image.

[0066] Step S102 , based on the first image, density-related features such as pixel value distribution and grayscale gradient features are extracted to construct a preliminary defect region candidate set to obtain a first candidate set.

[0067] Step 1: Get the first image. Step 2: Calculate the pixel value distribution of the first image. For each pixel in the first image, count the frequency of its pixel value to obtain a pixel value distribution histogram. Step 3: Calculate the grayscale gradient features of the first image. Use the Sobel operator to calculate the horizontal and vertical gradients of each pixel in the first image to obtain a gradient image. If the gradient value is greater than a preset threshold, the pixel is considered to be an edge point. Step 4: Determine candidate pixels based on the pixel value distribution and grayscale gradient features. Fusion of the pixel value distribution and gradient image obtained in steps 2 and 3. If the pixel gradient value is greater than the preset threshold and the pixel value is in the abnormal interval of the pixel value distribution histogram, the pixel is judged to be a candidate pixel. Step 5: Construct a connected region based on candidate pixels.

[0068] The Sobel operator is an edge detection operator in image processing, which is often used to calculate the gradient of an image and detect edges in the image. The Sobel operator has two directions: horizontal (X direction) and vertical (Y direction), which are used to detect edge changes in images.

[0069] Horizontal Sobel operator (Gx):

[0070]

[0071] Vertical Sobel operator (Gy):

[0072]

[0073] Adopting the depth-first search algorithm, adjacent candidate pixels are connected into connected regions to obtain multiple connected region sets. Step 6: Screen defect candidate regions based on the density characteristics of connected regions. Calculate the pixel density of each connected region. If the pixel density of the connected region is greater than the preset density threshold, the region is judged as a defect candidate region and constitutes the first candidate set. Step 7: Obtain the first candidate set.

[0074] The process of the depth-first search algorithm to identify connected regions includes:

[0075] Initialization: Select a starting pixel as the starting point of the search, usually this point is part of the connected area.

[0076] Mark visits: Create a matrix (or array) of the same size as the image to mark whether each pixel has been visited to avoid duplicate processing.

[0077] Recursive search: Starting from the starting point, recursively check its neighboring pixels (usually four directions, up, down, left, and right, or eight directions, up, down, left, right, and diagonal). If the neighboring pixel has not been visited and meets the connectivity condition (for example, the pixel value is the same or within a certain difference range), it is marked as visited and added to the connected region set.

[0078] Update region information: Each time a pixel is added to a connected region, some statistical information of the region can be updated, such as the total number of pixels in the region, the average pixel value, etc.

[0079] Continue searching: Continue to perform steps 3 and 4 for the pixels newly added to the connected area until no more adjacent pixels meet the conditions.

[0080] Repeat: Repeat the above process for each unvisited pixel in the image until all pixels have been visited.

[0081] Furthermore, obtaining the first image is to obtain the grayscale image of the tension clamp. This can be obtained from the image captured by the camera after grayscale processing. For example, the pixel value of the color image collected by the camera is RGB (100, 150, 200). To convert it into a grayscale value, the formula Gray = 0.299R + 0.587G + 0.114B can be used to calculate the grayscale value to be 142.8. This grayscale image is the first image for subsequent processing. The pixel value distribution of the first image is calculated to understand the frequency of occurrence of different grayscale values ​​in the image. The number of occurrences of each grayscale value can be counted, for example, grayscale value 0 appears 100 times, grayscale value 1 appears 200 times, and so on, to obtain a pixel value distribution histogram. This histogram can reflect the overall brightness and contrast information of the image. If the histogram is concentrated near a certain grayscale value, it means that the image contrast is low. The image contrast can be improved by histogram equalization to make the image details clearer. The grayscale gradient feature of the first image is calculated to find the edge information of the object in the image. The Sobel operator can be used to calculate the gradient value of each pixel. The Sobel operator calculates the gradient through convolution operation. For example, the values ​​of the pixels around a certain pixel are 100, 110, 120, 90, 100, 110, 80, 90, and 100. After calculation by the Sobel operator, the horizontal gradient is 20 and the vertical gradient is 10. If the gradient value is greater than the preset threshold, such as 30, the pixel is considered to be an edge point. Extracting edge information helps to locate the position of the tension clamp later. Fusion of pixel value distribution and gradient image to determine candidate pixels. If the gradient value of a pixel is large and its pixel value is in an abnormal interval (for example, very bright or very dark) in the pixel value distribution histogram, then the pixel is considered to be a candidate pixel for a defect. For example, if the grayscale value of a pixel is 250 (very bright) and its gradient value is greater than the threshold 30, and the overall grayscale value of the image is concentrated around 100, then the pixel may be a candidate pixel for a defect. Doing so can effectively eliminate the interference of background areas and normal areas and improve the accuracy of defect detection. Candidate pixels are usually scattered, and adjacent candidate pixels need to be connected into connected regions. This can be achieved using a depth-first search algorithm. Starting from a candidate pixel, search the eight adjacent pixels around it. If the adjacent pixel is also a candidate pixel, add it to the current connected region, and continue to search the surrounding pixels of this adjacent pixel until no new candidate pixels can be added. In this way, a connected region is obtained. Repeating this process can obtain multiple connected region sets. Connecting scattered candidate pixels into connected regions helps in the subsequent analysis of the shape and size of defects. Not all connected regions are defect regions, some may be noise or other interference. In order to screen out the true defect candidate regions, the pixel density of each connected region can be calculated.Pixel density refers to the ratio of the number of candidate pixels in a connected area to the total number of pixels in the area. If the pixel density of a connected area is very high, it means that the candidate pixels in this area are dense and more likely to be a defective area. For example, if a connected area containing 100 pixels has 80 candidate pixels, then its pixel density is 80%. If the density threshold is set to 50%, then this connected area is a defective candidate area and will be added to the first candidate set. This can effectively remove noise and interference and improve the reliability of defect detection. The first candidate set contains all connected areas that meet the density conditions, which are potential defective areas. Obtaining the first candidate set is a key step in defect detection. These candidate areas can be further analyzed and judged later to finally determine whether the tension clamp has defects. For example, the shape, size, position and other characteristics of the candidate areas can be analyzed, and combined with the prior knowledge of the tension clamp, it can be determined whether these areas are really defects.

[0082] Step S103: for each candidate region in the first candidate set, edge detection algorithm is used to extract edge information thereof, and morphological features such as edge curvature and roughness are calculated to obtain a morphological feature vector.

[0083] Obtain candidate region image: Segment the candidate region of interest from the input image data to generate the first image. Perform edge detection: Use the Canny edge detection algorithm to process the first image to obtain the edge information of the candidate region and generate the second image. If the edge detected by the Canny algorithm is discontinuous, the morphological closing operation is used to connect the edges and generate the second image again. Calculate edge curvature: Use the polynomial fitting method to calculate the curvature of each edge segment in the second image. If the number of edge pixels is less than the preset threshold, the curvature is calculated by linear fitting to obtain the curvature value. If the number of edge pixels is greater than the preset threshold, the curvature is calculated by quadratic polynomial fitting to obtain the curvature value. Calculate edge roughness: Calculate edge roughness according to the degree of fluctuation of the grayscale value of the edge pixels in the second image. Obtain the edge roughness value by calculating the standard deviation of the grayscale value of the edge pixels. Construct morphological feature vector: Combine the obtained edge curvature and edge roughness values ​​into a morphological feature vector. Arrange the curvature value and roughness value of each edge segment in sequence to form a feature vector.

[0084] Furthermore, the purpose of obtaining the candidate region image is to separate the target region of interest from the complex background so as to perform more detailed analysis later. For example, in an image of a circuit board containing electronic components, it is hoped to extract a specific chip, which is the candidate region. There are many methods of segmentation, such as color-based threshold segmentation or shape-based contour extraction. The specific method to be selected needs to be determined according to the characteristics of the image and actual needs. After obtaining the first image, its features can be further analyzed to determine whether it is the target you really want. Edge detection is to obtain the contour information of the candidate region. The Canny edge detection algorithm is a commonly used edge detection algorithm that can effectively suppress noise and retain real edge information. Assuming that a circular candidate region image is being processed now, a clear circular contour should be obtained after Canny edge detection. If the detected edge is discontinuous, such as a break, the morphological closing operation can be used to connect the edges to ensure the integrity of the contour. The closing operation is an operation of dilation followed by corrosion, which can bridge small breaks and make the edge smoother. The second image processed in this way will have more complete edge information, which is convenient for subsequent analysis. The purpose of calculating the edge curvature is to describe the degree of curvature of the edge. For each edge segment in the second image, its curvature can be calculated by polynomial fitting. If the number of edge pixels is small, such as only a few pixels, linear fitting can be used, which is equivalent to fitting this edge segment with a straight line, and the curvature value obtained is zero. If the number of edge pixels is large, such as dozens of pixels, quadratic polynomial fitting can be used, which can more accurately describe the curvature of the edge. For example, for a circular edge, its curvature value should be a constant, while for an elliptical edge, its curvature value will change along the edge. The purpose of calculating edge roughness is to describe the texture information of the edge. Edge roughness can be obtained by calculating the standard deviation of the grayscale values ​​of the edge pixels. For example, a smooth edge has a small change in the grayscale value of its pixels and a small standard deviation, so the roughness is low. A jagged edge has a large change in the grayscale value of its pixels and a large standard deviation, so the roughness is high. Edge roughness can be used to distinguish different types of defects, such as cracks and scratches. The purpose of constructing a morphological feature vector is to combine edge curvature and edge roughness into a vector for subsequent classification. Assuming that the curvature value of a certain edge is calculated to be 0.5 and the roughness value is 0.2, then these two values ​​can be combined into a morphological feature vector [0.5, 0.2]. For multiple edge segments, the curvature value and roughness value of each edge segment can be arranged in sequence to form a longer feature vector. The purpose of training the classifier is to establish a mapping relationship between the morphological feature vector and the candidate region category.

[0085] The expression of Canny edge detection is:

[0086] I smooth (x, y)=I(x, y)*G(x, y, σ);

[0087]

[0088] G(x,y) is the gradient magnitude, and I(x,y) is the original image.

[0089] Step S104, construct a sample library containing various defect types such as early corrosion and wear, wherein each sample contains a density feature and a morphological feature vector, and train a support vector machine model based on the sample library to distinguish defects with similar density but different morphology.

[0090] Obtain sample images containing various defect types such as early corrosion and wear. Extract density features from the acquired sample images, calculate the density value of each sample, and construct a density feature vector. At the same time, extract morphological features, and use Fourier descriptors, wavelet transforms, and other methods to construct a morphological feature vector. Combine the density feature vector and the morphological feature vector into a complete sample feature vector and store it in the sample library. Obtain the sample feature vector from the sample library, and input the sample feature vector into the support vector machine model for training. During the training process, if the density feature vectors of two samples are similar, focus on training the difference between their morphological feature vectors. Through training, a support vector machine model that can distinguish defects with similar density but different morphology is obtained. Obtain the image to be tested, extract the density features and morphological features of the image to be tested, and construct a feature vector of the image to be tested. Input the feature vector of the image to be tested into the trained support vector machine model for prediction. The support vector machine model determines the defect type of the image to be tested based on its feature vector. If the density feature of the image to be tested is similar to the density feature of a certain type of defect, the model makes a more detailed judgment based on the morphological feature to distinguish defects with similar density but different morphology. Obtain the prediction result. If the prediction result indicates that the image to be tested belongs to a certain type of defect, then the defect type and the corresponding confidence level are output. The severity of the defect is determined based on the defect type and confidence level. If the confidence level is higher than the preset threshold, the defect type is determined, otherwise the image to be tested is marked as "undetermined".

[0091] Furthermore, if Figure 2-3As shown in the figure, corrosion and wear are common types of defects in industrial production, which will affect the quality and service life of products. In order to effectively detect and distinguish these defects, it is necessary to combine density and morphological features for analysis. Density features reflect the degree of mass concentration in the defect area. For example, if corrosion occurs on a metal surface, the metal density in the corroded area will decrease. The average grayscale value of the defect area can be calculated by image processing technology and converted into a density value. Assuming that the average grayscale value of the normal metal surface is 200 and the average grayscale value of the corroded area is 100, the density value of the corroded area can be set to 0.5. The density value calculation method of the wear area is similar. By calculating the density value of each sample, a density feature vector can be constructed. Morphological features reflect the geometric information of the defect, such as shape and texture. Fourier descriptors can be used to describe the contour shape of the defect. For example, the Fourier descriptor of a circular corrosion will show different characteristics from that of a square wear. Wavelet transform can be used to analyze the texture information of the defect. For example, a rough wear area will show high-frequency components after wavelet transform, while a smooth corrosion area will show low-frequency components. Combining these morphological features can construct a morphological feature vector. Combining the density feature vector and the morphology feature vector into a complete sample feature vector and storing it in the sample library can provide a data basis for subsequent defect classification. For example, the feature vector of a sample may contain a density value of 0.6 and a series of Fourier descriptors and / or wavelet coefficients that describe its shape and texture. By training the sample feature vector using the support vector machine model, a mapping relationship between defect type and feature vector can be established. During the training process, if the density feature vectors of two samples are similar, for example, their density values ​​are both close to 0.5, the model will pay more attention to the difference between their morphology feature vectors. This enables the model to distinguish defects with similar density but different morphology, such as distinguishing a circular corrosion from a square wear with the same density value. In actual detection, it is first necessary to extract the density features and morphology features of the image to be tested and construct the feature vector of the image to be tested. For example, the density value of the image to be tested is 0.55, and the Fourier descriptor and / or wavelet coefficients also need to be calculated. Then the feature vector is input into the trained support vector machine model for prediction. The support vector machine model will determine the defect type of the image to be tested based on its feature vector. If the density features of the image to be tested are similar to the density features of a certain type of defect, for example, both are close to 0.5, the model will further make a more detailed judgment based on the morphological features. For example, if the morphological features of the image to be tested are closer to the morphological features of circular corrosion, the model will judge it as circular corrosion. The prediction result will give the defect type and the corresponding confidence level. For example, the model may predict that the image to be tested is circular corrosion with a confidence level of 0.9. If the confidence level is higher than the preset threshold, such as 0.8, the defect type is determined to be circular corrosion. Otherwise, the image to be tested is marked as "undetermined".Based on the defect type and confidence level, the severity of the defect can be judged. For example, if it is determined to be circular corrosion and the confidence level is high, its severity can be judged based on indicators such as the area or depth of the corrosion. Doing so can effectively distinguish different types of defects, even if they are similar in density, they can be distinguished by morphological characteristics, thereby improving the accuracy and reliability of defect detection.

[0092] Step S105, using the trained support vector machine model to classify each candidate area in the first candidate set, if the morphological features of the candidate area are more similar to the corrosion sample, it is marked as a corrosion defect; if the morphological features of the candidate area are more similar to the wear sample, it is marked as a wear defect.

[0093] Obtain the first candidate set image data and the pre-trained support vector machine model. The support vector machine model training data includes corrosion sample image data and wear sample image data, and the training has been completed. For each candidate area image in the first candidate set, extract its morphological features. The extracted morphological features include: perimeter, area, circularity, rectangularity, eccentricity, Hu invariant moment, etc. Convert the extracted morphological feature data into the input vector of the support vector machine model. Use the trained support vector machine model to classify and predict the converted morphological feature vector. The support vector machine model outputs the probability value of the candidate area being corrosion or wear. According to the probability value output by the support vector machine model, determine the category of the candidate area. If the corrosion probability is greater than the preset threshold, the candidate area is marked as a corrosion defect. If the wear probability is greater than the preset threshold, and the corrosion probability is less than the wear probability, the candidate area is marked as a wear defect. A clustering algorithm is used to cluster the candidate areas marked as corrosion defects.

[0094] Furthermore, if Figure 4-5As shown, the first candidate set image data is obtained, for example, the image data of the workpiece to be inspected is obtained by non-destructive testing equipment (such as ultrasound, X-ray, etc.), and these images may contain multiple defects such as corrosion and wear. A pre-trained support vector machine model, for example, a model trained using a large amount of known corrosion and wear sample image data, can distinguish corrosion and wear according to the input image features. Since the support vector machine has advantages in processing high-dimensional features and nonlinear classification problems, it is selected as a defect classification model. Morphological features are extracted for each candidate region image in the first candidate set. For example, for a candidate region image, its perimeter can be calculated to be 20 pixels, its area is 150 pixels, its circularity is 0.8, its rectangularity is 0.6, its eccentricity is 0.5, and the values ​​of seven Hu invariant moments. These morphological features describe the shape and geometric characteristics of the candidate region. Extracting morphological features can help distinguish different types of defects. For example, corrosion usually appears as an irregular shape, while wear may appear as a more regular shape. The extracted morphological feature data is converted into an input vector of the support vector machine model. For example, the eigenvalues ​​of perimeter, area, circularity, rectangularity, eccentricity and Hu invariant moment extracted above are combined into a vector [20, 150, 0.8, 0.6, 0.5, ...]. This vector is the input of the support vector machine model. The reason for converting into vector form is that the input of the support vector machine model needs to be a vector of fixed dimension. Use the trained support vector machine model to classify and predict the converted morphological feature vector. For example, the feature vector [20, 150, 0.8, 0.6, 0.5, ...] is input into the trained support vector machine model, and the model outputs a corrosion probability of 0.9 and a wear probability of 0.1. These probability values ​​indicate the possibility that the candidate area belongs to corrosion or wear. According to the probability value output by the support vector machine model, the category of the candidate area is judged. The preset threshold can be adjusted according to the actual situation. For example, the corrosion probability threshold is set to 0.8 and the wear probability threshold is set to 0.7. For the above example, since the corrosion probability of 0.9 is greater than 0.8, the candidate area is marked as a corrosion defect. If the probability of corrosion is 0.7 and the probability of wear is 0.8, the candidate area is marked as a wear defect. Setting a threshold can control the accuracy and sensitivity of defect recognition. A clustering algorithm is used to cluster candidate areas marked as corrosion defects. For example, the K-means algorithm is used to cluster candidate areas marked as corrosion. The distance between the center points of the candidate areas is calculated. If the distance between the center points of two candidate areas is less than a preset threshold (for example, 10 pixels), the two candidate areas are clustered into one cluster. Clustering can merge scattered corrosion areas together, so as to better identify and distinguish different corrosion areas. For example, multiple small corrosion points can be clustered into a large corrosion area.According to the clustering results, count the number of candidate regions contained in each cluster, obtain the average morphological features of each cluster, and calculate the area of ​​each cluster. For example, a cluster contains 10 candidate regions, with an average area of ​​200 pixels and an average circularity of 0.7.

[0095] Step S106, performing specific density feature correction on the wire clip images of different materials according to information such as the wire clip material and surface treatment process, for example, compensating for the reflectivity difference of different materials, to obtain a second image.

[0096] Furthermore, the material information and surface treatment process information of the wire clamp are obtained in order to establish a reflectivity compensation model, thereby eliminating the influence of different materials and surface treatment processes on the image density characteristics.

[0097] The expression of the reflectivity compensation model is:

[0098] I(x,y)=k d ·L(x,y)·cos(θ)+k s L(x, y) (cos(α)) n ;

[0099] Where: kd is the diffuse reflectance (usually indicating the diffuse reflectance of an object). ks is the specular reflectance (indicating the specular reflectance of an object's surface). θ is the angle between the incident light and the surface normal vector. α is the angle between the incident light and the reflected light. n is the glossiness index, which controls the sharpness of the specular reflection.

[0100] For example, if a wire clip is made of copper and the surface treatment process is nickel plating, then it is necessary to obtain parameters such as the reflectivity of copper and the reflectivity of the nickel plating layer from the wire clip database. These parameters will be used to establish a reflectivity compensation model later. To obtain the wire clip image, an industrial camera needs to be used for shooting. Choosing the right camera model, lens, and light source is essential for obtaining high-quality images. For example, an industrial camera equipped with a telecentric lens can be used with a uniform light source environment to shoot wire clips of different materials and surface treatment processes, and the images can be saved in a standard format such as TIFF or PNG. The purpose of preprocessing the wire clip image is to reduce noise and enhance image contrast. Grayscale converts color images into grayscale images, simplifying the subsequent image processing process. Histogram equalization can enhance image contrast and make image details clearer. For example, the pixel values ​​of a wire clip image are concentrated in a certain grayscale range, resulting in an overall dark image. Through histogram equalization, the pixel values ​​can be redistributed to make the image brightness more uniform and the details more prominent. If the texture information of the wire clip image is not clear, histogram equalization is performed again to further enhance the texture details of the image so that density features can be extracted later. The purpose of extracting density features is to quantify the texture information of the wire clamp image. The gray level co-occurrence matrix algorithm is a commonly used texture feature extraction method. It describes the texture characteristics of the image by calculating the spatial relationship between different grayscale pixels in the image. For example, the roughness of the wire clamp surface, texture directionality and other information can be described by calculating the energy, contrast, correlation and other indicators of the gray level co-occurrence matrix. Clear density features are essential for accurately establishing a reflectance compensation model. The purpose of establishing a reflectance compensation model is to eliminate the influence of different materials and surface treatment processes on the image density characteristics. Due to the different reflectivities of different materials and surface treatment processes to light, even the same texture will show different density characteristics on the image. Therefore, a model needs to be established to compensate for this difference. The multivariate linear regression algorithm is a commonly used regression analysis method that can be used to establish a mapping relationship between material reflectivity and density features. For example, the reflectivity of the wire clamp material, the surface treatment process parameters and the extracted density features can be used as input variables to establish a multivariate linear regression model to predict the corrected density features. The purpose of density feature correction is to eliminate the influence of materials and surface treatment processes on image density features. According to the established reflectivity compensation model, the density features of the wire clamp image are corrected to obtain density features that are independent of the material and surface treatment process. For example, for a copper wire clamp made of nickel, the reflectivity compensation model can be used to calculate the corrected density features based on its reflectivity and surface treatment process parameters, thereby eliminating the influence of the material and surface treatment process on the density features. The purpose of generating the second image is to visualize the corrected density features. By inverse grayscale transformation, the corrected density features are converted into pixel values, and the wire clamp image can be reconstructed.

[0101] The expression of gray level co-occurrence matrix is:

[0102]

[0103] P(i, j) is the gray-level co-occurrence matrix, count(·) is a counting operation, indicating the number of pixel pairs that meet the conditions, and I is the pixel.

[0104] Step S107, extracting the pixel value distribution and grayscale gradient features of the second image again, and combining with the morphological feature vector obtained in step 3, using the support vector machine model for classification again to improve the accuracy of defect classification.

[0105] Obtain the pixel value distribution of the second image: Calculate the number of occurrences of each pixel value of the second image, construct a pixel value distribution histogram, and obtain the pixel value distribution feature. Obtain the grayscale gradient feature of the second image: Use the Sobel operator to calculate the horizontal and vertical gradients of the second image, obtain the gradient amplitude and direction information, and form the grayscale gradient feature. Fuse feature vector: Splice the pixel value distribution feature obtained in step 1, the grayscale gradient feature obtained in step 2, and the existing morphological feature vector in step 3 to form a new fused feature vector. Divide the data set: Divide the fused feature vector and its corresponding defect category label into a training set and a test set. Train the support vector machine model: Use the training set data to train the support vector machine model and optimize the model parameters. If the number of training set samples is small, then use the cross-validation method to evaluate the model performance and select the optimal parameters. If the training set sample categories are unbalanced, use oversampling or undersampling technology to balance the sample categories. Test the support vector machine model: Use the test set data to evaluate the classification performance of the trained support vector machine model, and calculate indicators such as classification accuracy, precision, and recall. Output classification results: Use the trained support vector machine model to classify the new defect image and obtain the defect category judgment result.

[0106] Furthermore, the pixel value distribution of the second image is obtained in order to further extract image features and provide richer information for subsequent defect classification. For example, the pixel values ​​of a second image of a wire clamp are mainly concentrated between grayscale values ​​of 100 and 150, indicating that the overall brightness of the image is medium. If the pixel value distribution is relatively uniform, the image contrast is good. Constructing a pixel value distribution histogram can intuitively display the distribution of pixel values. For example, it can be counted that there are 1000 pixels with a grayscale value of 0 and 1200 pixels with a grayscale value of 1. And so on, a histogram is drawn to obtain the pixel value distribution characteristics. The grayscale gradient feature of the second image is obtained to capture the edge information of objects in the image. The edge is usually where defects appear. For example, the Sobel operator is used to calculate the horizontal and vertical gradients of the image. Assuming that the grayscale value of a pixel is 100, the grayscale value of the pixel to its right is 120, and the grayscale value of the pixel below is 90, then the horizontal gradient is 20 and the vertical gradient is -10. The gradient amplitude is √(20 2 +(-10) 2)≈22.36, and the gradient direction can be calculated by arctan(-10 / 20). These gradient information constitute the grayscale gradient features, which can be used to describe the texture changes and edge strength of the image. The fused feature vector combines the image features extracted from different angles to form a more comprehensive feature description. For example, the pixel value distribution feature, grayscale gradient feature and the morphological features extracted in the previous step (such as area, perimeter, circularity, etc.) are spliced ​​into a new feature vector. Assuming that the pixel value distribution feature is a 10-dimensional vector, the grayscale gradient feature is a 5-dimensional vector, and the morphological feature is a 3-dimensional vector, then the fused feature vector is an 18-dimensional vector, which contains richer image information. The purpose of dividing the data set is to evaluate the performance of the classification model. Divide the existing data into training sets and test sets. For example, 80% of the data is used as a training set to train the model, and the remaining 20% ​​is used as a test set to evaluate the generalization ability of the model. This can avoid overfitting of the model and ensure that the model can perform well on new data. The process of training the support vector machine model is to use the training set data to adjust the model parameters so that the model can distinguish between different categories of defects. If the number of training set samples is small, for example, there are only 100 images, then the cross-validation method can be used to divide the training set into several parts, and one of them is used as the validation set in turn, and the rest is used as the training set, so as to make more effective use of limited data. If the training set sample categories are unbalanced, for example, there are 90 Class A defect samples and only 10 Class B defect samples, then the oversampling technology can be used to copy or synthesize Class B defect samples, or the undersampling technology can be used to reduce the number of Class A defect samples, so that the sample category ratio is more balanced and the model is not biased towards the category with a large number. Testing the support vector machine model is to evaluate the generalization ability of the model. Use the test set data to test the trained model and calculate indicators such as classification accuracy, precision, and recall. For example, if the test set has 100 images and the model correctly classifies 90 images, the classification accuracy is 90%. The output classification result is to apply the trained support vector machine model to the new defect image to determine the defect category. For example, a new wire clamp image is input, and after feature extraction and model classification, the model output result is "Class A defect", which indicates that the wire clamp has Class A defects.

[0107] Embodiment 2

[0108] like Figure 6 As shown, this embodiment provides a method for intelligently classifying defects of power transmission line tension clamps based on two-dimensional images, including:

[0109] A first image processing module, used for acquiring a two-dimensional image of the tension clamp, and preprocessing the two-dimensional image of the tension clamp to obtain a first image;

[0110] A first candidate processing module, configured to extract relevant features of the first image and construct a first candidate set of defective areas based on the relevant features;

[0111] A morphological feature vector construction module, used to extract edge information of the first candidate set by using an edge detection algorithm, and calculate the curvature and roughness of the edge to obtain a morphological feature vector;

[0112] A model building module, used to obtain a defect sample set, train a support vector machine model based on the defect sample set, and obtain a trained support vector machine model;

[0113] A first defect classification result generating module, used for inputting the first candidate set into the trained support vector machine model for calculation to obtain a first defect classification result;

[0114] A second candidate processing module is used to perform density feature correction on the wire clip images of different materials to obtain a second image, extract pixel value distribution and grayscale gradient features of the second image, and obtain a second candidate set;

[0115] A second image processing module, used for inputting the second candidate set and the morphological feature vector into the trained support vector machine model for calculation to obtain a second defect classification result;

[0116] A fusion module is used to integrate the first defect classification result and the second defect classification result to generate a final classification result.

[0117] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for intelligent classification of transmission line tension clamp defects based on two-dimensional images, characterized in that: include: Acquire a two-dimensional image of the tension clamp, and pre-process the two-dimensional image of the tension clamp to obtain a first image; Extracting relevant features of the first image, and constructing a first candidate set of defect areas based on the relevant features; Extracting edge information of the first candidate set by an edge detection algorithm, and calculating the curvature and roughness of the edge to obtain a morphological feature vector; Acquire a defect sample set, train a support vector machine model based on the defect sample set, and obtain a trained support vector machine model; Inputting the first candidate set into the trained support vector machine model for calculation to obtain a first defect classification result; Density feature correction is performed on the wire clip images of different materials to obtain a second image, and pixel value distribution and grayscale gradient features of the second image are extracted to obtain a second candidate set; Inputting the second candidate set and the morphological feature vector into the trained support vector machine model for calculation to obtain a second defect classification result; The first defect classification result and the second defect classification result are integrated to generate a final classification result.

2. The method according to claim 1, characterized in that: The process of obtaining the first image includes: acquiring a two-dimensional image of the tension clamp to obtain an original image, performing image grayscale, normalization and denoising operations on the original image to obtain the first image.

3. The method according to claim 1, characterized in that The process of constructing a first candidate set of defect areas based on the relevant features includes: For each pixel in the first image, counting the frequency of occurrence of its pixel value to obtain a pixel value distribution histogram; The Sobel operator is used to calculate the horizontal and vertical gradients of each pixel in the first image to obtain a gradient image; The pixel value distribution histogram and the gradient image are merged, and if the pixel gradient value is greater than a preset threshold and the pixel value is located in an abnormal interval of the pixel value distribution histogram, the pixel is determined to be a candidate pixel; Adopting the depth-first search algorithm, adjacent candidate pixels are connected into connected regions to obtain multiple connected region sets; The pixel density of each connected region is calculated. If the pixel density of the connected region is greater than a preset density threshold, the region is determined to be a defect candidate region, forming the first candidate set.

4. The method according to claim 1, characterized in that: The process of obtaining the morphological feature vector comprises: Calculate the fluctuation degree of the grayscale value of the edge pixel points of the first candidate set to calculate the edge roughness, and calculate the curvature to obtain the curvature value; The obtained edge curvature and edge roughness values ​​are combined into a morphological feature vector.

5. The method according to claim 1, characterized in that The process of obtaining the trained support vector machine model includes: Construct a sample library containing various defect types such as early corrosion and wear, where each sample contains density features and morphological feature vectors; A sample feature vector is obtained from a sample library, and the sample feature vector is input into a support vector machine model for training. During the training process, if the density feature vectors of two samples are similar, the difference between the morphological feature vectors is trained to obtain the trained support vector machine model.

6. The method according to claim 1, characterized in that The process of obtaining the first defect classification result includes: Inputting the first candidate set into the trained support vector machine model to calculate the probability value of the candidate area being corroded or worn; Based on the probability value; determine the category of the candidate area, wherein, if the corrosion probability is greater than a preset threshold, the candidate area is marked as a corrosion defect, if the wear probability is greater than the preset threshold and the corrosion probability is less than the wear probability, the candidate area is marked as a wear defect, and the first defect classification result is generated.

7. The method according to claim 1, characterized in that The process of obtaining the second candidate set includes: Obtain material properties and surface treatment process parameters corresponding to each wire clamp from the wire clamp database; An industrial camera is used to shoot wire clamps of different materials and surface treatment processes to obtain a wire clamp image dataset; The wire clamp image is preprocessed by using grayscale and histogram equalization methods to obtain a preprocessed wire clamp image; The density features of the preprocessed wire clamp image are extracted by gray-level co-occurrence matrix algorithm to obtain the texture information of the wire clamp image. According to the material and surface treatment process information of the wire clamp, a mapping relationship model between the material reflectivity and density characteristics is established; According to the reflectivity compensation model, the density characteristics of the wire clamp image are corrected to compensate for the reflectivity differences caused by the material and surface treatment process, and the corrected density characteristics are obtained; The wire clip image is reconstructed according to the corrected density features to obtain the second candidate set.

8. The method according to claim 1, characterized in that The process of obtaining the second defect classification result includes: Obtaining pixel value distribution of the second image: calculating the number of occurrences of each pixel value of the second image, constructing a pixel value distribution histogram, and obtaining pixel value distribution characteristics; Obtain the grayscale gradient features of the second image: use the Sobel operator to calculate the horizontal and vertical gradients of the second image, obtain the gradient amplitude and direction information, and constitute the grayscale gradient features; obtain the second defect classification result based on the pixel value distribution characteristics and the grayscale gradient features.

9. An intelligent classification system for power transmission line tension clamp defects based on two-dimensional images, characterized in that: include: A first image processing module, used for acquiring a two-dimensional image of the tension clamp, and preprocessing the two-dimensional image of the tension clamp to obtain a first image; A first candidate processing module, configured to extract relevant features of the first image and construct a first candidate set of defective areas based on the relevant features; A morphological feature vector construction module, used to extract edge information of the first candidate set by using an edge detection algorithm, and calculate the curvature and roughness of the edge to obtain a morphological feature vector; A model building module, used to obtain a defect sample set, train a support vector machine model based on the defect sample set, and obtain a trained support vector machine model; A first defect classification result generating module, used for inputting the first candidate set into the trained support vector machine model for calculation to obtain a first defect classification result; A second candidate processing module is used to perform density feature correction on the wire clip images of different materials to obtain a second image, extract pixel value distribution and grayscale gradient features of the second image, and obtain a second candidate set; A second image processing module, used for inputting the second candidate set and the morphological feature vector into the trained support vector machine model for calculation to obtain a second defect classification result; A fusion module is used to integrate the first defect classification result and the second defect classification result to generate a final classification result.

Images