Wire rope surface defect recognition method based on feature fusion

Abstract

The present invention relates to a wire rope surface defect recognition method based on feature fusion, which is in the field of wire rope surface defect recognition technology. The method comprises the following steps: graying a wire rope defect image, dividing the image into blocks, improving a traditional LBP algorithm using a central multi-scale local binary pattern based on the image blocks, extracting texture feature information of the divided image, performing PCA dimensionality reduction based on the obtained image texture features, and finally extracting global image texture features using GLCM and performing feature fusion with the reduced image texture features; and performing wire rope surface defect recognition and classification using an SVM classifier. The method solves the problem that traditional local binary patterns (LBP) are easily affected by central pixels and noise and cannot accurately identify wire rope surface defects. The method achieves an overall recognition rate of 97.5% for wire rope surface defects, which is at least 5% higher than other algorithms. The method can effectively identify various defects on the wire rope surface.

Description

Technical Field

[0001] The present invention belongs to the technical field of wire rope surface defect recognition, and particularly relates to a wire rope surface defect recognition method based on feature fusion. Background Art

[0002] Wire ropes are widely used in various industries, such as coal mine shaft hoists, elevators, and cable cars. However, over time, wire ropes can develop defects such as broken wires, deformation, and missing strands. Their health seriously impacts equipment production and human safety, making research on nondestructive testing of wire ropes extremely valuable and significant. Currently, various methods for nondestructive testing of wire ropes have emerged, including electromagnetic, radiographic, ultrasonic, acoustic emission, and machine vision. However, most technologies remain in the theoretical and experimental stages, lacking an authoritative and rational approach to identify potential future research directions. Currently, wire rope inspection primarily relies on visual inspection and manual feel.

[0003] With the development of image processing technology, Ojala et al. proposed an algorithm for describing local texture features, namely the Local Binary Pattern (LBP). This method has attracted widespread attention in the fields of texture classification and face recognition. Zhao et al. enhanced wire rope images using digital image processing techniques, then obtained continuous information about oil well wire ropes using the integral projection method. Finally, they applied this information to a BP neural network for defect detection. This method can generally meet industrial production requirements. Zhou et al. extracted wire rope texture features using equivalent local binary patterns and compared them with three machine learning algorithms. Jiang and Dong et al. proposed a wire rope detection method that combines CPICS-LBP with HOG features, which effectively improves the description of wire rope texture features. Huang and Yang et al. classified wire rope defects based on the grayscale, grayscale distribution, and grayscale fluctuation characteristics of wire rope surface defects in different regions. Zhang et al. constructed a real-time spatial grayscale sample set using temporal and spatial grayscale similarity to detect wire rope surface defects. Liu and Qin et al. used principal component analysis and a BP neural network to specifically detect the number of broken wires in wire ropes, effectively solving the problem of broken wire count. Dai, Zhao, and others proposed a method for identifying broken wires in hyperspectral images that combines OTSU and Hough transforms. This method uses wire rope diameter pixels to determine wire breakage. Li and others used a template matching method to detect wire rope wear and then calculated the wear area to assess the severity of wire rope wear, which can intuitively understand the wear of elevator wire ropes. Zhou and others proposed a hybrid data-driven method based on texture features and optimized support vector machines. This method adds an inertia weight function to the fruit fly optimization algorithm to achieve a balance between global and local search, which can effectively improve the recognition of wire rope surface defects. Wu and Sun proposed a JLBP texture classification method. This method improves the description of macro textures through multi-scale fusion while maintaining the traditional LBP characteristics. Leila and Mehrez, et al. combined local and global features to fuse wavelet transform features with improved LTP, significantly improving the success rate and computational efficiency.

[0004] While the aforementioned methods have achieved significant progress in wire rope nondestructive testing, they still suffer from a low level of intelligence and rely heavily on traditional image processing. The limited variety and volume of wire rope defect data leads to weak generalization and robustness of the models. Algorithmically, the importance of central pixels is neglected, resulting in the omission of local image details and the failure to fully utilize global image features. Furthermore, the algorithm's recognition accuracy and robustness need further improvement. Summary of the Invention

[0005] The purpose of the present invention is to propose a wire rope surface defect recognition method based on feature fusion to solve the problem that the traditional local binary pattern (LBP) is easily affected by central pixels and noise and cannot accurately identify wire rope surface defects.

[0006] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0007] The wire rope surface defect recognition method based on feature fusion includes the following steps:

[0008] S1, preprocessing: grayscale the wire rope defect image;

[0009] S2, feature extraction and fusion: The image is divided into blocks and the traditional LBP algorithm is improved by the central multi-scale local binary pattern based on the image block to obtain the ISCM-LBP algorithm. The ISCM-LBP algorithm is used to extract the texture feature information of the image after the block. PCA dimensionality reduction is performed based on the obtained image texture features. Finally, the global texture features of the image are extracted through GLCM (i.e., gray-level co-occurrence matrix) and the reduced image texture features are combined for feature fusion;

[0010] S3, identification and classification: Wire rope surface defects are identified and classified using SVM classifier.

[0011] Furthermore, step S2 includes:

[0012] S21, introduces image segmentation and multi-scale into the local binary pattern algorithm to improve detail feature description and reduce the impact of noise;

[0013] S22, uses the standard deviation method to connect the central pixel with the neighboring pixels, and redefines the selection of the central pixel threshold to enhance the robustness of the central pixel and the role of the neighboring pixels.

[0014] Furthermore, in step S21, a method of regional grayscale average value in multi-scale local binary pattern is used to suppress noise and expand the range of texture feature value extraction, thereby compensating for the limitation of small LBP sampling range.

[0015] The traditional LBP algorithm was first proposed by Ojala et al. in 1996. It uses a 3*3 window to scan the image line by line, uses the size relationship between the center window and the neighboring window, performs binary encoding in a clockwise direction, and finally converts it into decimal as the LBP value of the window.

[0016] The formula of traditional LBP is as follows:

[0017]

[0018]

[0019] Among them, (x c ,y c ) is the center pixel of the 3*3 window, i c Represents the grayscale value of the center window, i P Represents the grayscale value of the neighborhood window, and S(x) is a sign function.

[0020] The original Multi-Scale Local Binary Pattern (MB-LBP) was first proposed by Liao et al. Its basic principle is: in the traditional LBP using a 3×3 window, each window contains n×n sub-windows. The average grayscale value of each window is calculated, and finally the average grayscale value of the neighboring windows is compared with the average grayscale value of the center window. If it is greater than the average grayscale value of the center window, it is set to 1, otherwise it is 0. 0 and 1 are concatenated in a certain order to form a binary code, and converted to decimal to obtain the MB-LBP value. The MB-LBP formula is as follows:

[0021]

[0022]

[0023] in, represents the average gray value of the center window, Represents the average grayscale value of eight neighborhood windows. S(x) is a sign function.

[0024] To further improve the recognition accuracy of wire rope surface defects using the traditional LBP algorithm, this application proposes improvements from the following three perspectives:

[0025] First, to extract more effective wire rope surface defect texture features, the image is segmented to better capture the detailed information of each image block. Second, the traditional LBP feature extraction algorithm is sensitive to noise, and the texture feature sampling range is limited to a small 3×3 region. Most wire rope surface defect texture features are complex and not composed of a single 3×3 region, but often cover a larger area. Therefore, the regional grayscale average method in the multi-scale local binary pattern is used to suppress noise and expand the range of texture feature extraction, thereby compensating for the limited LBP sampling range. Finally, the traditional LBP and most improved LBP algorithms directly use the center window pixel as the threshold without any enhancement of the center pixel's robustness or consideration of its relationship with neighboring pixels. When the center pixel is too large or too small, it is easy to miss detailed features. Therefore, the center pixel is linked to its eight neighboring pixels using the standard deviation method, and the center pixel threshold is redefined to enhance the robustness of the center pixel and the role of neighboring pixels.

[0026] Combining the above three points, the ISCM-LBP algorithm is proposed. The specific operations are as follows:

[0027] (1) Construct a 3×3 window, divide each window into sub - windows of size n×n, then calculate the average gray - scale value of the sub - windows contained in each window, and finally calculate the standard deviation S of the 9 pixels in the 3×3 window. The formula is as follows:

[0028]

[0029] In the formula: is the average gray - scale value of the sub - windows contained in the 3×3 window, is the average value of the 3×3 window;

[0030] (2) Set a threshold f, and determine whether σ is greater than the threshold f. If σ≥f, take the median of the 9 pixels in the 3×3 window as the threshold i t , and then calculate the LBP value using formula (1); if σ < f, calculate the maximum value a and the minimum value b of the 9 pixel values in the 3×3 window, and take as the threshold, and calculate the MMB - LBP value. The formula is as follows:

[0031]

[0032]

[0033]

[0034] In the formula: is the average gray - scale value of the neighborhood window, i t is the threshold in different cases, is the average gray - scale value of the central pixel. a and b are respectively the maximum and minimum values of the 8 pixel values in the neighborhood of the 3×3 window, σ is the standard deviation of the 9 pixels in the 3×3 window, m is the median of the 9 pixels in the 3×3 window. After many experiments, when the threshold f is 80, the recognition rate is relatively high, and S(x) is a sign function.

[0035] (3) Divide the image into blocks of size m×m, calculate the ISCM - LBP value in each block, obtain the ISCM - LBP feature statistical histogram of each block, and arrange them in a row in spatial order to form the ISCM - LBP feature value.

[0036] The gray - level co - occurrence matrix (GLCM) was first proposed by Haralick et al. in ired probability P(i,j,d,θ) of two pixel points appearing simultaneously in the image. In this application, the GLCM parameters are set as follows: θ takes four directions of 0°, 45°, 90°, 135°, the distance d = 1, and the gray - level L = 16.

[0037] The formula of GLCM is as follows:<

[0038] P(i,j,d,θ)=#{(x1,y1),(x2,y2)∈M*N|f(x1,y1)=i,f(x2,y2)=j} (9)

[0039] Where: #{X} is the number of elements in the set X, d is the distance between two pixels, θ is the angle between the line connecting the two pixels and the positive direction of the coordinate axis, (x1, y1) and (x2, y2) are two pixels of the image separated by d in the θ direction, x2 = x1 + d*cosθ, y2 = y1 + d*sinθ.

[0040] Traditional GLCM is generally not used as a texture feature of an image. Haralick et al. obtained 14 different statistical features from the gray-level co-occurrence matrix. This application selects the following four main global statistical features from these 14 statistical features:

[0041] (1) Angular second moment (ASM), which mainly reflects the uniformity of the image, is expressed as follows:

[0042]

[0043] (2) Contrast (CON), which mainly reflects the clarity of image texture. The formula is:

[0044]

[0045] (3) Correlation (COR), which reflects the consistency of image texture, is expressed as:

[0046]

[0047] (4) Inverse differential moment (IDM), which mainly describes the roughness of image texture, is expressed as follows:

[0048]

[0049] Image block-based centered multi-scale local binary pattern (ISCM-LBP) overly focuses on extracting local texture details in an image, fails to fully utilize the image's global texture features, and lacks a complete and accurate description of the image. Global features describe the entire image, while local features describe a small group of pixels. The fusion of global and local features can significantly improve classification and recognition rates.

[0050] As a further improvement to the above scheme, this application fuses GLCM and ISCM-LBP features. There is a key point in feature fusion, and different weights have a great impact on the recognition rate of the classifier. This application adopts serial feature fusion, which has a better recognition rate effect in the SVM classifier. The specific process is as follows:

[0051] The feature fusion formation method is as follows:

[0052] LBP:y1=(a0,a1,…a 79 ) (14)

[0053] GLCM:y2=(b0,b1,…,b 15 ) (15)

[0054] The fused feature Z is:

[0055] Z={y1,y2} (16)

[0056] The inventive concept of this application is to introduce image segmentation and multi-scale into the local binary pattern algorithm to improve the description of image details and suppress noise sensitivity. Secondly, the central pixel is linked to the neighboring pixels to enhance the robustness of the central pixel and improve the accuracy of the traditional LBP algorithm in identifying wire rope surface defects. To further improve the description of image integrity, the central multi-scale local binary pattern algorithm of the image segmentation is subjected to PCA dimensionality reduction and then fused with the gray-level co-occurrence matrix features to supplement the global image information. Finally, the wire rope surface defect is identified using the SVM classifier, thereby improving the classification and recognition rate of wire rope surface defects.

[0057] Compared with the prior art, the present invention has the following advantages:

[0058] The problem that traditional local binary pattern (LBP) is easily affected by central pixels and noise and cannot accurately identify surface defects of wire ropes is solved. The overall recognition rate of wire rope surface defects can reach 97.5%, which is at least 5% higher than other algorithms, and can effectively identify various defects on the wire rope surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A comparative analysis diagram of the steel wire ropes in the embodiment;

[0060] Figure 2 Identification framework diagram of the present invention in the embodiment;

[0061] Figure 3 FIG1 is a diagram of the original MB-LBP implementation process in the embodiment;

[0062] Figure 4 Schematic diagram of GLCM in the embodiment;

[0063] Figure 5 4 states of the steel wire rope collected in the embodiment are statistical tables;

[0064] Figure 6 The PCA explained variance curve and different characteristic dimension result graphs in the embodiment;

[0065] Figure 7 A comparison diagram of different kernel functions in the embodiment;

[0066] Figure 8 ISCM-LBP texture feature map of the wire rope defect sample in the embodiment when the sub-window size is 3 to 21;

[0067] Figure 9 Graph showing recognition rate and pattern recognition time for image segmentation and sub-window size in the embodiment;

[0068] Figure 10 This is a comparison table of recognition accuracy of different algorithms on the wire rope dataset in the embodiment;

[0069] Figure 11 This is a comparison table of feature extraction time of different algorithms in the embodiment;

[0070] Figure 12 This is a comparison table of time for pattern recognition of different algorithms in the embodiment;

[0071] Figure 13 This is a comparison chart of recognition rate and time loss of different feature fusion algorithms in the embodiment;

[0072] Figure 14 A comparison diagram of confusion matrices of different algorithms for wire rope surface defect recognition in the embodiment;

[0073] Figure 15 4 is a comparison table of prediction accuracy of different algorithms in the embodiments. DETAILED DESCRIPTION

[0074] The present invention is described below in conjunction with the accompanying drawings and specific embodiments of the present invention. The description herein is only used to explain the present invention and is not intended to limit the present invention. Based on the embodiments of the present invention, any modifications, equivalent substitutions, improvements, etc. made by those skilled in the art without creative work based on all other embodiments obtained in the present invention should be included in the scope of protection of the present invention.

[0075] Example

[0076] In this embodiment, 8×10 steel wire ropes with a diameter of 13 mm, 6×19 steel wire ropes with a diameter of 14 mm, and 18×7 steel wire ropes with a diameter of 22 mm are used as research objects. Figure 1The four most common states of wire ropes, namely intact (IN), missing strand (MS), broken wire (BW), and deformed (DE), as well as grayscale images and improved LBP texture feature images of the four states of wire ropes are displayed.

[0077] The wire rope surface defect recognition method based on feature fusion proposed in this embodiment has the following recognition framework: Figure 2 As shown, the specific steps include:

[0078] S1, preprocessing: grayscale the wire rope defect image;

[0079] S2, feature extraction and fusion: The image is divided into blocks and the traditional LBP algorithm is improved by the central multi-scale local binary pattern based on the image block to obtain the ISCM-LBP algorithm. The ISCM-LBP algorithm is used to extract the texture feature information of the image after the block. PCA dimensionality reduction is performed based on the obtained image texture features. Finally, the global texture features of the image are extracted by GLCM and the image texture features after dimensionality reduction are fused;

[0080] S3, identification and classification: Wire rope surface defects are identified and classified using SVM classifier.

[0081] Wherein, step S2 includes:

[0082] S21, introduces image segmentation and multi-scale into the local binary pattern algorithm to improve detail feature description and reduce the impact of noise;

[0083] S22, uses the standard deviation method to connect the central pixel with the neighboring pixels, and redefines the selection of the central pixel threshold to enhance the robustness of the central pixel and the role of the neighboring pixels.

[0084] In step S21, a method of regional grayscale average value in a multi-scale local binary pattern is used to suppress noise and expand the range of texture feature value extraction, thereby compensating for the limitation of the small LBP sampling range.

[0085] The traditional LBP algorithm was first proposed by Ojala et al. in 1996. It uses a 3*3 window to scan the image line by line, uses the size relationship between the center window and the neighboring window, performs binary encoding in a clockwise direction, and finally converts it into decimal as the LBP value of the window.

[0086] The formula of traditional LBP is as follows:

[0087]

[0088]

[0089] Among them, (x c ,yc ) is the center pixel of the 3*3 window, i c Represents the grayscale value of the center window, i P Represents the grayscale value of the neighborhood window, and S(x) is a sign function.

[0090] The original multi-scale local binary pattern (MB-LBP) was first proposed by Liao et al. Its basic principle is: under the traditional LBP using a 3×3 window, each window contains n×n sub-windows, and the average grayscale value of each window is counted. Finally, the average grayscale value of the neighboring window is compared with the average grayscale value of the center window. If it is greater than the average grayscale value of the center window, it is set to 1, otherwise it is 0. 0 and 1 are concatenated into binary code in a certain order, and converted into decimal to the MB-LBP value. The original MB-LBP implementation process is as follows Figure 3 The MB-LBP formula is as follows:

[0091]

[0092]

[0093] in, represents the average gray value of the center window, Represents the average grayscale value of eight neighborhood windows. S(x) is a sign function.

[0094] To further improve the recognition accuracy of wire rope surface defects using the traditional LBP algorithm, this application proposes improvements from the following three perspectives:

[0095] First, to extract more effective wire rope surface defect texture features, the image is segmented to better capture the detailed information of each image block. Second, the traditional LBP feature extraction algorithm is sensitive to noise, and the texture feature sampling range is limited to a small 3×3 region. Most wire rope surface defect texture features are complex and not composed of a single 3×3 region, but often cover a larger area. Therefore, the regional grayscale average method in the multi-scale local binary pattern is used to suppress noise and expand the range of texture feature extraction, thereby compensating for the limited LBP sampling range. Finally, the traditional LBP and most improved LBP algorithms directly use the center window pixel as the threshold without any enhancement of the center pixel's robustness or consideration of its relationship with neighboring pixels. When the center pixel is too large or too small, it is easy to miss detailed features. Therefore, the center pixel is linked to its eight neighboring pixels using the standard deviation method, and the center pixel threshold is redefined to enhance the robustness of the center pixel and the role of neighboring pixels.

[0096] This application proposes the ISCM-LBP algorithm based on the above three points. The specific operations are as follows:

[0097] (1) Construct a 3×3 window, divide each window into sub-windows of size n×n, then calculate the average gray value of the sub-windows contained in each window, and finally calculate the standard deviation S of the 9 pixels in the 3×3 window. The formula is as follows:

[0098]

[0099] In the formula: is the average gray value of the sub-windows contained in the 3×3 window, is the average value of the 3×3 window;

[0100] (2) Set a threshold f, and determine whether σ is greater than the threshold f. If σ≥f, take the median of the 9 pixels in the 3×3 window as the threshold i t , and then calculate the LBP value using formula (1); if σ<f, calculate the maximum value a and the minimum value b of the 9 pixel values in the 3×3 window, and take as the threshold, and calculate the MMB-LBP value. The formula is as follows:

[0101]

[0102]

[0103]

[0104] In the formula: <​​​​​​​​​​​​As shown, a pair of pixels (2, 2) is selected, where the grayscale value matrix on the left has 4 grayscale levels, the grayscale co-occurrence matrix on the right is 4*4, the direction θ = 0°, and the separation distance d = 1. For example, the original image (2, 2) appears 3 times, and the GLCM value is set to 3. In this application, the GLCM parameters are set as follows: θ takes four directions of 0°, 45°, 90°, and 135°, the separation distance d = 1, and the grayscale level is L = 16.

[0107] The formula for GLCM is as follows:

[0108] P(i,j,d,θ)=#{(x1,y1),(x2,y2)∈M*N|f(x1,y1)=i,f(x2,y2)=j} (9)

[0109] Where: #{X} is the number of elements in the set X, d is the distance between two pixels, θ is the angle between the line connecting the two pixels and the positive direction of the coordinate axis, (x1, y1) and (x2, y2) are two pixels of the image separated by d in the θ direction, x2 = x1 + d*cosθ, y2 = y1 + d*sinθ.

[0110] Traditional GLCM is generally not used as a texture feature of an image. Haralick et al. obtained 14 different statistical features from the gray-level co-occurrence matrix. This application selects the following four main global statistical features from these 14 statistical features:

[0111] (1) Angular second moment (ASM), which mainly reflects the uniformity of the image, is expressed as follows:

[0112]

[0113] (2) Contrast (CON), which mainly reflects the clarity of image texture. The formula is:

[0114]

[0115] (3) Correlation (COR), which reflects the consistency of image texture, is expressed as:

[0116]

[0117] (4) Inverse differential moment (IDM), which mainly describes the roughness of image texture, is expressed as follows:

[0118]

[0119] Image block-based centered multi-scale local binary pattern (ISCM-LBP) overly focuses on extracting local texture details in an image, fails to fully utilize the image's global texture features, and lacks a complete and accurate description of the image. Global features describe the entire image, while local features describe a small group of pixels. The fusion of global and local features can significantly improve classification and recognition rates.

[0120] As a further improvement to the above scheme, this application fuses GLCM and ISCM-LBP features. There is a key point in feature fusion, and different weights have a great impact on the recognition rate of the classifier. This application adopts serial feature fusion, which has a better recognition rate effect in the SVM classifier. The specific process is as follows:

[0121] The feature fusion formation method is as follows:

[0122] LBP:y1=(a0,a1,…a 79 ) (14)

[0123] GLCM:y2=(b0,b1,…,b 15 ) (15)

[0124] The fused feature Z is:

[0125] Z={y1,y2} (16)

[0126] The inventive concept of this application is to introduce image segmentation and multi-scale into the local binary pattern algorithm to improve the description of image details and suppress noise sensitivity. Secondly, the central pixel is linked to the neighboring pixels to enhance the robustness of the central pixel and improve the accuracy of the traditional LBP algorithm in identifying wire rope surface defects. To further improve the description of image integrity, the central multi-scale local binary pattern algorithm of the image segmentation is subjected to PCA dimensionality reduction and then fused with the gray-level co-occurrence matrix features to supplement the global image information. Finally, the wire rope surface defect is identified using the SVM classifier, thereby improving the classification and recognition rate of wire rope surface defects.

[0127] Specific implementation and application of this application

[0128] 1. Experimental dataset and configuration

[0129] 240 different intact wire rope images, 300 different strand-missing wire rope images, 200 different broken wire rope images, and 260 different deformed wire rope images were collected from industrial sites and laboratories to create a wire rope image dataset with a total of 1,000 images. Figure 5The four captured wire rope conditions are shown in Figure 1. It is important to note that the collected wire rope surface defect images are random in number, location, and shape and were acquired under natural lighting. 60% of the images in this dataset were randomly selected as the training set, and the remaining 40% as the test set for the wire rope condition recognition experiment. The experimental configuration used was: an Intel Core i5-9300HF @ 2.40 GHz quad-core processor, 16 GB of memory, and the operating system Windows 10. The experiment was run under Python 3.7.

[0130] 2. PCA feature dimensionality reduction

[0131] Figure 6 The PCA explained variance curve and the results of different feature dimensions are shown. It can be seen that when the feature dimension is 20 dimensions, 97.5% of the information can be extracted, and when the feature dimension is 100 dimensions, 99% of the information can be extracted. This application reduces the original 2304-dimensional data features to between 20 and 100 dimensions. It can be seen that as the feature dimension increases, the pattern recognition time also increases. The recognition rate reaches the highest when the feature dimension is 80 dimensions, and the recognition rate no longer increases with the increase in dimension. The pattern recognition time at this time is 0.0729 seconds. Although the pattern recognition time increases by 0.018 seconds compared to the 20-dimensional feature, the recognition rate is greatly improved, increasing by 7.75%. Comprehensive comparison shows that reducing the 2304-dimensional data features to 80 dimensions through PCA can effectively improve the recognition time of surface defects of wire ropes.

[0132] 3. Selection of kernel function in support vector machine

[0133] The selection of different kernel functions in support vector machines will have a great impact on the recognition of wire rope surface defects. The recognition rate and pattern recognition time of three different kernel functions, polynomial (Poly), linear (Linear) and Gaussian radial basis (Rbf), are compared experimentally. Figure 7 As shown in the figure, the Gaussian radial basis kernel function is the best in both recognition rate and pattern recognition time, while the linear kernel function is similar to the Gaussian radial basis kernel function in recognition rate, but the pattern recognition time is relatively long. The polynomial kernel function is similar to the Gaussian radial basis kernel function in pattern recognition time, but the recognition rate is relatively low. Therefore, this application selects the Gaussian radial basis kernel function.

[0134] 4. Experiments with different block and sub-window sizes

[0135] The ISCM-LBP texture feature maps obtained with different sub-window sizes are also different. Figure 8The ISCM-LBP texture feature maps of wire rope defect samples with sub-window sizes ranging from 3 to 21 are shown. It can be seen that the texture feature maps extracted with different sub-window sizes are somewhat different. The sub-window size is represented by n, where n=3 represents the traditional LBP texture algorithm.

[0136] Figure 8 The effect of different image block sizes m and different sub-window selections n on the ISCM-LBP recognition performance is shown. It can be clearly seen that when the image block size m=1 increases to m=3, the recognition rate of ISCM-LBP under each sub-window selection is significantly improved. This may be because the increase in the number of blocks provides a more detailed description of the image details. However, as the number of blocks continues to increase, the recognition performance of ISCM-LBP tends to saturate or even shows a downward trend. This may be because the excessive description of the image details leads to a lot of interference in the details. Secondly, Figure 9 The effect of different image block sizes m and different subwindow selections n on the ISCM-LBP pattern recognition time is shown. As the number of image blocks increases, the pattern recognition time increases continuously. From m = 1 to m = 3, the pattern recognition time increases slowly and remains below 1 second. When the number of image blocks exceeds 3, the pattern recognition time increases significantly, reaching a maximum of approximately 10 seconds, a nearly 10-fold increase. This is likely because as the number of image blocks and subwindows increases, the computational complexity of the wire rope surface texture features also increases exponentially, leading to a significant increase in pattern recognition time. Therefore, this paper selects image block size m = 3 and subwindow n = 9, achieving the optimal recognition rate and pattern recognition time.

[0137] 5. Comparison between this application and other single algorithms

[0138] In order to demonstrate the effectiveness of the algorithm proposed in this paper, a texture defect recognition experiment was carried out on the established wire rope surface defect dataset, and nine texture feature extraction algorithms were compared: local binary pattern (LBP), extended local binary pattern (Extended LBP), equivalent local binary pattern (Uniform Pattern LBP), rotation invariant local binary pattern (Rotation Invariant LBP), local ternary pattern (LTP), local directional pattern (LDP), local optimal pattern (LOOP), multi-scale local binary pattern (MB-LBP) and gray-level co-occurrence matrix (GLCM).

[0139] Figure 10The recognition rates of different algorithms were compared. It can be seen that traditional LBP is sensitive to noise and has low robustness of central pixels, resulting in low recognition rate. Uniform Pattern LBP features have increased statistics, but it is easy to lose a lot of detail information. The Rotation Invariant LBP has a low recognition rate, which may be due to its weak description ability and sparse angular space sampling, which easily misses texture features. LTP can reduce the impact of noise, but the fixed threshold selected by LTP is not suitable for wire rope texture images. LDP is not very sufficient in feature extraction, and the recognition effect is unstable. MB-LBP can better suppress noise and extract large-scale information, but it is easy to miss local detail features. The poor recognition rate of GLCM may be because the algorithm is better at extracting global overall information and easily ignores local detail features. Comprehensive comparison, the ISCM-LBP algorithm proposed in this application shows excellent recognition performance on the established wire rope dataset.

[0140] Figure 9 The comparison of running time of different algorithms is shown. It can be seen that the algorithm with lower eigenvalue pattern generally has lower pattern recognition time, which means that the amount of texture feature calculation is also correspondingly smaller. The pattern recognition time of the other algorithms is about 0.1 seconds apart. The pattern recognition time of the equivalent pattern and rotation invariant algorithms is shorter, but the feature extraction time is relatively long. The feature extraction time of the other algorithms is about 1 second. The ISCM-LBP algorithm proposed in this application is at a moderate level in pattern recognition time and feature extraction time. The pattern recognition time and feature extraction time of GLCM are 0.064 seconds and 0.005 seconds respectively, which is the lowest time-consuming among all algorithms, but its overall recognition rate is also the lowest, so it is not adopted. Comprehensive Figure 9 and Figure 10 ,The ISCM-LBP algorithm of this application has good performance in ,recognition rate and running time, and can provide defect recognition and ,prediction functions for wire rope surface defects.

[0141] Figure 11 A comparison of feature extraction times for different algorithms reveals that Extended LBP, Uniform Pattern LBP, and Rotation Invariant LBP take significantly longer than other texture feature extraction algorithms. This is likely due to the slightly more complex feature extraction process and the limited simplicity of these algorithms. Conversely, GLCM takes significantly less time to extract features than the other algorithms, but also has the lowest recognition rate of all, so it was not used. The ISCM-LBP algorithm used in this application achieves a moderate feature extraction time.

[0142] Figure 12Comparing the pattern recognition times of different algorithms, we can see that Uniform Pattern LBP, Rotation Invariant LBP, and GLCM achieve significantly lower recognition times than the other algorithms. This is because these algorithms have fewer eigenvalue patterns and therefore extract the fewest texture features, resulting in less texture computation. The ISCM-LBP algorithm presented in this application performs moderately well compared to other algorithms in terms of pattern recognition time.

[0143] In summary, it can be concluded that the feature extraction time and pattern recognition time of ISCM-LBP are close to those of other algorithms, while the recognition rate is significantly higher than other algorithms. Therefore, the ISCM-LBP of this application can provide good recognition and prediction functions for wire rope surface defects.

[0144] 6. Comparison of different feature fusion algorithms

[0145] Figure 13 The figure shows that the time loss represents the sum of the time for pattern recognition and feature extraction. It can be seen that the original LBP and GLCM feature fusion algorithm has a time loss of 2.5 seconds and an overall recognition rate of 94%; while the MB-LBP and GLCM feature fusion algorithm consumes an additional 0.38 seconds and the overall recognition rate is only improved by 1.25%. The ISCM-LBP and GLCM feature fusion algorithm of this application has a time loss of 1.56 seconds and an overall defect recognition rate of 97.5%. After the PCA feature dimensionality reduction, the time loss is reduced by about 1 second due to the reduction of a large amount of redundant information, and the recognition rate is also increased by 3.5%. In summary, the ISCM-LBP+GLCM method proposed in this application has excellent recognition performance in wire rope surface defect detection.

[0146] In order to more comprehensively understand the recognition effect of the algorithm proposed in this application and other algorithms on each type of wire rope surface defects, Figure 14 and Figure 15The confusion matrix and recognition rate of each category of each algorithm in wire rope surface defect recognition are shown. It can be seen that MS has the highest recognition rate and basically no misclassification occurs among the four algorithms. IN has a high misrecognition rate in LBP and MB-LBP, which may be because IN and DE are very similar in the surface texture of the wire rope. The ISCM-LBP algorithm proposed in this paper has greatly improved the recognition of IN and DE states compared with the previous two algorithms, but it did not achieve the expected effect. Therefore, this application integrates the global information of image texture defects by fusing the features of the ISCM-LBP algorithm and the GLCM algorithm, and the description of the image is more comprehensive and accurate, which further improves the defect recognition performance. As can be seen from the figure, compared with the previously proposed algorithm, the fused algorithm has further improved the recognition rate of IN and DE, but the misrecognition of BW defects has increased, which may be due to the increase in global information, which reduces the judgment of detailed features. The results show that this method has significantly improved each type of surface defect and overall recognition rate of wire rope, meeting the recognition requirements.

[0147] The above description is merely a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiment. It should be noted that simple modifications, improvements, substitutions, simplifications, and combinations of the present invention made within the scope of this technical field without departing from the technical concept of the present invention are all included in the scope of protection of the present invention.

Claims

1. A wire rope surface defect recognition method based on feature fusion, characterized in that: The following steps are involved: S1, preprocessing: grayscale the wire rope defect image; S2, feature extraction and fusion: The image is divided into blocks and the traditional LBP algorithm is improved by the central multi-scale local binary pattern based on the image block to obtain the ISCM-LBP algorithm. The ISCM-LBP algorithm is used to extract the texture feature information of the image after the block. PCA dimensionality reduction is performed based on the obtained image texture features. Finally, the global texture features of the image are extracted by GLCM and the image texture features after dimensionality reduction are fused; The ISCM-LBP algorithm includes the following steps: (1) Construct a 3×3 window, divide each window into n×n sub-windows, then calculate the average grayscale value of each window including the sub-window, and finally calculate the standard deviation of the 9 pixels in the 3×3 window , the formula is: Where: is the average grayscale value of the sub-window contained in the 3×3 window, is the average value of the 3×3 window; (2) Set the limit threshold f and judge Is it greater than the threshold f? f, 3 The median of the 9 pixels in the 3 windows is used as the threshold , calculate the LBP value; if f, calculate the maximum value a and minimum value b of the 9 pixel values ​​in the 3×3 window, and As the threshold, the MMB-LBP value is calculated, and the formula is as follows: Where: is the average gray value of the neighborhood window, is the threshold value in different situations, is the average gray value of the center pixel, and are the maximum and minimum values ​​of 8 pixels in the 3×3 window neighborhood, is the median of 9 pixels in a 3×3 window, is a symbolic function; (3) Perform image The ISCM-LBP value in each block is calculated to obtain the ISCM-LBP feature statistical histogram of each block, and the ISCM-LBP feature values ​​are arranged in a row according to the spatial order. This step also includes: S21, introduces image segmentation and multi-scale into the local binary pattern algorithm to improve the description of detailed features and reduce the impact of noise; in particular, the method of regional grayscale average in multi-scale local binary pattern is used to suppress noise and expand the range of texture feature value extraction, thereby compensating for the limitation of the small sampling range of LBP; S22, uses the standard deviation method to link the central pixel with the neighboring pixels and redefines the selection of the central pixel threshold to enhance the robustness of the central pixel and the role of the neighboring pixels; S3, identification and classification: Wire rope surface defects are identified and classified using SVM classifier.

2. The method for identifying surface defects of a wire rope based on feature fusion according to claim 1, characterized in that: The formula of the GLCM is: Where: is the probability of two pixels appearing simultaneously in the image, gather The number of elements in is the distance between two pixels, is the angle between the line connecting two pixel points and the positive direction of the coordinate axis, and For the image The direction interval is Two pixels of , ,in Value 、 、 、 Four directions, distance apart , the gray level is L=16; Four main global statistical features, namely, angular second moment, contrast, correlation and inverse gap, are selected and fused in series. The feature fusion method is as follows: The fused features for: 。 3. The method for identifying surface defects of a wire rope based on feature fusion according to claim 1, characterized in that: The value of the limiting threshold f is 80.

Images