Computer vision-based bolt surface defect detection method and system

Abstract

The application discloses a bolt surface defect detection method and system based on computer vision, comprising: 360-degree image acquisition and preprocessing of the bolt through a multi-camera array arranged in a ring; Fourier transform is used to analyze periodic texture features of the thread, and an adaptive filter is designed to enhance the image; multi-scale morphological operation is used to extract morphological gradient features using a structure element for crack, pit and scratch type defects; an adaptive threshold segmentation method based on local statistical features is used to dynamically adjust the segmentation threshold combined with thread direction information to segment out the defect candidate region; geometric and texture features are extracted for the candidate region, and a random forest classifier is used to identify the defect type, position and severity. The application realizes high-precision automatic detection and classification of various defects on the bolt surface.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial automation detection, and particularly relates to a bolt surface defect detection method and system based on computer vision. BACKGROUND

[0002] Bolts are widely used in key industrial fields such as aerospace, petrochemical industry, electric power, and transportation as basic parts for mechanical connection. The surface quality of bolts directly affects the connection strength and service life. Surface defects such as cracks, pits, and scratches can seriously weaken the load-bearing capacity of bolts and may lead to connection failure or even major safety accidents. With the development of industrial equipment towards high precision and high reliability, the detection requirements for bolt surface quality are becoming increasingly stringent. Traditional manual visual inspection methods are not only inefficient and subjective, but also difficult to find small defects, which cannot meet the quality control needs of modern industry. Therefore, developing high-precision and automated bolt surface defect detection technology has become an urgent need in the field of industrial quality control.

[0003] Existing computer vision defect detection methods have many limitations in bolt surface defect detection applications. For example, the hardware part defect detection method disclosed in patent CN202211229527.2 uses simple threshold segmentation based on gray scale boundary value and pixel membership degree calculation to identify defects. This method only relies on pixel gray scale information for processing and lacks in-depth analysis of the periodic texture features of bolt surface threads. This method uses a fixed gray scale boundary value formula to distinguish high and low brightness pixel points. This global threshold method cannot adapt to local illumination changes and texture complexity caused by the thread structure on the bolt surface. In addition, this method only calculates the membership degree by the number of low brightness pixel points and the gray scale value structure entropy in the neighborhood, which lacks targeted analysis of defect geometric shape features and is difficult to distinguish different types of defects such as cracks, pits, and scratches, limiting the detection accuracy and defect classification ability.

[0004] Another important defect of existing technology is the lack of multi-angle detection capability. As a three-dimensional cylindrical part, the surface defects of bolts can be distributed in any position and direction. Single-angle detection methods are prone to detection blind spots, leading to missed defects. At the same time, existing methods generally lack effective use of periodic texture features of threads. The regular thread structure on the bolt surface contains rich frequency domain information, which can be used to distinguish normal texture and defect features. However, traditional methods mostly use spatial domain processing and fail to fully exploit the discriminative value of frequency domain features. In addition, existing methods mostly use single-scale or fixed template processing, lacking adaptive processing capability for different sizes and types of defects, making it difficult to detect both small cracks and large-area surface damage.

[0005] In the aspect of classification recognition, the prior art mostly adopts simple threshold judgment or a method based on empirical rules, lacking intelligent classification algorithm support. These methods are difficult to handle complex multi-feature fusion and nonlinear classification problems, and when facing complex texture background and diversified defect types on the bolt surface, often have high false detection rate and high missed detection rate. Meanwhile, the existing method lacks a quantitative evaluation mechanism for the detection result, cannot provide accurate defect severity information for engineering maintenance decision, and limits its popularization in actual industrial application. SUMMARY

[0006] Therefore, the present application provides a bolt surface defect detection method and system based on computer vision, aiming to combine multi-angle image acquisition, frequency domain texture analysis, multi-scale morphological feature extraction, adaptive threshold segmentation and intelligent classification recognition technologies, realize high-precision automatic detection and classification of various defects such as cracks, pits and scratches on the bolt surface, and provide reliable technical support for the safe maintenance of industrial equipment.

[0007] To achieve the above-mentioned purpose, the bolt surface defect detection method based on computer vision provided by the present application comprises the following steps:

[0008] S1: A multi-camera array arranged in a ring shape is used to perform 360-degree image acquisition on the bolt, to obtain a set of original multi-angle images of the bolt surface and perform denoising and brightness equalization processing, and output a set of pre-processed bolt surface images;

[0009] S2: A frequency domain analysis method based on Fourier transform is used to process the set of pre-processed bolt surface images, to identify the periodic texture features of the thread; according to the periodic features of the thread, an adaptive filter is designed and used to enhance the images in the set of pre-processed bolt surface images, and output a set of enhanced bolt surface images;

[0010] S3: Multi-scale morphological operation is performed on the set of enhanced bolt surface images, to extract morphological gradient features at different scales; by using a set of structure elements for bolt defect types, morphological feature maps of different types of defects such as cracks, pits and scratches are extracted respectively, and a set of multi-scale feature maps is output;

[0011] S4: Local statistical features are extracted from the set of multi-scale feature maps, an adaptive threshold segmentation method based on local statistical features is used, combined with thread direction information and local contrast, to dynamically adjust the segmentation threshold, segment out potential defect regions from the set of multi-scale feature maps, and output a set of binary defect candidate region maps;

[0012] S5: Geometric features and texture features are extracted from the segmented defect candidate region binary image set; based on the extracted features, a random forest classifier is used to identify and classify the defects, and finally the detection results of the bolt surface defects are output, including defect type, position and severity.

[0013] Optionally, in the S1 step, the annularly arranged multi-camera array performs 360-degree image acquisition on the bolt, obtains a set of original multi-angle images of the bolt surface, and performs denoising and brightness equalization processing, and outputs a set of preprocessed bolt surface images, including:

[0014] S11: A multi-camera array arranged in a ring shape is used to perform 360-degree omnidirectional image acquisition on the bolt, n high-resolution industrial cameras are arranged at equal angle intervals around the bolt to form a ring-shaped camera array, and a set of original multi-angle images of the bolt surface is obtained;

[0015] S12: A bilateral filtering algorithm is used to smooth and denoise each image in the set of original multi-angle images, to obtain denoised images; in view of the difference in light conditions when shooting at different angles, a histogram equalization algorithm is used to adjust the brightness distribution of the denoised images, and a set of preprocessed bolt surface images is output.

[0016] Optionally, in the S2 step, a frequency domain analysis method based on Fourier transform is used to identify the periodic texture features of the thread; according to the periodic features of the thread, an adaptive filter is designed and the images in the set of preprocessed bolt surface images are enhanced, and a set of enhanced bolt surface images is output, including:

[0017] S21: Fourier transform is performed on each image in the set of preprocessed bolt surface images to convert the image from the spatial domain to the frequency domain, the periodic texture features of the thread structure are identified by analyzing the power spectral density distribution of the frequency domain representation corresponding to each image, and the fundamental frequency and harmonic frequency set of the thread are determined;

[0018] S22: Sobel edge detection is performed on each image in the set of preprocessed bolt surface images, the gradient direction angle is calculated, the direction angle histogram is constructed, and the thread direction vector is determined;

[0019] S23: Based on the identified fundamental frequency and harmonic frequency set of the thread, an adaptive band-pass filter is designed, and a normal thread frequency response function and a defect frequency response function are defined;

[0020] S24: The adaptive filter is applied to frequency domain filtering processing, and a set of enhanced bolt surface images is obtained through inverse Fourier transform.

[0021] Optionally, the S22 step includes:

[0022] To determine the thread direction vector First, regarding the first Preprocessed bolt surface images from various angles To perform Sobel edge detection, the gradient direction angle of each pixel is calculated by using a Sobel operator with a convolution kernel in the horizontal direction. and vertical convolution kernels calculate exist gradient components of direction and gradient components of direction :

[0023] , ;

[0024] Calculate gradient direction angle : ,in It is the arctangent function;

[0025] Calculate the gradient direction angles of all pixels and construct a histogram of direction angles. ,in The range of values ​​is The angle range is divided into 36 intervals, each interval being 5° wide;

[0026] Thread direction vector Determined as a direction angle histogram The center angle value of the angle range that appears most frequently.

[0027] Optionally, in step S3, multi-scale morphological operations are performed on the enhanced bolt surface image set to extract morphological gradient features at different scales; by using a set of structural elements for different bolt defect types, morphological feature maps of cracks, pits, and scratches are extracted respectively, and a multi-scale feature map set is output, including:

[0028] S31: Design corresponding sets of structural elements for the three common defect types on bolt surfaces, including: a set of linear structural elements for detecting linear crack defects, a set of circular structural elements for detecting circular pit defects, and a set of elliptical structural elements for detecting irregular scratch defects.

[0029] S32: Morphological gradient operations are performed on the enhanced bolt surface image using sets of structural elements of different scales to obtain linear gradient feature map sets, circular gradient feature map sets, and elliptical gradient feature map sets, respectively.

[0030] S33: combine all morphological gradient features of each angular image in the enhanced bolt surface image set to construct a complete multi-scale feature atlas set.

[0031] Optionally, step S32 comprises:

[0032] perform morphological gradient operation on the enhanced bolt surface image using a linear structural element to obtain a linear gradient feature atlas set; wherein the morphological gradient operation is defined as the difference between the dilation operation result and the erosion operation result;

[0033] perform morphological gradient operation on the enhanced bolt surface image using a circular structural element to obtain a circular gradient feature atlas set;

[0034] perform morphological gradient operation on the enhanced bolt surface image using an elliptical structural element set to obtain an elliptical gradient feature atlas set;

[0035] wherein the linear structural element length value range is pixels, the circular structural element radius value range is pixels, the elliptical structural element major axis value range is pixels, and the minor axis value range is pixels.

[0036] Optionally, the adaptive threshold segmentation method based on local statistical features is adopted in the S4 step, combined with thread direction information and local contrast, to dynamically adjust the segmentation threshold, segment out potential defect regions from the multi-scale feature atlas set, and output a defect candidate region binary graph set, including:

[0037] S41: perform local statistical feature extraction on each multi-scale feature atlas in the multi-scale feature atlas set, and calculate the local mean and local standard deviation;

[0038] S42: dynamically calculate the adaptive segmentation threshold of each pixel point of the multi-scale feature atlas based on the local statistical features and the contrast weight;

[0039] S43: use the adaptive segmentation threshold to perform binary segmentation on the multi-scale feature atlas, and output a defect candidate region binary graph set using morphological post-processing and connected region analysis algorithm.

[0040] Optionally, the S5 step extracts geometric features and texture features from the defect candidate region binary graph set obtained by segmentation; based on the extracted features, a random forest classifier is used to identify and classify defects, and finally output the detection result of the bolt surface defects, including defect type, position and severity, including:

[0041] S51: Perform connected region labeling on each defect candidate region binary image, calculate the geometric features of each connected region, including area, circumscribed rectangle aspect ratio and circularity;

[0042] S52: Extract the texture features of each defect candidate region binary image, calculate the gray level co-occurrence matrix within the connected region, and calculate the contrast feature, energy feature, homogeneity feature and entropy feature;

[0043] S53: Construct a feature vector and use a random forest classifier for classification, set the number of decision trees to 100, and use a majority voting mechanism to determine the final classification result;

[0044] S54: Generate the final detection result of the bolt surface defect based on the final classification result, including the defect type, position and severity.

[0045] The application also discloses a bolt surface defect detection system based on computer vision, comprising:

[0046] An image acquisition and preprocessing module: a multi-camera array arranged in a ring is used to perform 360-degree image acquisition on the bolt, to obtain a raw multi-angle image set of the bolt surface and to perform denoising and brightness equalization processing;

[0047] An image enhancement module: a frequency domain analysis method based on Fourier transform is used to identify the periodic texture features of the thread; and an adaptive filter is designed according to the periodic features of the thread to enhance the images in the preprocessed bolt surface image set;

[0048] A multi-scale feature extraction module: multi-scale morphological operations are performed on the enhanced bolt surface image set to extract morphological gradient features at different scales; morphological feature maps of different types of defects, such as cracks, pits and scratches, are extracted by using structure elements specific to the bolt defect types;

[0049] A segmentation module: local statistical features are extracted from the multi-scale feature map set, and an adaptive threshold segmentation method based on local statistical features is used; the segmentation threshold is dynamically adjusted in combination with the thread direction information and the local contrast, to segment the potential defect regions from the multi-scale feature map and output a set of defect candidate region binary images;

[0050] A defect recognition module: geometric features and texture features are extracted from the set of defect candidate region binary images segmented; based on the extracted features, a random forest classifier is used to recognize and classify the defects, and finally the detection result of the bolt surface defect is output, including the defect type, position and severity.

[0051] Compared with the prior art, the application has at least the following beneficial effects:

[0052] The application adopts a periodic texture analysis and adaptive filter enhancement algorithm based on Fourier transform, can accurately identify the frequency characteristics of the bolt thread and design a targeted filter. Compared with the traditional spatial filtering method, the frequency domain analysis can accurately extract the fundamental frequency and harmonic information of the thread, and through the construction of normal thread frequency response function and defect frequency response function, the normal texture is inhibited and the defect characteristics are enhanced.

[0053] The multi-scale morphological feature extraction algorithm designed by the application can detect the geometric features of different types of defects. By designing three different structural element sets of linear, circular and elliptical, corresponding to three typical defect types of cracks, pits and scratches, the matching detection based on defect geometric features is realized. The multi-scale processing mechanism can detect various size defects from small cracks to large scratches through different size structural element combinations, avoiding the limitations of single scale method.

[0054] The intelligent recognition method combining the adaptive threshold segmentation algorithm based on local statistical features and the random forest classifier proposed by the application significantly improves the accuracy and reliability of defect detection. The adaptive threshold algorithm dynamically adjusts the optimal segmentation threshold for each pixel point through the statistical feature calculation in the local window combined with the contrast weight mechanism, effectively solving the segmentation difficulty problem caused by the complex bolt surface texture and uneven illumination. The random forest classifier realizes accurate classification of cracks, pits, scratches and normal areas through the voting mechanism of integrating multiple decision trees, using the comprehensive discriminant ability of geometric features and texture features. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 The figure is a flowchart of the bolt surface defect detection method based on computer vision of an embodiment of the application;

[0056] Figure 2 The figure is a schematic diagram of the original image of the bolt surface of an embodiment of the application;

[0057] Figure 3 The figure is a schematic diagram of the enhanced bolt surface image of an embodiment of the application;

[0058] Figure 4 The figure is a schematic diagram of the defect candidate region binary image of an embodiment of the application. DETAILED DESCRIPTION

[0059] The application will be further described below with reference to the accompanying drawings, but the application is not limited in any way by the application. Any transformation or replacement based on the teaching of the application belongs to the protection scope of the application.

[0060] Example 1:

[0061] The application provides a computer vision-based bolt surface defect detection method, which comprises the following steps as shown in the accompanying drawings: Figure 1 . .

[0062] S1: 360-degree image acquisition of the bolt is performed through a multi-camera array arranged in a ring shape, a raw multi-angle image set of the bolt surface is obtained, and the collected images are denoised and subjected to brightness equalization processing, and a preprocessed bolt surface image set is outputted: .

[0063] S11: a raw multi-angle image set of the bolt surface is collected; .

[0064] The 360-degree omnidirectional image acquisition of the bolt is performed through a camera array arranged in a ring shape, specifically: .

[0065] The high-resolution industrial cameras are arranged around the bolt at equal-angle intervals to form a ring-shaped camera array, wherein the angle interval between the cameras is ; each camera is denoted as , wherein , a raw multi-angle image set of the bolt surface is obtained , wherein and represent the raw bolt surface images collected by the first and the first cameras, as shown in the accompanying drawings; Figure 2 . .

[0066] S12: image denoising and brightness equalization are performed; .

[0067] Each image in the raw multi-angle image set is subjected to denoising processing, a bilateral filtering algorithm is adopted to smooth and denoise the image, and a corresponding denoised image is obtained. . .

[0068] In view of the differences in light conditions when images are taken at different angles, the denoised image is subjected to brightness equalization processing, the brightness distribution of is adjusted through a histogram equalization algorithm, so that all the images at different angles have consistent brightness characteristics, a preprocessed bolt surface image is obtained, and a preprocessed bolt surface image set is outputted, wherein and represent the preprocessed bolt surface images at the first and the first angles. .

[0069] It should be noted that this step realizes the full-range image acquisition of the bolt surface through the annular multi-camera array system, compared with the traditional single camera or fixed view detection method, the complete three-dimensional geometric information and texture details of the bolt surface can be obtained. This 360-degree full-coverage acquisition method ensures that any position defect of the bolt surface can be clearly captured by at least one camera.

[0070] The bilateral filtering algorithm used in this step can well maintain the edge information while removing image noise, which is crucial for subsequent defect detection, because the cracks, scratches and other defects on the bolt surface often exhibit subtle edge features. Traditional Gaussian filtering or mean filtering will blur these key edge information, while bilateral filtering can maintain the sharpness of defect edges while smoothing noise areas by considering both spatial distance and pixel intensity difference

[0071] S2: A frequency domain analysis method based on Fourier transform is used on the pre-processed bolt surface image set to identify the periodicity characteristics of the thread; according to the periodicity characteristics of the thread, an adaptive filter is designed to enhance the images in the pre-processed bolt surface image set, and an enhanced bolt surface image set is output:

[0072] S21: Analyze the periodicity characteristics of the thread;

[0073] The pre-processed bolt surface image set Each image in the pre-processed bolt surface image set is subjected to Fourier transform to convert the image from spatial domain to frequency domain, obtaining the corresponding frequency domain representation

[0074] By analyzing the power spectral density distribution of , the periodic texture features of the thread structure are identified; the peak point in the power spectrum is found to determine the fundamental frequency of the thread as the non-zero frequency component with the largest amplitude in the power spectrum; The harmonic frequency set of the thread is calculated as

[0075] , where represents the th harmonic frequency component, represents the th harmonic frequency component, and is the set number of effective harmonics;

[0076] S22: Thread direction detection and quantization;

[0077] To determine the thread direction vector , first, the pre-processed image at the Sobel edge detection is performed to calculate the gradient direction of each pixel point; a convolution kernel in the horizontal direction and a convolution kernel in the vertical direction are used

[0078]

[0079]

[0080]

[0081]

[0082] S23: Adaptive filter design and construction

[0083] Based on the identified thread fundamental frequency and the harmonic frequency set , an adaptive band-pass filter is designed

[0084] The normal thread frequency response function is defined as a Gaussian function centered on the thread fundamental frequency and harmonic frequencies:

[0085]

[0086] where is the value of the normal thread frequency response function at the frequency component ; indicates the standard deviation parameter of the normal thread frequency response; wherein and represent the frequency components in the horizontal and vertical directions, respectively is a natural constant

[0087] ​​​​​​​​​​​​​​​​​​​​​​​​​Definition of the defect frequency response function For the frequency components other than the normal thread frequency in the frequency domain:

[0088] ;

[0089] The transfer function of the adaptive filter is:

[0090] ;

[0091] wherein represents a defect enhancement coefficient, represents a normal texture suppression coefficient;

[0092] S24: Perform image enhancement processing;

[0093] The designed adaptive filter is applied to the corresponding frequency domain representation , and a frequency domain filtering process is performed: , to obtain the corresponding enhanced frequency domain image ;

[0094] The inverse Fourier transform is performed on the corresponding enhanced frequency domain image to obtain the enhanced bolt surface image at the th angle , and the set of enhanced bolt surface images is output , wherein represents the enhanced bolt surface image at the th angle, as shown in Figure 3 .

[0095] It should be noted that this step adopts a frequency domain analysis method based on Fourier transform, which can accurately identify the periodic texture features of the thread. This is difficult to achieve by traditional spatial image processing methods. The thread on the bolt surface has a clear periodic geometric structure, which is manifested as energy concentration of specific frequency components in the frequency domain. Through power spectral density analysis, this step can accurately locate the fundamental frequency and harmonic frequency of the thread, thereby establishing a frequency feature model of the thread texture. This frequency domain feature extraction method is not affected by the direction and local deformation of the thread, and has strong robustness.

[0096] The step realizes accurate quantification of the thread direction by the method of Sobel edge detection and direction angle statistics. The traditional texture analysis method often assumes that the texture direction is known or fixed, while the installation angle of the bolt may deviate in actual detection. The step can automatically adapt to different thread directions by constructing a direction angle histogram and finding the angle interval with the highest frequency, ensuring the accuracy of subsequent filter design. This adaptive direction detection mechanism significantly improves the practicality and reliability of the detection system.

[0097] S3: Multi-scale morphological operation is performed on the enhanced bolt surface image set to extract morphological gradient features at different scales; by using a set of structure elements for bolt defect types, morphological features of different types of defects such as cracks, pits, and scratches are extracted respectively, and a multi-scale feature atlas set is output:

[0098] S31: Design multi-scale structure elements;

[0099] For the three common types of bolt surface defects, the corresponding structure element set is designed, specifically:

[0100] For detecting linear crack defects, linear structure elements are designed, which have a linear structure with a length of and a width of 1 pixel, where has a value range of pixels, forming a linear structure element set , where represents a linear structure element with a length of pixels;

[0101] For detecting circular pit defects, circular structure elements are designed, which have a circular structure with a radius of , where has a value range of pixels, forming a circular structure element set , where represents a circular structure element with a radius of pixels;

[0102] For detecting irregular scratch defects, elliptical structure elements are designed, which have an elliptical structure with a major axis of and a minor axis of , where has a value range of pixels, and has a value range of pixels, forming an elliptical structure element set , where represents an elliptical structure element with a major axis of Pixels, minor axis is An elliptical structural element of pixels;

[0103] S32: Perform multi-scale morphological gradient calculations;

[0104] Enhanced bolt surface image set Each image in Morphological gradient operations were performed using structuring element sets of different scales, specifically:

[0105] Using a linear structure element set right Perform morphological gradient operations to obtain a set of linear gradient feature maps. ,in Indicates the use of length is Linear structuring elements of pixels For the first Enhanced bolt surface images at various angles The result of morphological gradient operation, where ⊕ represents morphological dilation operation. Represents morphological erosion operations;

[0106] Use a circular structural element set right Perform morphological gradient operations to obtain a set of circular gradient feature maps. ,in Indicates the use of radius is Pixel circular structural elements For the first Enhanced bolt surface images at various angles The result of morphological gradient calculation;

[0107] Using an elliptical structure element set right Perform morphological gradient operations to obtain a set of elliptical gradient feature maps. ,in Indicates the use of the major axis as Pixels, minor axis is Pixel elliptical structural element For the first Enhanced bolt surface images at various angles The result of morphological gradient calculation;

[0108] S33: Construct a multi-scale feature map set;

[0109] Images from each angle All morphological gradient feature maps are combined to form the first... Multi-scale feature maps from various angles , specifically:

[0110] ;

[0111] For all angle images, a complete multi-scale feature atlas set is constructed , wherein represents the multi-scale feature atlas of the angle.

[0112] It should be noted that this step uses multi-scale morphological operation to realize the targeted feature extraction of different types of bolt defects. This method has stronger defect recognition ability compared with traditional single filter or edge detection algorithm. By designing special structural elements for three typical defects of cracks, pits and scratches, this step can fully utilize the geometric shape features of the defects for matching detection. Linear structural elements have the best response to slender crack defects, circular structural elements can effectively detect circular or nearly circular pit defects, and elliptical structural elements are specially used to identify irregular scratch defects. This defect geometric feature-based structural element design strategy significantly improves the specificity and accuracy of detection.

[0113] S4: Extract local statistical features from the multi-scale feature atlas set, and use an adaptive threshold segmentation method based on local statistical features according to the local texture characteristics of the bolt surface; combine the thread direction information and the local contrast to dynamically adjust the segmentation threshold, segment the potential defect area from the multi-scale feature atlas set, and output the binary graph set of the defect candidate area:

[0114] S41: Extract local statistical features and construct a morphological gradient feature graph set;

[0115] For each multi-scale feature atlas in the complete multi-scale feature atlas set , extract local statistical features, specifically:

[0116] In the multi-scale feature atlas of the angle , perform local statistical analysis on each morphological gradient feature graph, and set the local window size to , wherein pixels.

[0117] For each feature graph in the linear gradient feature graph set , calculate the local mean and the local standard deviation in each local window to obtain the local statistical feature set of the linear gradient feature graph ;

[0118] For each feature map in the circular gradient feature map set Calculate the local mean and the local standard deviation in each local window, to obtain the local statistical feature set of the circular gradient feature map ;

[0119] For each feature map in the elliptical gradient feature map set Calculate the local mean and the local standard deviation in each local window, to obtain the local statistical feature set of the elliptical gradient feature map ;

[0120] S42: dynamically adjust the adaptive segmentation threshold;

[0121] Based on the local statistical features, dynamically calculate the adaptive segmentation threshold , specifically:

[0122] For each multi-scale feature map in the multi-scale feature map set, calculate the base threshold :

[0123] ;

[0124] Where represents the global mean of the multi-scale feature map, represents the global standard deviation of the multi-scale feature map, represents the threshold adjustment factor;

[0125] Combine the contrast weight , calculate the adaptive segmentation threshold of each pixel position :

[0126] ;

[0127] Where represents the contrast weight adjustment coefficient; is the contrast weight of the pixel position , , and are the local mean and local standard deviation of the current calculation feature map at the pixel position ;

[0128] S43: segment the defect candidate region;

[0129] Use the dynamically adjusted adaptive segmentation threshold​​ The multi-scale feature map is binarized and segmented, specifically:

[0130] For each pixel point in each multi-scale feature map in the multi-scale feature map set , threshold judgment is performed:

[0131] When the multi-scale feature map pixel value is greater than , the pixel point is marked as a defect candidate pixel and assigned a value of 1; when the multi-scale feature map pixel value is less than or equal to , the pixel point is marked as a background pixel and assigned a value of 0;

[0132] The segmentation result is subjected to morphological post-processing, and a circular structural element with a radius of pixels is used for closing operation to fill small holes in the defect area;

[0133] A connected region analysis algorithm is used to remove noise regions with an area less than pixels, and connected regions with an area greater than or equal to are retained as effective defect candidate regions;

[0134] The defect candidate region binary map set is output, where and represent the defect candidate region binary map of the th and the th angle, as shown in Figure 4 .

[0135] It should be noted that this step uses an adaptive threshold segmentation method based on local statistical features, which has significant advantages over traditional global threshold or fixed threshold segmentation. Due to the presence of thread structure on the bolt surface, the texture characteristics and lighting conditions of different regions are significantly different, and a fixed threshold often cannot simultaneously meet the segmentation needs of all regions. This step calculates statistical features such as mean and standard deviation within a local window, which can fully reflect the local texture characteristics of each region, thereby calculating the most suitable segmentation threshold for each pixel point. This local adaptive strategy ensures that ideal segmentation results can be obtained in different texture regions.

[0136] S5: Extract geometric features and texture features from the defect candidate region binary map set obtained by segmentation; based on the extracted features, use a random forest classifier to recognize and classify defects, and finally output the detection results of bolt surface defects, including defect type, location, and severity:

[0137] S51: Extract geometric features of the defect candidate region binary map set;

[0138] The defect candidate region binary map set Binary map of each defect candidate region in Extract the geometric features of each connected region, specifically:

[0139] For the first Binary map of candidate defect regions from various angles By labeling connected components, we obtain a set of connected components. ,in Indicates the first The first angle binary image of the first angle A connected region, Indicates the first The first angle binary image of the first angle A connected region, Indicates the first The total number of connected regions in a binary graph from each angle;

[0140] For each connected region Calculate its geometric features, specifically including:

[0141] Calculate the area of ​​the connected region Defined as a connected region The total number of pixels contained;

[0142] Calculate the circumscribed rectangle of the connected region to obtain the width of the rectangle. and height And calculate the aspect ratio. ,in functions and The function retrieves the maximum and minimum values ​​of the input numbers, respectively.

[0143] Calculate the circularity of a connected region Defined as: ,in Represents connected regions circumference, Represents pi;

[0144] S52: Extract texture features from candidate defect regions;

[0145] For each connected region From the corresponding number A pre-processed bolt surface image Extracting the texture features of this region specifically involves:

[0146] Calculate the gray-level co-occurrence matrix within connected regions. Set the pixel spacing to The value is 1, and the direction angle is... ;

[0147] Based on a gray level co-occurrence matrix , a texture feature parameter is calculated:

[0148] A contrast feature is calculated : , wherein and respectively represent a row index and a column index of the gray level co-occurrence matrix; is a value at the row index and the column index ;

[0149] An energy feature is calculated : ;

[0150] A homogeneity feature is calculated : ;

[0151] An entropy feature is calculated : , wherein is a logarithm operation with a base of 2;

[0152] S53: A random forest classifier is constructed and feature classification is performed;

[0153] Geometric features and texture features of each connected region are combined, a feature vector is constructed, and classification is performed using the random forest classifier, specifically:

[0154] For each connected region , a feature vector is constructed: ;

[0155] A random forest classifier is constructed , a number of decision trees is set , and a number of features randomly selected at a split node of each decision tree is set ;

[0156] The construction process of the random forest classifier includes:

[0157] For each decision tree , , a sample set is randomly sampled with replacement from the training sample set;

[0158] At each split node of each decision tree , a number of features is randomly selected from the 7 features of the feature vector, and an optimal split feature and a split threshold are selected based on an information gain criterion;

[0159] Constructing complete decision tree until stopping criteria is met: number of samples in the node is less than 5 or depth of the tree reaches 10;

[0160] Combining the constructed decision trees into a random forest classifier For a new feature vector , the majority voting mechanism is adopted to determine the final classification result:

[0161] ;

[0162] where denotes the classification voting result of the th decision tree on the feature vector ;

[0163] The final classification result is the class label with the highest frequency among ; ;

[0164] S54: Output the bolt surface defect detection result;

[0165] Based on the random forest classification result, the final detection result of the bolt surface defect is generated, specifically:

[0166] For each connected region classified as an abnormal class, record its defect information, including:

[0167] Defect type ;

[0168] Defect position , defined as the centroid coordinates of the connected region ;

[0169] Defect severity , quantified based on the defect area: when the defect area is less than 50 pixels, it is slight; when the defect area is greater than or equal to 50 pixels and less than 200 pixels, it is moderate; when the defect area is greater than or equal to 200 pixels, it is severe.

[0170] It should be noted that this step realizes multi-dimensional feature description of bolt defects by extracting the combination of geometric features and texture features. This comprehensive feature extraction strategy has stronger discrimination ability than single feature. Geometric features such as area, aspect ratio and circularity can effectively characterize the shape and size features of defects. The aspect ratio is particularly suitable for distinguishing linear cracks and circular pits, and the circularity can accurately identify regular-shaped defects. Texture features such as contrast, energy, homogeneity and entropy calculated by the gray level co-occurrence matrix can deeply analyze the gray level distribution pattern and texture complexity of the defect area. These features are of great significance for distinguishing different types of surface defects.

[0171] This step uses a random forest classifier which has better generalization ability and anti-overfitting performance than traditional single classifiers. Random forest can effectively reduce the risk of overfitting of single decision tree and improve the stability and reliability of classification by constructing multiple decision trees and using voting mechanism. Each decision tree uses different sample subsets and feature subsets during training, which makes the classifier fully utilize the diversity of feature space and avoid misclassification caused by local noise or abnormal values of certain features. At the same time, the ensemble learning characteristics of random forest make the final classification result more robust, which can handle various complex situations that may occur in bolt defect detection.

[0172] Embodiment 2: The application also discloses a bolt surface defect detection system based on computer vision, which comprises the following five modules:

[0173] Image acquisition and preprocessing module: a multi-camera array arranged in a ring is used to acquire 360-degree images of the bolt, and a set of original multi-angle images of the bolt surface are obtained and denoised and brightness equalized;

[0174] Image enhancement module: a frequency domain analysis method based on Fourier transform is used to identify the periodic texture features of the thread; and an adaptive filter is designed according to the periodic features of the thread to enhance the images in the set of preprocessed bolt surface images;

[0175] Multi-scale feature extraction module: multi-scale morphological operation is used on the enhanced set of bolt surface images to extract morphological gradient features at different scales; morphological features of different types of defects such as cracks, pits and scratches are extracted by using structure elements specific to bolt defect types;

[0176] Segmentation module: local statistical features are extracted from the set of multi-scale feature maps, and an adaptive threshold segmentation method based on local statistical features is used; the segmentation threshold is dynamically adjusted in combination with the thread direction information and local contrast to segment potential defect regions from the multi-scale feature maps, and a set of binary defect candidate region maps is output;

[0177] The defect identification module: the binary image set of the segmented defect candidate region is extracted for geometric features and texture features; based on the extracted features, a random forest classifier is used to identify and classify the defects, and finally the detection results of the bolt surface defects are output, including the defect type, position and severity.

[0178] It should be noted that the above-mentioned embodiment numbers of the application are only for description, not representing the advantages and disadvantages of the embodiments. And the terms "include", "contain" or any other variants thereof in this paper are intended to cover non-exclusive inclusion, so that the process, device, article or method including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, device, article or method. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, device, article or method including the element.

[0179] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by software and necessary general hardware platform, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the application can be embodied in the form of software product, which is stored in a storage medium (such as ROM / RAM, magnetic disc, optical disc) as described above, including a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server or network device) execute the methods described in various embodiments of the application.

[0180] The above is only the preferred embodiment of the application, and does not limit the patent scope of the application, and any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields based on the content of the specification and drawings of the application, are also included in the patent protection scope of the application.

Claims

1. A method for detecting bolt surface defects based on computer vision, characterized in that, Includes the following steps: S1: The bolt is captured by a multi-camera array arranged in a ring, and the original multi-angle image set of the bolt surface is obtained. The image set is then processed for noise reduction and brightness equalization, and the pre-processed bolt surface image set is output. S2: The frequency domain analysis method based on Fourier transform is used to identify the periodic texture features of the thread after the preprocessed bolt surface image set; based on the periodic features of the thread, an adaptive filter is designed to enhance the images in the preprocessed bolt surface image set, and the enhanced bolt surface image set is output. Step S2 includes: S21: Perform Fourier transform on each image in the preprocessed bolt surface image set to convert the image from the spatial domain to the frequency domain. By analyzing the power spectral density distribution represented by the frequency domain of each image, identify the periodic texture features of the thread structure and determine the fundamental frequency and harmonic frequency set of the thread. S22: Perform Sobel edge detection on each image in the preprocessed bolt surface image set, calculate the gradient direction angle, construct the direction angle histogram, and determine the thread direction vector; Step S22 includes: To determine the thread direction vector First, regarding the first Preprocessed bolt surface images from various angles To perform Sobel edge detection, the gradient direction angle of each pixel is calculated by using a Sobel operator with a convolution kernel in the horizontal direction. and vertical convolution kernels calculate exist gradient components of direction and gradient components of direction : ， ； Calculate gradient direction angle : ,in It is the arctangent function; Calculate the gradient direction angles of all pixels and construct a histogram of direction angles. ,in The range of values ​​is The angle range is divided into 36 intervals, each interval being 5° wide; Thread direction vector Determined as a direction angle histogram The center angle value of the angle interval that appears most frequently in the range; S23: Based on the identified set of thread fundamental frequency and harmonic frequencies, design an adaptive bandpass filter and define the normal thread frequency response function and the defect frequency response function; S24: Apply the adaptive filter to the frequency domain for frequency domain filtering, and obtain the enhanced bolt surface image set through inverse Fourier transform; S3: Multi-scale morphological operations are performed on the enhanced bolt surface image set to extract morphological gradient features at different scales; by using a set of structural elements for bolt defect types, morphological feature maps of different types of defects such as cracks, pits and scratches are extracted respectively, and a set of multi-scale feature maps is output. S4: Extract local statistical features from the multi-scale feature map set, adopt an adaptive threshold segmentation method based on local statistical features, combine thread direction information and local contrast, dynamically adjust the segmentation threshold, segment the potential defect region from the multi-scale feature map set, and output a set of binary maps of defect candidate regions. S5: Extract geometric and texture features from the binary map set of defect candidate regions obtained by segmentation; based on the extracted features, use a random forest classifier to identify and classify defects, and finally output the detection results of bolt surface defects, including defect type, location and severity.

2. The method for detecting bolt surface defects based on computer vision according to claim 1, characterized in that, Step S1 includes: S11: A multi-camera array arranged in a ring is used to acquire 360-degree images of the bolt. n high-resolution industrial cameras are arranged around the bolt at equal angular intervals to form a ring camera array, thereby obtaining a set of original multi-angle images of the bolt surface. S12: Apply a bilateral filtering algorithm to each image in the original multi-angle image set for smoothing and denoising to obtain the denoised image; adjust the brightness distribution of the denoised image by using a histogram equalization algorithm to take into account the differences in lighting conditions when shooting from different angles, and output the preprocessed bolt surface image set.

3. The method for detecting bolt surface defects based on computer vision according to claim 1, characterized in that, Step S3 includes: S31: Design corresponding sets of structural elements for the three common defect types on bolt surfaces, including: a set of linear structural elements for detecting linear crack defects, a set of circular structural elements for detecting circular pit defects, and a set of elliptical structural elements for detecting irregular scratch defects. S32: Morphological gradient operations are performed on the enhanced bolt surface image using sets of structural elements of different scales to obtain linear gradient feature map sets, circular gradient feature map sets, and elliptical gradient feature map sets, respectively. S33: Combine all morphological gradient feature maps of each angle image in the enhanced bolt surface image set to construct a complete multi-scale feature map set.

4. The method for detecting bolt surface defects based on computer vision according to claim 3, characterized in that, Step S32 includes: Morphological gradient operations are performed on the enhanced bolt surface image using linear structuring elements to obtain a set of linear gradient feature maps; where the morphological gradient operation is defined as the result of dilation operation minus the result of corrosion operation. The enhanced bolt surface image is subjected to morphological gradient operation using circular structuring elements to obtain a set of circular gradient feature maps. The enhanced bolt surface image is subjected to morphological gradient operation using an elliptical structuring element set to obtain an elliptical gradient feature map set. The length of the linear structuring element can take values ​​ranging from 1 to 2. The radius of the circular structuring element is in the range of pixels. The range of values ​​for the major axis of the pixel elliptical structuring element is: The range of pixel and minor axis values ​​is: Pixel.

5. The method for detecting bolt surface defects based on computer vision according to claim 3, characterized in that, Step S4 includes: S41: Extract local statistical features for each multi-scale feature map in the multi-scale feature map set, and calculate the local mean and local standard deviation; S42: Dynamically calculate the adaptive segmentation threshold for each pixel in the multi-scale feature map based on local statistical features and contrast weights; S43: Use an adaptive segmentation threshold to perform binarization segmentation on the multi-scale feature map, and use morphological post-processing and connected component analysis algorithms to output a set of binary maps of defect candidate regions.

6. The method for detecting bolt surface defects based on computer vision according to claim 5, characterized in that, Step S5 includes: S51: Mark the connected regions in the binary map of each defect candidate region, and calculate the geometric features of each connected region, including area, aspect ratio of the bounding rectangle, and roundness; S52: Extract the texture features of the binary map of each defect candidate region, calculate the gray-level co-occurrence matrix in the connected regions, and calculate the contrast features, energy features, homogeneity features and entropy features; S53: Construct feature vectors and use a random forest classifier for classification, set the number of decision trees to 100, and use a majority voting mechanism to determine the final classification result; S54: Generate the final detection results of bolt surface defects based on the final classification results, including defect type, location and severity.

7. A computer vision-based bolt surface defect detection system, characterized in that, include: Image acquisition and preprocessing module: The bolt is captured by a multi-camera array arranged in a ring, and the original multi-angle image set of the bolt surface is obtained and then denoised and brightness equalized. Image enhancement module: Employs a frequency domain analysis method based on Fourier transform to identify the periodic texture features of the thread; Based on the periodic characteristics of the thread, an adaptive filter is designed to enhance the images in the preprocessed bolt surface image set; Multi-scale feature extraction module: Multi-scale morphological operations are performed on the enhanced bolt surface image set to extract morphological gradient features at different scales; Morphological feature maps of different types of defects such as cracks, pits and scratches are extracted by using structural elements for bolt defect types. Segmentation module: Extracts local statistical features from the multi-scale feature map set and adopts an adaptive threshold segmentation method based on local statistical features; combines thread direction information and local contrast to dynamically adjust the segmentation threshold, segment the potential defect region from the multi-scale feature map, and outputs a set of binary maps of defect candidate regions; Defect identification module: Extracts geometric and texture features from the binary image set of segmented defect candidate regions; Based on the extracted features, a random forest classifier is used to identify and classify defects, and finally outputs the detection results of bolt surface defects, including defect type, location and severity. To achieve the computer vision-based bolt surface defect detection method as described in any one of claims 1-6.

Images