Bolt surface defect detection method and system based on computer vision

Abstract

The invention discloses a bolt surface defect detection method and system based on computer vision. The method comprises the steps that 360-degree image collection and preprocessing are conducted on a bolt through an annularly-arranged multi-camera array; fourier transform is adopted to analyze thread periodic texture features, and an adaptive filter is designed to enhance the image; multi-scale morphological operation is adopted, and morphological gradient features are extracted by using structural elements for crack, pit and scratch type defects; a self-adaptive threshold segmentation method based on local statistical characteristics is combined with thread direction information to dynamically adjust a segmentation threshold, and defect candidate areas are segmented; and extracting geometric and textural features of the candidate region, and identifying the type, position and severity of the defect by adopting a random forest classifier. According to the invention, high-precision automatic detection and classification of various defects on the surface of the bolt are realized.

Description

Technical Field

[0001] This invention relates to the technical field of industrial automation inspection, and in particular to a method and system for detecting bolt surface defects based on computer vision. Background Technology

[0002] Bolts, as fundamental components of mechanical connections, are widely used in key industrial sectors such as aerospace, petrochemicals, power, and transportation. The surface quality of bolts directly affects connection strength and service life. Surface defects such as cracks, pits, and scratches can severely weaken the bolt's load-bearing capacity, potentially leading to connection failure or even major safety accidents. As industrial equipment develops towards higher precision and reliability, the requirements for bolt surface quality inspection are becoming increasingly stringent. Traditional manual visual inspection methods are not only inefficient and highly subjective but also struggle to detect minute defects, failing to meet the quality control needs of modern industry. Therefore, developing high-precision, automated bolt surface defect detection technology has become an urgent need in the field of industrial quality control.

[0003] Existing computer vision defect detection methods have many limitations in bolt surface defect detection applications. For example, the hardware part defect detection method disclosed in patent CN202211229527.2 uses simple threshold segmentation based on grayscale boundary values ​​and pixel membership calculation to identify defects. This method relies solely on pixel grayscale information and lacks in-depth analysis of the periodic texture features of the bolt surface threads. This method uses a fixed grayscale boundary value formula to distinguish between high and low brightness pixels. This global thresholding method cannot adapt to the local illumination changes and texture complexity caused by the thread structure on the bolt surface. Furthermore, this method only calculates membership based on the number of low-brightness pixels in the neighborhood and the grayscale value structure entropy, lacking targeted analysis of the defect's geometric features. It is difficult to distinguish different types of defects such as cracks, pits, and scratches, resulting in limited detection accuracy and defect classification capabilities.

[0004] Another significant drawback of existing technologies is the lack of multi-angle detection capabilities. As three-dimensional cylindrical parts, bolts may have surface defects distributed in any location and orientation. Single-view detection methods are prone to creating blind spots, leading to missed defects. Furthermore, existing methods generally lack effective utilization of the periodic texture features of the threads. The regular thread structure on the bolt surface contains rich frequency domain information, which can be used to distinguish normal textures from defect features. However, traditional methods mostly employ spatial domain processing, failing to fully exploit the discriminative value of frequency domain features. In addition, existing methods often use single-scale or fixed-template processing for feature extraction, lacking adaptive processing capabilities for defects of different sizes and types, making it difficult to simultaneously detect micro-cracks and large-area surface damage.

[0005] In terms of classification and recognition, existing technologies mostly employ simple threshold judgments or empirical rule-based methods, lacking the support of intelligent classification algorithms. These methods struggle to handle complex multi-feature fusion and nonlinear classification problems, often exhibiting high false detection and false negative rates when faced with complex texture backgrounds and diverse defect types on bolt surfaces. Furthermore, existing methods lack quantitative evaluation mechanisms for detection results, failing to provide accurate information on defect severity for engineering maintenance decisions, thus limiting their widespread adoption in practical industrial applications. Summary of the Invention

[0006] In view of this, the present invention provides a method and system for detecting bolt surface defects based on computer vision. The purpose is to combine technologies such as multi-angle image acquisition, frequency domain texture analysis, multi-scale morphological feature extraction, adaptive threshold segmentation and intelligent classification and recognition to achieve high-precision automatic detection and classification of various defects such as cracks, pits and scratches on bolt surfaces, and to provide reliable technical support for the safe maintenance of industrial equipment.

[0007] To achieve the above objectives, the present invention provides a computer vision-based method for detecting surface defects in bolts, comprising the following steps: S1: The bolt is captured by a multi-camera array arranged in a ring, and the original multi-angle image set of the bolt surface is obtained. The image set is then processed for noise reduction and brightness equalization, and the pre-processed bolt surface image set is output. S2: The frequency domain analysis method based on Fourier transform is used to identify the periodic texture features of the thread after the preprocessed bolt surface image set; based on the periodic features of the thread, an adaptive filter is designed to enhance the images in the preprocessed bolt surface image set, and the enhanced bolt surface image set is output. S3: Multi-scale morphological operations are performed on the enhanced bolt surface image set to extract morphological gradient features at different scales; by using a set of structural elements for bolt defect types, morphological feature maps of different types of defects such as cracks, pits and scratches are extracted respectively, and a set of multi-scale feature maps is output. S4: Extract local statistical features from the multi-scale feature map set, adopt an adaptive threshold segmentation method based on local statistical features, combine thread direction information and local contrast, dynamically adjust the segmentation threshold, segment the potential defect region from the multi-scale feature map set, and output a set of binary maps of defect candidate regions. S5: Extract geometric and texture features from the binary map set of defect candidate regions obtained by segmentation; based on the extracted features, use a random forest classifier to identify and classify defects, and finally output the detection results of bolt surface defects, including defect type, location and severity.

[0008] Optionally, in step S1, a multi-camera array arranged in a ring performs 360-degree image acquisition on the bolt, obtaining a set of original multi-angle images of the bolt surface. These images are then denoised and brightness equalized to output a pre-processed bolt surface image set, including: S11: A multi-camera array arranged in a ring is used to acquire 360-degree images of the bolt. n high-resolution industrial cameras are arranged around the bolt at equal angular intervals to form a ring camera array, thereby obtaining a set of original multi-angle images of the bolt surface. S12: Apply a bilateral filtering algorithm to each image in the original multi-angle image set for smoothing and denoising to obtain the denoised image; adjust the brightness distribution of the denoised image by using a histogram equalization algorithm to take into account the differences in lighting conditions when shooting from different angles, and output the preprocessed bolt surface image set.

[0009] Optionally, in step S2, a frequency domain analysis method based on Fourier transform is used to identify the periodic texture features of the thread; based on the periodic features of the thread, an adaptive filter is designed and the images in the preprocessed bolt surface image set are enhanced, outputting an enhanced bolt surface image set, including: S21: Perform Fourier transform on each image in the preprocessed bolt surface image set to convert the image from the spatial domain to the frequency domain. By analyzing the power spectral density distribution represented by the frequency domain of each image, identify the periodic texture features of the thread structure and determine the fundamental frequency and harmonic frequency set of the thread. S22: Perform Sobel edge detection on each image in the preprocessed bolt surface image set, calculate the gradient direction angle, construct the direction angle histogram, and determine the thread direction vector; S23: Based on the identified set of thread fundamental frequency and harmonic frequencies, design an adaptive bandpass filter and define the normal thread frequency response function and the defect frequency response function; S24: Apply the adaptive filter to the frequency domain for frequency domain filtering, and obtain the enhanced bolt surface image set through inverse Fourier transform.

[0010] Optionally, step S22 includes: To determine the thread direction vector First, regarding the first Preprocessed bolt surface images from various angles To perform Sobel edge detection, the gradient direction angle of each pixel is calculated by using a Sobel operator with a convolution kernel in the horizontal direction. and vertical convolution kernels calculate exist gradient components of direction and gradient components of direction : , ; Calculate gradient direction angle : ,in It is the arctangent function; Calculate the gradient direction angles of all pixels and construct a histogram of direction angles. ,in The range of values ​​is The angle range is divided into 36 intervals, each interval being 5° wide; Thread direction vector Determined as a direction angle histogram The center angle value of the angle range that appears most frequently.

[0011] Optionally, in step S3, multi-scale morphological operations are performed on the enhanced bolt surface image set to extract morphological gradient features at different scales; by using a set of structural elements for different bolt defect types, morphological feature maps of cracks, pits, and scratches are extracted respectively, and a multi-scale feature map set is output, including: S31: Design corresponding sets of structural elements for the three common defect types on bolt surfaces, including: a set of linear structural elements for detecting linear crack defects, a set of circular structural elements for detecting circular pit defects, and a set of elliptical structural elements for detecting irregular scratch defects. S32: Morphological gradient operations are performed on the enhanced bolt surface image using sets of structural elements of different scales to obtain linear gradient feature map sets, circular gradient feature map sets, and elliptical gradient feature map sets, respectively. S33: Combine all morphological gradient feature maps of each angle image in the enhanced bolt surface image set to construct a complete multi-scale feature map set.

[0012] Optionally, step S32 includes: Morphological gradient operations are performed on the enhanced bolt surface image using linear structuring elements to obtain a set of linear gradient feature maps; where the morphological gradient operation is defined as the result of dilation operation minus the result of corrosion operation. The enhanced bolt surface image is subjected to morphological gradient operation using circular structuring elements to obtain a set of circular gradient feature maps. The enhanced bolt surface image is subjected to morphological gradient operation using an elliptical structuring element set to obtain an elliptical gradient feature map set. The length of the linear structuring element can take values ​​ranging from 1 to 2. The radius of the circular structuring element is in the range of pixels. The range of values ​​for the major axis of the pixel elliptical structuring element is: The range of pixel and minor axis values ​​is: Pixel.

[0013] Optionally, in step S4, an adaptive threshold segmentation method based on local statistical features is used. This method combines thread direction information and local contrast to dynamically adjust the segmentation threshold, segmenting potential defect regions from a multi-scale feature map set and outputting a set of binary defect candidate regions, including: S41: Extract local statistical features for each multi-scale feature map in the multi-scale feature map set, and calculate the local mean and local standard deviation; S42: Dynamically calculate the adaptive segmentation threshold for each pixel in the multi-scale feature map based on local statistical features and contrast weights; S43: Use an adaptive segmentation threshold to perform binarization segmentation on the multi-scale feature map, and use morphological post-processing and connected component analysis algorithms to output a set of binary maps of defect candidate regions.

[0014] Optionally, in step S5, geometric and texture features are extracted from the binary image set of the segmented defect candidate regions; based on the extracted features, a random forest classifier is used to identify and classify the defects, and finally, the detection results of bolt surface defects are output, including defect type, location, and severity, including: S51: Mark the connected regions in the binary map of each defect candidate region, and calculate the geometric features of each connected region, including area, aspect ratio of the bounding rectangle, and roundness; S52: Extract the texture features of the binary map of each defect candidate region, calculate the gray-level co-occurrence matrix in the connected regions, and calculate the contrast features, energy features, homogeneity features and entropy features; S53: Construct feature vectors and use a random forest classifier for classification, set the number of decision trees to 100, and use a majority voting mechanism to determine the final classification result; S54: Generate the final detection results of bolt surface defects based on the final classification results, including defect type, location and severity.

[0015] This invention also discloses a computer vision-based bolt surface defect detection system, comprising: Image acquisition and preprocessing module: The bolt is captured by a multi-camera array arranged in a ring, and the original multi-angle image set of the bolt surface is obtained and then denoised and brightness equalized. Image enhancement module: A frequency domain analysis method based on Fourier transform is used to identify the periodic texture features of the thread; based on the periodic features of the thread, an adaptive filter is designed to enhance the images in the preprocessed bolt surface image set; Multi-scale feature extraction module: Multi-scale morphological operations are performed on the enhanced bolt surface image set to extract morphological gradient features at different scales; Morphological feature maps of different types of defects such as cracks, pits and scratches are extracted by using structural elements for bolt defect types. Segmentation module: Extracts local statistical features from the multi-scale feature map set and adopts an adaptive threshold segmentation method based on local statistical features; combines thread direction information and local contrast to dynamically adjust the segmentation threshold, segment the potential defect region from the multi-scale feature map, and outputs a set of binary maps of defect candidate regions; Defect identification module: Extracts geometric and texture features from the binary map set of segmented defect candidate regions; based on the extracted features, a random forest classifier is used to identify and classify defects, and finally outputs the detection results of bolt surface defects, including defect type, location and severity.

[0016] Compared with the prior art, the present invention has at least the following beneficial effects: This invention employs a Fourier transform-based periodic texture analysis and adaptive filtering enhancement algorithm to accurately identify the frequency characteristics of bolt threads and design targeted filters. Compared to traditional spatial domain filtering methods, frequency domain analysis can accurately extract the fundamental frequency and harmonic information of the thread. By constructing normal thread frequency response functions and defect frequency response functions, it achieves the suppression of normal textures and the enhancement of defect features.

[0017] The multi-scale morphological feature extraction algorithm designed in this invention can perform specialized detection of the geometric features of different types of defects. By designing three different sets of structural elements—linear, circular, and elliptical—corresponding to three typical defect types—cracks, dents, and scratches—matching detection based on defect geometric features is achieved. The multi-scale processing mechanism, through the combination of structural elements of different sizes, can detect defects of various sizes, from micro-cracks to large scratches, avoiding the limitations of single-scale methods.

[0018] This invention proposes an intelligent identification method that combines an adaptive threshold segmentation algorithm based on local statistical features with a random forest classifier, significantly improving the accuracy and reliability of defect detection. The adaptive threshold algorithm calculates statistical features within a local window and, combined with a contrast weighting mechanism, dynamically adjusts the optimal segmentation threshold for each pixel, effectively solving the segmentation difficulties caused by complex bolt surface textures and uneven lighting. The random forest classifier, through a voting mechanism integrating multiple decision trees, utilizes the comprehensive discriminative power of geometric and texture features to accurately classify cracks, dents, scratches, and normal areas. Attached Figure Description

[0019] Figure 1This is a flowchart illustrating a computer vision-based bolt surface defect detection method according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the original image of the bolt surface according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the enhanced bolt surface according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a binary image of a defect candidate region according to an embodiment of the present invention. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings, but this is not intended to limit the present invention in any way. Any modifications or substitutions made based on the teachings of the present invention shall fall within the protection scope of the present invention.

[0021] Example 1: This invention provides a method for detecting bolt surface defects based on computer vision, such as... Figure 1 As shown, it includes the following steps: S1: A multi-camera array arranged in a ring is used to acquire 360-degree images of the bolt, obtaining a set of original multi-angle images of the bolt surface. The acquired images are then denoised and brightness equalized, and the pre-processed bolt surface image set is output. S11: Acquire a set of original multi-angle images of the bolt surface; A circular camera array is used to acquire 360-degree images of the bolts, specifically: Will Several high-resolution industrial cameras are arranged at equal angular intervals around the bolt, forming a circular camera array, wherein the angular interval between the cameras is [missing information]. Each camera is recorded as ,in Obtain the original multi-angle image set of the bolt surface. ,in and They represent the first Taiwan and the Raw bolt surface images captured by a camera, such as Figure 2 As shown; S12: Perform image denoising and brightness equalization; For the original multi-angle image set Each image in Denoising is performed by using a bilateral filtering algorithm to smooth and denoise the image, resulting in... The corresponding denoised image ; To address the differences in lighting conditions when shooting from different angles, the noise-reduced image was modified accordingly. Brightness equalization is performed by adjusting the brightness using a histogram equalization algorithm. The brightness distribution ensures that images from all angles have consistent brightness characteristics, resulting in preprocessed bolt surface images and outputting a set of preprocessed bolt surface images. ,in and They represent the first The and the first Preprocessed bolt surface images from various angles.

[0022] It should be noted that this step utilizes a ring-shaped multi-camera array system to achieve omnidirectional image acquisition of the bolt surface. Compared to traditional single-camera or fixed-viewpoint detection methods, this approach can acquire complete three-dimensional geometric information and texture details of the bolt surface. This 360-degree full-coverage acquisition method ensures that defects at any location on the bolt surface can be clearly captured by at least one camera.

[0023] The bilateral filtering algorithm used in this step effectively removes image noise while preserving edge information, which is crucial for subsequent defect detection. Defects such as cracks and scratches on bolt surfaces often exhibit subtle edge features. Traditional Gaussian or mean filtering blurs this critical edge information, while bilateral filtering, by simultaneously considering spatial distance and pixel intensity differences, can smooth noisy areas while maintaining the sharpness of defect edges. S2: The frequency domain analysis method based on Fourier transform is used to identify the periodicity characteristics of the threads in the preprocessed bolt surface image set; based on the periodicity characteristics, an adaptive filter is designed to enhance the images in the preprocessed bolt surface image set, and the enhanced bolt surface image set is output: S21: Analyze the periodic characteristics of the thread; Preprocessed bolt surface image set Each image in Performing a Fourier transform converts the image from the spatial domain to the frequency domain, yielding... Corresponding frequency domain representation ; Through analysis power spectral density distribution Identify the periodic texture features of the thread structure; locate peak points in the power spectrum to determine the fundamental frequency of the thread. This is the non-zero frequency component with the largest amplitude in the power spectrum; Calculate the set of harmonic frequencies of the thread. ,in Indicates the first One harmonic frequency component Indicates the first One harmonic frequency component The number of effective harmonics is set; S22: Perform thread direction detection and quantification; To determine the thread direction vector First, regarding the first Image preprocessing completed from each angle Perform Sobel edge detection, calculate the gradient direction for each pixel; use the Sobel operator to perform convolution in the horizontal direction. and vertical convolution kernels calculate exist gradient components of direction and gradient components of direction : , ; and They are: ; Calculate gradient direction angle : ,in, It is the arctangent function; Calculate the gradient direction angles of all pixels and construct a histogram of direction angles. ,in The range of values ​​is The angle range is divided into 36 intervals, each interval being 5° wide; Thread direction vector Determined as a histogram The center angle value of the angle interval that appears most frequently in the range; S23: Adaptive Filter Design and Construction; Based on the identified thread fundamental frequency Harmonic frequency set Design an adaptive bandpass filter ; Define the normal thread frequency response function The Gaussian function centered at the fundamental frequency and harmonic frequency of the thread: ; in The normal thread frequency response function in frequency components The value at; The parameter representing the standard deviation of the normal thread frequency response; where and These represent the frequency components in the horizontal and vertical directions, respectively; It is a natural constant; Define the defect frequency response function Other frequency components in the frequency domain besides the normal thread frequency: ; The transfer function of the adaptive filter is: ; in Indicates the defect enhancement factor. Indicates the normal texture suppression coefficient; S24: Perform image enhancement processing; The designed adaptive filter Applied to Corresponding frequency domain representation Frequency domain filtering is performed: ,get The corresponding enhanced frequency domain image ; right The corresponding enhanced frequency domain image Performing the inverse Fourier transform, we obtain the first... Enhanced bolt surface images from various angles Output enhanced bolt surface image set ,in Indicates the first Enhanced bolt surface images from various angles, such as Figure 3 As shown.

[0024] It should be noted that this step employs a frequency domain analysis method based on Fourier transform, which can accurately identify the periodic texture features of the thread, a feat difficult to achieve with traditional spatial domain image processing methods. The threads on the bolt surface exhibit a distinct periodic geometric structure, manifesting as energy concentrations of specific frequency components in the frequency domain. Through power spectral density analysis, this step can accurately locate the fundamental frequency and harmonic frequencies of the thread, thereby establishing a frequency characteristic model of the thread texture. This frequency domain feature extraction method is unaffected by thread direction and local deformation, exhibiting strong robustness.

[0025] This step achieves precise quantification of thread direction using Sobel edge detection and orientation angle statistics. Traditional texture analysis methods often assume that the texture direction is known or fixed, while the installation angle of bolts may deviate in actual inspection. This step automatically adapts to different thread directions by constructing an orientation angle histogram and finding the most frequent angle intervals, ensuring the accuracy of subsequent filter design. This adaptive orientation detection mechanism significantly improves the practicality and reliability of the detection system.

[0026] S3: Multi-scale morphological operations are applied to the enhanced bolt surface image set to extract morphological gradient features at different scales; by using a set of structural elements for bolt defect types, morphological features of different types of defects such as cracks, pits, and scratches are extracted, and a multi-scale feature map set is output: S31: Design multi-scale structural elements; For the three common types of defects on bolt surfaces, a corresponding set of structural elements is designed, specifically as follows: To detect linear crack defects, a linear structural element was designed. Its shape is a length of A linear structure with a width of 1 pixel, wherein The range of values ​​is Pixels form a set of linear structural elements. ,in Indicates length is Linear structuring elements of pixels; To detect circular pit defects, a circular structural element was designed. Its shape is a radius of The circular structure, in which The range of values ​​is Pixels form a circular structural element set. ,in Indicates radius as A circular structural element of pixels; To detect irregular scratch defects, an elliptical structural element was designed. Its shape is such that its major axis is The minor axis is The elliptical structure, in which The range of values ​​is Pixels The range of values ​​is Pixels form an elliptical structural element set. ,in Indicates the major axis as Pixels, minor axis is An elliptical structural element of pixels; S32: Perform multi-scale morphological gradient calculations; Enhanced bolt surface image set Each image in Morphological gradient operations were performed using structuring element sets of different scales, specifically: Using a linear structure element set right Perform morphological gradient operations to obtain a set of linear gradient feature maps. ,in Indicates the use of length is Linear structuring elements of pixels For the Enhanced bolt surface images from various angles The result of morphological gradient operation, where ⊕ represents morphological dilation operation. Represents morphological erosion operations; Use a circular structural element set right Perform morphological gradient operations to obtain a set of circular gradient feature maps. ,in Indicates the use of radius is Pixel circular structural elements For the Enhanced bolt surface images from various angles The result of morphological gradient calculation; Using an elliptical structure element set right Perform morphological gradient operations to obtain a set of elliptical gradient feature maps. ,in Indicates the use of the major axis as Pixels, minor axis is Pixel elliptical structural element For the Enhanced bolt surface images from various angles The result of morphological gradient calculation; S33: Construct a multi-scale feature map set; Images from each angle All morphological gradient feature maps are combined to form the first... Multi-scale feature maps from various angles Specifically: ; For all Images from various angles are used to construct a complete multi-scale feature map set. ,in Indicates the first Multi-scale feature maps from multiple perspectives.

[0027] It should be noted that this step employs multi-scale morphological operations to achieve targeted feature extraction for different types of bolt defects. This method has a stronger defect recognition capability compared to traditional single-filter or edge detection algorithms. By designing structural elements specifically for three typical defects—cracks, dents, and scratches—this step can fully utilize the geometric features of the defects for matching and detection. Linear structural elements have the best response to slender crack defects, circular structural elements can effectively detect circular or near-circular dent defects, while elliptical structural elements are specifically used to identify irregular scratch defects. This structural element design strategy based on defect geometry significantly improves the targeting and accuracy of detection.

[0028] S4: Extract local statistical features from the multi-scale feature map set. Based on the local texture characteristics of the bolt surface, adopt an adaptive threshold segmentation method based on local statistical features. Combine thread direction information and local contrast, dynamically adjust the segmentation threshold to segment potential defect regions from the multi-scale feature map set, and output a set of binary images of defect candidate regions. S41: Extract local statistical features and construct a set of morphological gradient feature maps; For a complete multi-scale feature map set Each multi-scale feature map in Extracting local statistical features, specifically: In the Multi-scale feature maps from various angles In this process, local statistical analysis is performed on each morphological gradient feature map, with the local window size set to [value missing]. ,in Pixel; For a set of linear gradient feature maps Each feature map in Calculate the local mean within each local window. and local standard deviation This yields the set of local statistical features of the linear gradient feature map. ; For a circular gradient feature map set Each feature map in Calculate the local mean within each local window. and local standard deviation This yields the set of local statistical features of the circular gradient feature map. ; For an elliptical gradient feature map set Each feature map in Calculate the local mean within each local window. and local standard deviation This yields the set of local statistical features of the elliptical gradient feature map. ; S42: Dynamically adjust the adaptive segmentation threshold; Dynamically calculate adaptive segmentation threshold based on local statistical features. Specifically: For each multi-scale feature map in the multi-scale feature map set, calculate the basic threshold. : ; in This represents the global mean of the multi-scale feature map. This represents the global standard deviation of the multi-scale feature map. Indicates the threshold adjustment factor; Combine contrast weight Calculate the position of each pixel Adaptive segmentation threshold : ; in This represents the contrast weight adjustment coefficient; pixel position Contrast weights, , and These represent the pixel positions of the currently computed feature map. Local mean and local standard deviation at a given location; S43: Segment the candidate region for defects; Using dynamically adjusted adaptive segmentation thresholds Binarization segmentation of the multi-scale feature map is performed as follows: For each pixel in the multi-scale feature map set Perform threshold determination: When the pixel value of the multi-scale feature map is greater than When the pixel value is less than or equal to 1, it is marked as a defect candidate pixel and assigned a value of 1; when the pixel value of the multi-scale feature map is less than or equal to 1, it is marked as a defect candidate pixel and assigned a value of 1. When this happens, the pixel is marked as a background pixel and assigned a value of 0; Morphological post-processing was performed on the segmentation results, using a radius of... The circular structuring element of the pixel is closed to fill the small holes in the defective area; Using a connected component analysis algorithm, remove components with areas smaller than [the specified area]. The noise region of a pixel, the retained area is greater than or equal to Connected regions are considered as valid candidate regions for defects; Output a set of binary maps of candidate defect regions ,in and They represent the first The and the first Binary images of candidate defect regions from various angles, such as... Figure 4 As shown.

[0029] It should be noted that this step employs an adaptive thresholding method based on local statistical features, which offers significant advantages over traditional global or fixed thresholding. Due to the presence of the threaded structure, the texture characteristics and lighting conditions vary considerably across different regions of the bolt surface, making fixed thresholds often unsuitable for the segmentation needs of all regions simultaneously. This step, by calculating statistical features such as the mean and standard deviation within a local window, fully reflects the local texture characteristics of each region, thereby calculating the most suitable segmentation threshold for each pixel. This locally adaptive strategy ensures ideal segmentation results across different texture regions.

[0030] S5: Extract geometric and texture features from the binary image set of the segmented defect candidate regions; based on the extracted features, use a random forest classifier to identify and classify defects, and finally output the detection results of bolt surface defects, including defect type, location, and severity: S51: Extract the geometric features of the binary map set of defect candidate regions; Binary map set of defect candidate regions Binary map of each defect candidate region in Extract the geometric features of each connected region, specifically: For the Binary map of candidate defect regions from various angles By labeling connected components, we obtain a set of connected components. ,in Indicates the first The first angle binary image of the first angle A connected region, Indicates the first The first angle binary image of the first angle A connected region, Indicates the first The total number of connected regions in a binary graph from each angle; For each connected region Calculate its geometric features, specifically including: Calculate the area of ​​the connected region Defined as a connected region The total number of pixels contained; Calculate the circumscribed rectangle of the connected region to obtain the width of the rectangle. and height And calculate the aspect ratio. ,in functions and The function retrieves the maximum and minimum values ​​of the input numbers, respectively. Calculate the circularity of a connected region Defined as: ,in Represents connected regions circumference, Represents pi; S52: Extract texture features from candidate defect regions; For each connected region From the corresponding number A pre-processed bolt surface image Extracting the texture features of this region specifically involves: Calculate the gray-level co-occurrence matrix within connected regions. Set the pixel spacing to The value is 1, and the direction angle is... ; Based on gray-level co-occurrence matrix Calculate texture feature parameters: Calculate contrast features : ,in and These represent the row and column indices of the gray-level co-occurrence matrix, respectively. for In row index and column indexes The value at; Calculate energy characteristics : ; Calculate homogeneity features : ; Calculate entropy features : ,in This refers to logarithmic operations with base 2. S53: Construct a random forest classifier and perform feature classification; Each connected region The geometric and texture features are combined to construct a feature vector, which is then classified using a random forest classifier. Specifically: For each connected region Construct feature vectors : ; Building a Random Forest Classifier Set the number of decision trees The number of features randomly selected by each decision tree when splitting nodes. ; Random Forest Classifier The construction process includes: For each decision tree , A sample set is randomly drawn with replacement from the training sample set. ; In each decision tree At each split node, features are randomly selected from the seven features of the feature vector. Based on the information gain criterion, the optimal splitting feature and splitting threshold are selected using one feature; Build a complete decision tree The stopping criterion is met: the number of samples in a node is less than 5 or the depth of the tree reaches 10. To build A random forest classifier is formed by combining several decision trees. For the new feature vector The final classification result is determined by majority voting. ; in Indicates the first Each decision tree pairs the feature vectors The results of the categorized voting; Final classification results for The most frequently occurring category label, ; S54: Outputs the results of bolt surface defect detection; Based on the random forest classification results, the final detection results of bolt surface defects are generated, as follows: For each connected region classified as abnormal Record its defect information, including: Defect types ; Defect location Defined as a connected region The coordinates of the centroid; Defect severity Quantization is based on defect area: when the defect area is less than 50 pixels, It is considered minor; when the defect area is greater than or equal to 50 pixels and less than 200 pixels, The value is moderate; when the defect area is greater than or equal to 200 pixels, It is serious.

[0031] It should be noted that this step achieves a multi-dimensional feature description of bolt defects by extracting a combination of geometric and texture features. This comprehensive feature extraction strategy has a stronger discriminative ability compared to single features. Geometric features such as area, aspect ratio, and roundness can effectively characterize the shape and size of defects. Among them, aspect ratio is particularly suitable for distinguishing between linear cracks and circular pits, while roundness can accurately identify defects with regular shapes. Texture features, such as contrast, energy, homogeneity, and entropy calculated through the gray-level co-occurrence matrix, can deeply analyze the gray-level distribution pattern and texture complexity of the defect area. These features are of great significance for distinguishing different types of surface defects.

[0032] This step utilizes a random forest classifier, which offers better generalization ability and resistance to overfitting compared to traditional single classifiers. By constructing multiple decision trees and employing a voting mechanism, random forests effectively reduce the overfitting risk that can occur with single decision trees, improving classification stability and reliability. Each decision tree uses a different subset of samples and features during training; this randomness allows the classifier to fully utilize the diversity of the feature space, avoiding misclassification due to local noise or outliers in certain features. Furthermore, the ensemble learning characteristics of random forests make the final classification results more robust, capable of handling various complex situations that may arise in bolt defect detection.

[0033] Example 2: This invention also discloses a computer vision-based bolt surface defect detection system, comprising the following five modules: Image acquisition and preprocessing module: The bolt is captured by a multi-camera array arranged in a ring, and the original multi-angle image set of the bolt surface is obtained and then denoised and brightness equalized. Image enhancement module: A frequency domain analysis method based on Fourier transform is used to identify the periodic texture features of the thread; based on the periodic features of the thread, an adaptive filter is designed to enhance the images in the preprocessed bolt surface image set; Multi-scale feature extraction module: Multi-scale morphological operations are performed on the enhanced bolt surface image set to extract morphological gradient features at different scales; Morphological feature maps of different types of defects such as cracks, pits and scratches are extracted by using structural elements for bolt defect types. Segmentation module: Extracts local statistical features from the multi-scale feature map set and adopts an adaptive threshold segmentation method based on local statistical features; combines thread direction information and local contrast to dynamically adjust the segmentation threshold, segment the potential defect region from the multi-scale feature map, and outputs a set of binary maps of defect candidate regions; Defect identification module: Extracts geometric and texture features from the binary map set of segmented defect candidate regions; based on the extracted features, a random forest classifier is used to identify and classify defects, and finally outputs the detection results of bolt surface defects, including defect type, location and severity.

[0034] It should be noted that the sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0035] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0036] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for detecting bolt surface defects based on computer vision, characterized in that, Includes the following steps: S1: The bolt is captured by a multi-camera array arranged in a ring, and the original multi-angle image set of the bolt surface is obtained. The image set is then processed for noise reduction and brightness equalization, and the pre-processed bolt surface image set is output. S2: The frequency domain analysis method based on Fourier transform is used to identify the periodic texture features of the thread after the preprocessed bolt surface image set; based on the periodic features of the thread, an adaptive filter is designed to enhance the images in the preprocessed bolt surface image set, and the enhanced bolt surface image set is output. S3: Multi-scale morphological operations are performed on the enhanced bolt surface image set to extract morphological gradient features at different scales; by using a set of structural elements for bolt defect types, morphological feature maps of different types of defects such as cracks, pits and scratches are extracted respectively, and a set of multi-scale feature maps is output. S4: Extract local statistical features from the multi-scale feature map set, adopt an adaptive threshold segmentation method based on local statistical features, combine thread direction information and local contrast, dynamically adjust the segmentation threshold, segment the potential defect region from the multi-scale feature map set, and output a set of binary maps of defect candidate regions. S5: Extract geometric and texture features from the binary map set of defect candidate regions obtained by segmentation; based on the extracted features, use a random forest classifier to identify and classify defects, and finally output the detection results of bolt surface defects, including defect type, location and severity.

2. The method for detecting bolt surface defects based on computer vision according to claim 1, characterized in that, The step S1 comprises: S11: A multi-camera array arranged in a ring is used to acquire 360-degree images of the bolt. n high-resolution industrial cameras are arranged around the bolt at equal angular intervals to form a ring camera array, thereby obtaining a set of original multi-angle images of the bolt surface. S12: Apply a bilateral filtering algorithm to each image in the original multi-angle image set for smoothing and denoising to obtain the denoised image; adjust the brightness distribution of the denoised image by using a histogram equalization algorithm to take into account the differences in lighting conditions when shooting from different angles, and output the preprocessed bolt surface image set.

3. The method for detecting bolt surface defects based on computer vision according to claim 2, characterized in that, The step S2 comprises: S21: Perform Fourier transform on each image in the preprocessed bolt surface image set to convert the image from the spatial domain to the frequency domain. By analyzing the power spectral density distribution represented by the frequency domain of each image, identify the periodic texture features of the thread structure and determine the fundamental frequency and harmonic frequency set of the thread. S22: Perform Sobel edge detection on each image in the preprocessed bolt surface image set, calculate the gradient direction angle, construct the direction angle histogram, and determine the thread direction vector; S23: Based on the identified set of thread fundamental frequency and harmonic frequencies, design an adaptive bandpass filter and define the normal thread frequency response function and the defect frequency response function; S24: Apply the adaptive filter to the frequency domain for frequency domain filtering, and obtain the enhanced bolt surface image set through inverse Fourier transform.

4. The method for detecting bolt surface defects based on computer vision according to claim 3, characterized in that, Step S22 includes: To determine the thread direction vector First, regarding the first Preprocessed bolt surface images from various angles To perform Sobel edge detection, the gradient direction angle of each pixel is calculated by using a Sobel operator with a convolution kernel in the horizontal direction. and vertical convolution kernels calculate exist gradient components of direction and gradient components of direction : ， ； Calculate gradient direction angle : ,in It is the arctangent function; Calculate the gradient direction angles of all pixels and construct a histogram of direction angles. ,in The range of values ​​is The angle range is divided into 36 intervals, each interval being 5° wide; Thread direction vector Determined as a direction angle histogram The center angle value of the angle range that appears most frequently.

5. The method for detecting bolt surface defects based on computer vision according to claim 3, characterized in that, The step S3 comprises: S31: Design corresponding sets of structural elements for the three common defect types on bolt surfaces, including: a set of linear structural elements for detecting linear crack defects, a set of circular structural elements for detecting circular pit defects, and a set of elliptical structural elements for detecting irregular scratch defects. S32: Morphological gradient operations are performed on the enhanced bolt surface image using sets of structural elements of different scales to obtain linear gradient feature map sets, circular gradient feature map sets, and elliptical gradient feature map sets, respectively. S33: Combine all morphological gradient feature maps of each angle image in the enhanced bolt surface image set to construct a complete multi-scale feature map set.

6. The method for detecting bolt surface defects based on computer vision according to claim 5, characterized in that, Step S32 includes: Morphological gradient operations are performed on the enhanced bolt surface image using linear structuring elements to obtain a set of linear gradient feature maps; where the morphological gradient operation is defined as the result of dilation operation minus the result of corrosion operation. The enhanced bolt surface image is subjected to morphological gradient operation using circular structuring elements to obtain a set of circular gradient feature maps. The enhanced bolt surface image is subjected to morphological gradient operation using an elliptical structuring element set to obtain an elliptical gradient feature map set. The length of the linear structuring element can take values ​​ranging from 1 to 2. The radius of the circular structuring element is in the range of pixels. The range of values ​​for the major axis of the pixel elliptical structuring element is: The range of pixel and minor axis values ​​is: Pixel.

7. The method for detecting bolt surface defects based on computer vision according to claim 5, characterized in that, Step S4 includes: S41: Extract local statistical features for each multi-scale feature map in the multi-scale feature map set, and calculate the local mean and local standard deviation; S42: Dynamically calculate the adaptive segmentation threshold for each pixel in the multi-scale feature map based on local statistical features and contrast weights; S43: Use an adaptive segmentation threshold to perform binarization segmentation on the multi-scale feature map, and use morphological post-processing and connected component analysis algorithms to output a set of binary maps of defect candidate regions.

8. The method for detecting bolt surface defects based on computer vision according to claim 7, characterized in that, Step S5 includes: S51: Mark the connected regions in the binary map of each defect candidate region, and calculate the geometric features of each connected region, including area, aspect ratio of the bounding rectangle, and roundness; S52: Extract the texture features of the binary map of each defect candidate region, calculate the gray-level co-occurrence matrix in the connected regions, and calculate the contrast features, energy features, homogeneity features and entropy features; S53: Construct feature vectors and use a random forest classifier for classification, set the number of decision trees to 100, and use a majority voting mechanism to determine the final classification result; S54: Generate the final detection results of bolt surface defects based on the final classification results, including defect type, location and severity.

9. A computer vision-based bolt surface defect detection system, characterized in that, include: Image acquisition and preprocessing module: The bolt is captured by a multi-camera array arranged in a ring, and the original multi-angle image set of the bolt surface is obtained and then denoised and brightness equalized. Image enhancement module: Employs a frequency domain analysis method based on Fourier transform to identify the periodic texture features of the thread; Based on the periodic characteristics of the thread, an adaptive filter is designed to enhance the images in the preprocessed bolt surface image set; Multi-scale feature extraction module: Multi-scale morphological operations are performed on the enhanced bolt surface image set to extract morphological gradient features at different scales; Morphological feature maps of different types of defects such as cracks, pits and scratches are extracted by using structural elements for bolt defect types. Segmentation module: Extracts local statistical features from the multi-scale feature map set and adopts an adaptive threshold segmentation method based on local statistical features; combines thread direction information and local contrast to dynamically adjust the segmentation threshold, segment the potential defect region from the multi-scale feature map, and outputs a set of binary maps of defect candidate regions; Defect identification module: Extracts geometric and texture features from the binary image set of segmented defect candidate regions; Based on the extracted features, a random forest classifier is used to identify and classify defects, and finally outputs the detection results of bolt surface defects, including defect type, location and severity. To achieve the computer vision-based bolt surface defect detection method as described in any one of claims 1-8.

Images