A spring surface defect detection method based on machine vision

Abstract

The present application belongs to the technical field of spring product surface defect detection, and relates to a spring surface defect detection method based on machine vision, comprising the following steps: collecting surface image sequences of a spring to be detected at multiple preset rotation angles; for the images to be registered in the image sequence, extracting a spiral contour line of the spring through edge detection and curve fitting, calculating the curvature along the spiral contour line, and identifying local extreme points with absolute curvature values exceeding a preset curvature threshold as key points; for each key point, calculating local binary pattern features and gradient direction histogram features in the key point neighborhood, and generating a hybrid feature representation sub according to the weighted fusion of the two features according to the local curvature value at the key point, which can improve the reliability of spring surface defect detection and reduce the missed detection rate and false detection rate.

Description

Technical Field

[0001] This invention belongs to the field of metal spring surface defect detection technology under machine vision, and relates to a spring surface defect detection method based on machine vision, which is used for automated quality inspection of springs, accurately detects surface defects, reduces missed detections and false detections, and achieves efficient and stable visual inspection. Background Technology

[0002] Cracks, scratches, dents, and corrosion defects on the surface of springs can lead to stress concentration and fatigue fracture during service. Image processing technology is used to detect defects in springs, but the three-dimensional helical structure and repetitive surface texture of springs pose challenges to visual inspection: a single image cannot capture complete 360° surface information of the spring, requiring multi-view image acquisition and registration stitching technology to unfold the spring surface information into a complete two-dimensional image; the periodic texture of the spring coil itself has a high visual similarity to the background, which can easily be confused with real, small surface defects, interfering with defect identification and segmentation.

[0003] Existing technologies typically employ a strategy of capturing images from multiple angles and then stitching them together. General feature point detection operators such as SIFT, SURF, or ORB are used to find matching point pairs in the overlapping regions of adjacent images. Based on these matching point pairs, a global homography transformation matrix is ​​estimated, transforming the image to be registered to the coordinate system of the reference image, thus completing the stitching. In the defect detection stage, Fourier transform frequency domain analysis or spatial domain filtering methods are often used to suppress the periodic background texture of the spring. However, general feature point detection operators cannot extract a sufficient number of uniformly distributed stable key points when dealing with surfaces like springs with repetitive textures and sparse features. Springs may experience minute non-rigid deformations during rotation; a single global homography transformation model cannot represent the spatial mapping relationships of different local regions, easily leading to ghosting, misalignment, and false defects at the stitching seam. Regarding background texture suppression, filters fail to adapt to the texture frequency variations of springs of different specifications, resulting in incomplete background filtering or over-smoothing, affecting the detection rate of minute defects.

[0004] While various solutions exist in the prior art, they still have shortcomings. For example, Chinese patent CN116152260B discloses a spring defect detection method and system based on image processing. This method acquires spring images, preprocesses them to identify defect areas, establishes free-state and stretched-state defect shape maps, and compares deviations in a virtual coordinate system to determine whether the spring is qualified. However, this method only analyzes local defect areas and does not achieve complete imaging of the entire circumference of the spring surface, thus failing to detect defects across the entire helical sidewall. Chinese patent application CN119068239A discloses a feature-enhanced classification and recognition algorithm for spring end-face defect detection. This algorithm extracts sub-images of the spring end-face image, performs filtering, contour extraction, and feature enhancement, and then uses HOG features to train an SVM model to classify end-face defects. This method is only applicable to spring end-face detection, cannot cover defects across the entire circumference of the helical sidewall, and the single HOG feature has poor robustness to repetitive textures. Chinese patent CN119515780B discloses a method and system for industrial defect detection and localization based on frequency domain enhancement. This method utilizes frequency domain enhancement and dual-domain feature fusion, reconstructing images using an improved U-Net network and calculating anomaly scores to achieve defect localization. However, this method lacks a dedicated filter designed for the spring's helical structure and periodic texture, making it unable to accurately suppress interference from the spring's inherent texture and prone to missing minor defects. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and solve the technical problems of unstable general feature point detection, easy distortion and misalignment of image stitching, and missed or false detection of defects caused by periodic background texture interference in the prior art. This invention provides a machine vision-based method for detecting defects on the surface of springs.

[0006] To achieve the above-mentioned objectives, this invention provides a machine vision-based method for detecting surface defects in springs, comprising the following steps: Acquire a sequence of surface images of the spring to be tested at multiple preset rotation angles; For the image to be registered in the image sequence, the spiral contour of the spring is extracted by edge detection and curve fitting, the curvature is calculated along the spiral contour, and the local extreme points where the absolute value of curvature exceeds the preset curvature threshold are identified as key points. For each keypoint, local binary pattern features and gradient direction histogram features are calculated in the neighborhood of the keypoint, and the two features are weighted and fused according to the local curvature value at the keypoint to generate a hybrid feature representation. Initial matching of key points in adjacent images is performed based on hybrid feature representations, and a random sampling consensus algorithm is used to identify fine matching point pairs from the initial matching pairs; Based on the spatial density distribution of the finely matched point pairs, the overlapping region of the image is divided into a quadrilateral grid; For each quadrilateral mesh, a local homography transformation matrix is ​​calculated to construct a local transformation matrix field. Based on the local transformation matrix field, a local homography transformation is performed on each quadrilateral mesh. All transformed quadrilateral meshes are then merged to generate a full-circumference mosaic image of the spring surface. Based on the geometric parameters of the spiral profile, a composite band-stop filter is constructed in the frequency domain to filter out the periodic background texture in the full-circumference stitched image of the spring surface and obtain the residual image. By setting a threshold and performing connected component analysis on the residual image, surface defects of the spring can be identified and located.

[0007] This invention solves the problems of instability in the extraction of repetitive texture regions of springs by general feature operators in the prior art and the inability of global homography models to cope with the small non-rigid deformations when the spring rotates. By using curvature-guided key point extraction and block-based local homography transformation, ghosting and misalignment at the splicing points are eliminated, and a high-precision full-circumference unfolded image of the spring is obtained, which fundamentally improves the reliability of subsequent defect detection.

[0008] The present invention describes a method for extracting the helical contour of a spring from an image sequence to be registered by edge detection and curve fitting, calculating the curvature along the helical contour, and identifying local extrema points whose absolute curvature exceeds a preset curvature threshold as key points, including: The image is smoothed using Gaussian filtering, and the Canny operator is used to extract the spring edge point set; The least squares method is used to fit the wavy curve model to the edge point set to obtain a smooth and continuous spiral profile. The discrete curvature is calculated point by point along the spiral contour line, and the points that are local maxima or local minima of curvature and whose absolute curvature value exceeds a preset threshold are identified as key points.

[0009] This invention uses a wavy curve model to fit key points and utilizes discrete curvature extrema to locate key points. Compared with general feature operators that rely solely on pixel gradients, it can more accurately capture the geometric skeleton features of the spring helical structure, ensuring that key point extraction has extremely high repeatability and positioning accuracy in cases of metal surface reflection or sparse features.

[0010] The present invention describes calculating local binary pattern features and gradient direction histogram features in the neighborhood of each keypoint, and weighting and fusing the two features based on the local curvature values ​​at the keypoint to generate a hybrid feature representation, including: Define a neighborhood of a predetermined size around each keypoint, and calculate the local binary pattern histogram features and gradient orientation histogram features of the neighborhood respectively. Based on the local curvature values ​​at key points and the statistical distribution of curvature values ​​at all key points, a normalized weighting coefficient is determined. The gradient direction histogram features and local binary pattern features are multiplied by their respective weights and then concatenated to obtain a hybrid feature representation.

[0011] This invention dynamically adjusts the fusion weights of local binary patterns and gradient direction histogram features by adjusting the local curvature values, enabling the generated hybrid feature representation to simultaneously take into account the subtle texture information of the spring surface and the macroscopic geometric gradient features. This significantly enhances the discriminative power of the features in specific application scenarios of springs and significantly reduces the mismatch rate between adjacent images.

[0012] The present invention describes an initial matching of key points in adjacent images based on a hybrid feature representation, and an identification of fine-matching point pairs from the initial matching pairs using a random sampling consensus algorithm, comprising: Set the maximum number of iterations and the distance threshold for interior point detection; In each iteration, a subset of the minimum set of matching point pairs required to compute the geometric transformation model is randomly selected from the initial set of matching point pairs; Candidate global geometric transformation models are calculated based on subsets; The global geometric transformation model is applied to all initial matching point pairs, and the geometric error between the transformed points and the actual corresponding points is calculated. Mark the matching point pairs whose error is less than the distance threshold as inliers in the current iteration; After all iterations are completed, the geometric transformation model that produces the largest set of interior points is selected as the optimal model, and all interior points corresponding to the optimal model are selected as fine-matching point pairs.

[0013] This invention utilizes an improved random sampling consensus algorithm for precise matching point pair identification. Through multiple rounds of iteration, it selects a set of interior points that conform to the optimal geometric transformation model, effectively eliminating outliers and interference points in the initial matching, thus laying a solid data foundation for constructing a high-precision local transformation matrix field.

[0014] The present invention describes dividing the overlapping region of an image into a quadrilateral grid based on the spatial density distribution of precisely matched point pairs, comprising: Use the entire overlapping area of ​​the image as the initial root grid; A quadtree splitting strategy is adopted. For any grid cell, if the number of finely matched point pairs contained in the grid cell exceeds the preset upper limit of the number of points, the grid cell is evenly split into four sub-grids. This splitting process is executed recursively until the number of perfectly matched point pairs in all grid cells is no greater than the maximum number of points, or the size of the grid cell is smaller than the preset minimum size.

[0015] This invention employs a quadtree-based splitting strategy to divide overlapping regions of an image into grids, enabling the grid density to be adaptively adjusted according to the spatial distribution of matching points. The grid is automatically refined in areas of severe deformation, thereby enabling more precise modeling of complex local non-rigid projection deformations.

[0016] The present invention describes calculating a local homography transformation matrix for each quadrilateral mesh, including: Iterate through all generated quadrilateral meshes; For each quadrilateral grid, collect all perfectly matched point pairs that fall within the grid. If the number of perfectly matched point pairs within the quadrilateral mesh meets the minimum requirement for calculating the homography transformation matrix, then the local homography transformation matrix of the quadrilateral mesh is calculated using the direct linear transformation algorithm. If the number of perfectly matched point pairs within the quadrilateral mesh is insufficient, the nearest ancestor mesh whose transformation matrix has been successfully computed is recursively searched, and the homography transformation matrix of the ancestor mesh is inherited as the transformation matrix of the quadrilateral mesh.

[0017] The present invention describes a method for constructing a composite band-stop filter in the frequency domain based on the geometric parameters of a spiral profile to filter out periodic background textures in a full-circumference mosaic image of a spring surface, including: Based on the fitted helical profile geometric parameters, the equivalent texture tilt angle and spatial period of the spring in the full-circumference stitching diagram are calculated. On the two-dimensional Fourier spectrum of the full-circle mosaic, a series of points orthogonal to the texture direction and whose distance from the center of the spectrum is an integer multiple of the product of the reciprocal of the period and the corresponding dimension of the full-circle mosaic are determined as the center positions of the main peak of the texture and each harmonic. Around each peak center position, a Gaussian band-stop filter with preset bandwidth and directionality is constructed; Multiple Gaussian band-stop filters constructed at all main peak and harmonic locations are combined to obtain a matched band-stop filter, which is then applied to the spectrogram to suppress periodic texture components.

[0018] The present invention describes identifying and locating surface defects of a spring by setting a threshold and performing connected component analysis on a residual image, comprising: The Otsu algorithm is used to binarize the residual image; Apply the opening operation to the binary image to smooth the region contours and eliminate noise; Connected component labeling is performed on the processed binary image; Calculate the area of ​​each connected component, and identify connected components with an area greater than a preset area threshold as real defects, and output the defect location and size information.

[0019] The present invention describes the calculation of gradient orientation histogram features by calling the HOG function of the feature module of the scikit-image library.

[0020] The present invention uses the concatenate function of the NumPy library to multiply the gradient direction histogram features and the local binary pattern features by their respective weights and then concatenate them to obtain a hybrid feature representation.

[0021] Compared with existing technologies, the advantages of this invention are as follows: First, by using the curvature extrema points on the spring helical contour as key points, high repeatability and positioning accuracy of key points are achieved. Second, by fusing local binary pattern features and gradient direction histogram features, and weighting the two features according to the local curvature information at the key points, a discriminative feature representation is generated. Third, by constructing a local transformation matrix field to perform block-based local homography transformation on the overlapping areas of the image, the splicing distortion and artifacts caused by the spring surface are weakened, and a full-circumference view of the spring surface is obtained. Fourth, by constructing a composite band-stop filter based on the geometric parameters of the helical contour, the inherent strong periodic background texture of the spring can be filtered out, the contrast between defects and background is enhanced, the reliability of spring surface defect detection is improved overall, and the false negative and false positive rates are reduced. Attached Figure Description

[0022] Figure 1 This is a schematic flowchart of a machine vision-based spring surface defect detection method according to the present invention.

[0023] Figure 2 This is a schematic diagram of the overall architecture principle involved in the present invention. Detailed Implementation

[0024] The technical solution of the present invention will now be clearly and completely described in conjunction with the embodiments and accompanying drawings.

[0025] Example 1: The machine vision-based spring surface defect detection method provided in this embodiment like Figure 1 As shown, its process includes the following steps: S1, multi-angle rotation to acquire spring surface image sequence.

[0026] Collect a sequence of surface images of the spring to be tested at multiple preset rotation angles.

[0027] A CMOS camera is used in conjunction with a rotary stage controlled by a stepper motor to make the spring under test rotate at a constant speed. The camera is synchronously triggered to acquire images at specific angular intervals. To reduce reflections on the metal surface, a ring-shaped shadowless light source or a bowl-shaped diffuse reflection light source is used for supplementary lighting. The acquired continuous images are stored as a surface image sequence.

[0028] S2, extract the spiral contour line and identify key points according to the curvature extreme value.

[0029] For the image to be registered in the image sequence, the spiral contour of the spring is extracted by edge detection and curve fitting, the curvature is calculated along the spiral contour, and the local extreme points where the absolute value of curvature exceeds the preset curvature threshold are identified as key points.

[0030] The OpenCV library's `cvtColor` function is used to convert the color image to grayscale, and then the `GaussianBlur` function is used for Gaussian filtering to suppress noise. The `Canny` function is used for edge detection, setting appropriate low and high thresholds to extract the complete edges of the spring. The `findContours` function is applied to the Canny edge image to find all contours, filtering by contour area or perimeter to identify the outermost top and bottom contours representing the spring's main body; these two contours constitute the helical contour lines. For each point on the contour lines, the `gradient` function from the NumPy library is used to calculate the first and second numerical derivatives of the x and y coordinates with respect to the arc length, and then the curvature is calculated using the curvature calculation formula. The curvature value at that point is obtained. ,in, express The first derivative with respect to arc length, express The first derivative with respect to arc length, express The second derivative with respect to arc length, express The second derivative with respect to the arc length. Take the absolute value of all curvature values, and use the find_peaks function of the signal module in the scipy library to find local peak points in the set of points where the curvature value exceeds a preset threshold, and identify the peak points as key points.

[0031] In some implementations, for the image to be registered in an image sequence, the spiral contour of the spring is extracted through edge detection and curve fitting. The curvature is calculated along the spiral contour, and local extrema points whose absolute curvature exceeds a preset curvature threshold are identified as key points, including: The image is smoothed using Gaussian filtering, and the Canny operator is used to extract the spring edge point set; The least squares method is used to fit the wavy curve model to the edge point set to obtain a smooth and continuous spiral profile. The discrete curvature is calculated point by point along the spiral contour line, and the points that are local maxima or local minima of curvature and whose absolute curvature value exceeds a preset threshold are identified as key points.

[0032] The input image to be registered, for example, is preprocessed by applying a 5×5 Gaussian filter with a standard deviation of [missing information]. The threshold is set to 1.5 to suppress high-frequency noise introduced during image acquisition. The Canny edge detection operator is used, with a low threshold of 50 and a high threshold of 150, to extract the binary edge point set of the spring. For discrete edge points, a wavy curve model that can represent the projected shape of the spring is selected, such as... By fitting the edge point set using the least squares method, the optimal amplitude A, angular frequency ω, phase φ, slope B, and intercept C are obtained. This represents the precise position of the vertical coordinate (y) corresponding to the horizontal coordinate (x) on the spring spiral contour line in the image coordinate system. This yields a smooth and continuous spiral contour line function. Along this fitted spiral contour line, sampling is performed in steps of 1 pixel. For each sampling point... Using two points before and after and The coordinates of the point are used to approximate the first and second derivatives of the point using the finite difference method, and then the discrete curvature of the point is calculated. Set a curvature threshold. This threshold can be set as the mean of the set of all calculated absolute values ​​of curvature plus 2.5 times the standard deviation, i.e. ,in, This represents the average of the absolute values ​​of curvature. This represents the standard deviation of the absolute value of curvature. Iterate through the curvature values ​​of all sampled points; if a point... curvature It is a local maximum or minimum value within a neighborhood, such as five points before and after it, and the absolute value of curvature. Then this point Identify and mark as key points.

[0033] S3: Extract features from the neighborhood and weight them together to generate a hybrid feature representation.

[0034] For each keypoint, local binary pattern features and gradient direction histogram features are calculated in the neighborhood of the keypoint, and the two features are weighted and fused according to the local curvature value at the keypoint to generate a hybrid feature representation.

[0035] Centered on each keypoint, a 32×32 pixel neighborhood block is extracted from the image. Within this neighborhood block, the `local_binary_pattern` function of the `feature` module in the `scikit-image` library is used to calculate local binary pattern features. The number of sampling points is set to 8, and the radius to 1. The generated local binary pattern codemap is then plotted as a 256-dimensional histogram; this is the local binary pattern feature. Simultaneously, within this neighborhood block, the `HOG` function of the `feature` module in the `scikit-image` library is used to calculate gradient orientation histogram features. The cell size is set to 8×8 pixels, the block size to 2×2 cells, and the number of orientation channels to 9, generating a 324-dimensional feature vector. The curvature values ​​at the keypoints are normalized to obtain the weight factor `w`. The normalization method is to apply the sigmoid function to the curvature values. The normalized local binary pattern feature vector is multiplied by a weight factor 1-w, and the normalized gradient direction histogram feature vector is multiplied by a weight factor w. The two weighted feature vectors are then concatenated into a high-dimensional hybrid feature representation using the concatenate function of the NumPy library.

[0036] In some implementations, for each keypoint, local binary pattern features and gradient direction histogram features are calculated in the neighborhood of the keypoint, and the two features are weighted and fused based on the local curvature values ​​at the keypoint to generate a hybrid feature representation, including: Define a neighborhood of a predetermined size around each keypoint, and calculate the local binary pattern histogram features and gradient orientation histogram features of the neighborhood respectively. Based on the local curvature values ​​at key points and the statistical distribution of curvature values ​​at all key points, a normalized weighting coefficient is determined. The gradient direction histogram features and local binary pattern features are multiplied by their respective weights and then concatenated to obtain a hybrid feature representation.

[0037] For each identified keypoint, a 32×32 pixel square neighborhood is defined around the point as the feature computation region. Local Binary Pattern (LBP) features are calculated using a uniform local binary pattern (LBP) with rotation invariance and equivalent patterns, i.e., LBP U2, with 8 sampling points P and a sampling radius R of 2. The 32×32 neighborhood is divided into 4×4 sub-regions, each 8×8 pixels. A 59-dimensional LBP U2 histogram is calculated for each sub-region, and the 16 histograms are concatenated into a 944-dimensional LBP feature vector. Simultaneously, Histogram of Gradient Directions (HOG) features are calculated. The 32×32 neighborhood is treated as a block, consisting of four 2×2 cells, each 16×16 pixels in size. The gradient direction is divided into 9 bins. The calculated HOG feature representation has a sub-dimensionality of 36. Weighted fusion is then performed, and normalized weight coefficients are calculated. Calculate the absolute value of curvature at all key points and find its minimum value. and maximum value For the current critical point, the absolute value of curvature is The weights of the gradient direction histogram features are then... Calculated as Weights of local binary pattern features Then it is Before fusion, the local binary pattern and gradient direction histogram feature vectors are normalized using the L2 norm. The mixed feature representation D is generated by weighted concatenation. , This represents the normalized HOG features. This represents the normalized LBP features, with a total dimension of 980.

[0038] S4, initial matching with mixed features, and random sampling for consistency screening of finely matched point pairs.

[0039] Initial matching of key points in adjacent images is performed based on hybrid feature representation, and a random sampling consensus algorithm is used to identify fine matching point pairs from the initial matching pairs.

[0040] Using the BFMatcher brute-force matcher from the OpenCV library, with the distance metric set to NORM_L2 (Euclidean distance), we match the mixed feature representation subsets of the source and target images. For each representation in the source image, we find the two closest representations in the target image and apply the Lowe ratio test: a match is considered valid only if the ratio of the nearest neighbor distance to the second nearest neighbor distance is less than a threshold, such as 0.75. We then call the `findHomography` function from the OpenCV library, taking the coordinates of all initial matching pairs as input and setting the method parameter to RANSAC. Through iterative calculation of the homography transformation matrix, we identify matching pairs that conform to the same transformation model as inliers (i.e., fine-matched point pairs) and discard outliers.

[0041] In some implementations, initial matching of keypoints in adjacent images is performed based on hybrid feature representations, and a random sample consensus algorithm is used to identify fine-matching point pairs from the initial matching pairs, including: Set the maximum number of iterations and the distance threshold for interior point detection; In each iteration, a subset of the minimum set of matching point pairs required to compute the geometric transformation model is randomly selected from the initial set of matching point pairs; Candidate global geometric transformation models are calculated based on subsets; The global geometric transformation model is applied to all initial matching point pairs, and the geometric error between the transformed points and the actual corresponding points is calculated. Mark the matching point pairs whose error is less than the distance threshold as inliers in the current iteration; After all iterations are completed, the geometric transformation model that produces the largest set of interior points is selected as the optimal model, and all interior points corresponding to the optimal model are selected as fine-matching point pairs.

[0042] Set the core parameter of the algorithm: maximum number of iterations. The iteration count is set to 1000, and the geometric distance threshold t for determining the inlier is set to 3.0 pixels. In each iteration, k ranges from 1 to... In, the initial set of matching point pairs obtained from all sub-matchings using mixed feature representations. In the process, a minimum subset is randomly selected without replacement. Since the geometric transformation model to be solved is a global homography matrix, at least four pairs of non-collinear matching points are required. Therefore, this minimum subset contains four pairs of matching points. Using these four pairs, a candidate 3×3 homography matrix is ​​calculated using the Direct Linear Transformation (DLT) algorithm. . This matrix Applied to Each matching point pair in Calculate the symmetric transmission error ,in This represents the Euclidean distance. If the error... If the distance is less than the square of the distance threshold t, then the matching point pair is... Add to the set of interior points in the current iteration After 1000 iterations, compare the size of the interior point set generated in each iteration. Find the iteration with the most interior points, and the corresponding homography matrix. Considered the optimal global transformation model, the set of all interior points generated in this iteration Then it is determined to be a set of precisely matched point pairs.

[0043] S5, according to the density of fine-matching points, divide the overlapping area into quadrilateral grids.

[0044] Based on the spatial density distribution of the finely matched point pairs, the overlapping areas of the image are divided into quadrilateral grids.

[0045] Determine the bounding box of the overlapping region between two images. Using this bounding box as the root node, construct a quadtree data structure. Recursively split each node of the quadtree, with the split condition being that the number of perfectly matched point pairs within the quadrilateral region represented by the node exceeds a preset threshold, such as 10. If the split condition is met, the quadrilateral is uniformly split into four sub-quadrilaterals, and the point pairs within each sub-quadrilateral are assigned to the corresponding child nodes. Continue recursively splitting until the number of point pairs within all leaf node regions is less than the threshold or the maximum recursion depth is reached. The set of quadrilaterals formed by all leaf nodes constitutes the mesh partition.

[0046] In some implementations, the overlapping regions of the image are divided into quadrilateral grids based on the spatial density distribution of the finely matched point pairs, including: Use the entire overlapping area of ​​the image as the initial root grid; A quadtree splitting strategy is adopted. For any grid cell, if the number of finely matched point pairs contained in the grid cell exceeds the preset upper limit of the number of points, the grid cell is evenly split into four sub-grids. This splitting process is executed recursively until the number of perfectly matched point pairs in all grid cells is no greater than the maximum number of points, or the size of the grid cell is smaller than the preset minimum size.

[0047] Based on the set of finely matched point pairs obtained in the previous step, the convex hull of the coordinates of the set on the image to be registered is determined as the initial image overlap region, and the minimum bounding rectangle of this convex hull is used as the root grid of the quadtree. Two recursion termination conditions are set: an upper limit on the number of finely matched point pairs in the grid. Set to 30, and the minimum size of the grid cells. Set to 32x32 pixels. Starting from the root grid, check the number of perfectly matched point pairs contained within it. If this number is greater than... And the current grid's width and height are both greater than The grid is then uniformly divided into four equal-sized sub-grids along the horizontal and vertical midlines: top left, top right, bottom left, and bottom right. The same checking and splitting process is recursively performed on the four newly generated sub-grids. This process continues until the number of perfectly matched point pairs within a grid cell is less than or equal to 30, or the size of the grid cell has reached or decreased to 32×32 pixels. At this point, the grid cell stops splitting and becomes a leaf node grid. The entire overlapping area of ​​the image is divided into a set of quadrilateral leaf node grids of different sizes.

[0048] S6 calculates the local transformation matrix grid by grid to generate a full-circumference spliced ​​image of the spring.

[0049] For each quadrilateral mesh, a local homography transformation matrix is ​​calculated to construct a local transformation matrix field. Based on the local transformation matrix field, a local homography transformation is performed on each quadrilateral mesh. All transformed quadrilateral meshes are then merged to generate a full-circumference mosaic image of the spring surface.

[0050] The algorithm iterates through each quadrilateral grid, extracting all precisely matched point pairs falling within that grid. If the number of point pairs within a grid is greater than or equal to 4, the `findHomography` function from the OpenCV library is called again to calculate a local homography transformation matrix specific to that grid, using only the local point pairs. The local transformation matrices of all grids together constitute the local transformation matrix field. A sufficiently large blank canvas is created as the stitching image. For each pixel in the image to be registered, the grid to which the pixel belongs is determined, and the new coordinates of the pixel on the canvas are calculated using the local homography transformation matrix corresponding to that grid, generating a pixel mapping relationship from the target image to the source image. Using the `remap` function from OpenCV, the entire image to be registered is resampled and distorted according to the pixel mapping relationship to obtain the transformed image. In the overlapping areas of the images, a multi-band fusion algorithm is used. By constructing Laplacian pyramids of the images to be stitched and the already stitched images, and using Gaussian pyramids as weights for weighted averaging at each layer of the pyramid, the image is reconstructed from the fused pyramid, achieving seamless stitching of all transformed grids and generating a full-circumference stitched image of a spring surface.

[0051] In some implementations, a local homography transformation matrix is ​​computed for each quadrilateral mesh, including: Iterate through all generated quadrilateral meshes; For each quadrilateral grid, collect all perfectly matched point pairs that fall within the grid. If the number of perfectly matched point pairs within the quadrilateral mesh meets the minimum requirement for calculating the homography transformation matrix, then the local homography transformation matrix of the quadrilateral mesh is calculated using the direct linear transformation algorithm. If the number of perfectly matched point pairs within the quadrilateral mesh is insufficient, the nearest ancestor mesh whose transformation matrix has been successfully computed is recursively searched, and the homography transformation matrix of the ancestor mesh is inherited as the transformation matrix of the quadrilateral mesh.

[0052] For each grid cell, count and collect all perfectly matching point pairs within that grid cell's spatial range. Calculate the minimum number of point pairs required to compute the local homography transformation matrix. The value is 8, which is greater than the theoretical minimum of 4. If the number of point pairs in the current grid is greater than or equal to 8, then a 3×3 local homography matrix is ​​obtained using the point pairs through the Direct Linear Transform (DLT) algorithm. The matrix is ​​then associated with the grid. If the number of point pairs in the grid is less than 8, the transformation matrix cannot be stably calculated. In this case, an inheritance mechanism is initiated: the program backtracks upwards along the quadtree structure, searching for the parent grid. If the parent grid has already calculated its own transformation matrix, it inherits that matrix; if the parent grid does not have one either, it continues searching upwards for the grandparent grid, and so on, until an ancestor grid that has successfully calculated its transformation matrix is ​​found. In the worst case, it backtracks to the root grid, which contains all the perfectly matched points and can certainly calculate a global transformation matrix. The transformation matrix of the first ancestor grid found is assigned to the current grid. Through this process, it is ensured that each leaf node grid has a local homography transformation matrix, which together form a local transformation matrix field covering the entire overlapping region.

[0053] In yet another embodiment, when the number of finely matched point pairs within a certain leaf grid is less than Using the geometric center of the leaf mesh as a reference, its feature search receptive field is softly expanded by a preset ratio (e.g., expanding the mesh's side length outward by 50%) to form an overlapping buffer that includes parts of adjacent meshes. All finely matched point pairs falling within this overlapping buffer are collected. If the total number of point pairs collected after expansion reaches 8 or more, these points are used to calculate the local homography matrix of the leaf mesh.

[0054] S7: Construct a frequency domain filter based on the contour parameters to filter out texture and obtain the residual image.

[0055] Based on the geometric parameters of the spiral profile, a composite band-stop filter is constructed in the frequency domain to filter out the periodic background texture in the full-circumference stitched image of the spring surface, thereby obtaining the residual image.

[0056] A two-dimensional discrete Fourier transform (DFT) is performed on the full-circumference mosaic image of the spring surface. The spectrum is calculated using the `fft2` function from the `fft` module of the NumPy library, and the `fftshift` function is used to shift the zero-frequency component to the center of the spectrum. The logarithmic amplitude spectrum is calculated and visualized. In the amplitude spectrum, the periodic texture of the spring is represented by multiple symmetrical bright spots, i.e., energy peaks, excluding the central DC component. The position coordinates of these energy peaks are identified using peak detection algorithms in image processing, such as threshold-based and local maximum search. The coordinates represent the frequency and directional geometric parameters of the texture. For each pair of detected symmetrical peak points, a Gaussian band-stop filter is constructed, mathematically creating a Gaussian-shaped indentation at the peak point location. The multiple Gaussian band-stop filters constructed for all peak points are multiplied together to obtain a composite band-stop filter. This composite filter is multiplied element-wise with the spectrum of the mosaic image to suppress the texture frequency components. An inverse Fourier transform is performed using the `ifftshift` and `ifft2` functions to obtain an image with the background texture filtered out. The original stitched image and the filtered image are compared by performing a difference operation. The absolute value of the difference between the two is calculated using OpenCV's absdiff function to generate a residual image.

[0057] In some implementations, based on the geometric parameters of the helical profile, a composite band-stop filter is constructed in the frequency domain to filter out the periodic background texture in the full-circumference mosaic image of the spring surface, including: Based on the fitted helical profile geometric parameters, the equivalent texture tilt angle and spatial period of the spring in the full-circumference stitching diagram are calculated. On the two-dimensional Fourier spectrum of the full-circle mosaic, a series of points orthogonal to the texture direction and whose distance from the center of the spectrum is an integer multiple of the product of the reciprocal of the period and the corresponding dimension of the full-circle mosaic are determined as the center positions of the main peak of the texture and each harmonic. Around each peak center position, a Gaussian band-stop filter with preset bandwidth and directionality is constructed; Multiple Gaussian band-stop filters constructed at all main peak and harmonic locations are combined to obtain a matched band-stop filter, which is then applied to the spectrogram to suppress periodic texture components.

[0058] Using the previously fitted spiral profile model The equivalent texture parameters are re-estimated on the full-circumference mosaic map of the spring surface. Tilt angle. Determined by the slope B, Spatial frequency The number of cycles per pixel is determined by the angular frequency ω. A two-dimensional Fast Fourier Transform (FFT) is performed on the full-circle mosaic of size M×N to obtain the spectrogram. In the spectrogram, the periodic texture appears as a series of concentrated bright spot peaks. To avoid localization errors caused by scaling in different dimensions, the spatial frequency... Along the direction angle orthogonal to the texture Decomposition yields frequency components in the horizontal and vertical directions. and The coordinate offsets were calculated independently based on the width and height of the image, and the two-dimensional coordinate offset of the main peak relative to the spectral center (M / 2, N / 2) was determined as follows: the horizontal coordinate offset. and vertical coordinate offset The precise coordinates of the main peak are The peak values ​​of each harmonic order appear at a position offset by k times the center, which is the precise coordinate of the k-th spring texture peak on the spectrum. Where k = 2, 3, ... . Usually only the first three harmonics are considered. For each peak position... and its symmetrical point about the center of the spectrum Construct a Gaussian band-stop filter: ; Let (u, v) be the band-stop filter value at the k-th harmonic position, and (u, v) represent the coordinates of any point on the spectrum. This Gaussian band-stop filter is a rotated... An elliptic Gaussian function of angles, where the standard deviation of the major axis is... Set as 0.2 times, used to control the suppression bandwidth, This represents the radial distance from the main peak of the texture to the center of the spectrum on the spectrogram; the standard deviation of the minor axis. Set as 0.1 times that of the composite band-stop filter achieves directionality. By multiplying all the individual filters together, we obtain: , The multiplication symbol is used. This composite filter is multiplied element-wise with the spectrogram to suppress the frequency components corresponding to the texture. The residual image, which is almost free of periodic background texture, is obtained by inverse Fourier transform (IFFT).

[0059] S8, residual image thresholding and connected component analysis, to locate surface defects on springs.

[0060] By setting a threshold and performing connected component analysis on the residual image, surface defects of the spring can be identified and located.

[0061] Since the pixel values ​​of the defect region in the residual image are much higher than those of the background region, the `threshold` function from the OpenCV library is called, and the `THRESH_OTSU` method is used to calculate the optimal global threshold, thus binarizing the residual image. The binarized image is then analyzed using the `connectedComponentsWithStats` function from the OpenCV library. This function identifies all white pixel blocks, assigns a unique label to each connected component, and calculates the statistical information for each connected component, including the coordinates, width, height, and area of ​​its bounding rectangle. The statistical information of all connected components is iterated, and an area filtering threshold is set to remove false defects that are too small (potentially caused by noise) and regions that are too large. The remaining connected components are the identified spring surface defects, such as... Figure 2 As shown, the circumscribed rectangle information provides the location of the defect on the full-circle mosaic image.

[0062] In some implementations, surface defects of the spring are identified and located on the residual image by setting a threshold and performing connected component analysis, including: The Otsu algorithm is used to binarize the residual image; Apply the opening operation to the binary image to smooth the region contours and eliminate noise; Connected component labeling is performed on the processed binary image; Calculate the area of ​​each connected component, and identify connected components with an area greater than a preset area threshold as real defects, and output the defect location and size information.

[0063] After obtaining the residual image, Otsu's method is applied to calculate a globally optimal threshold. This algorithm maximizes the inter-class variance and effectively distinguishes potential defects, which typically have high grayscale values ​​and background. Using this threshold to binarize the residual image, grayscale values ​​greater than [a certain threshold] are [values]. Pixels are set to 255 (white) to represent candidate defects, and the rest are set to 0 (black) to represent the background. To eliminate noise and glitch that may occur during binarization, a morphological opening operation is performed on the binary image. This opening operation consists of an erosion operation followed by a dilation operation, using a 3×3 pixel circular or square kernel as the structuring element. Connectivity analysis is then performed on the binary image after the opening operation, using the 8-neighborhood connectivity rule to group all interconnected white pixels and assign a unique label to each connected region. All labeled connected regions are traversed, and the total number of pixels contained in each region, i.e., the area, is calculated. A minimum area threshold is then set. For example, 25 pixels, this value is determined based on the minimum size of the target to be detected and the image resolution. Any area smaller than... All connected components are treated as noise or spurious defects and filtered out. For all areas greater than or equal to... If a connected component is identified as a real defect, detailed information is output, including the coordinates (x, y, width, height) of the minimum bounding rectangle of the defect region as positional information, and the actual pixel area of ​​the connected component as size information.

[0064] To verify the effectiveness of the keypoint detection and feature representation sub-scheme, a control experiment was set up. The experimental conditions were to test on a dataset containing 100 pairs of spring images with different illuminations and poses. The evaluation metrics were the average number of correctly matched point pairs, the matching accuracy, and the root mean square error (RMSE) of the registration.

[0065] The specific experimental data are as follows: Scheme A, i.e., the method in this paper, had an average of 215 correctly matched point pairs, an accuracy of 92.3 percentage points, and an RMSE of 0.85 pixels; Control scheme B, using the standard SIFT algorithm, had an average of 128 correctly matched point pairs, an accuracy of 81.5 percentage points, and an RMSE of 2.13 pixels; Control scheme C, using the keypoint detection method in this paper but only using the gradient orientation histogram representation, had an average of 186 correctly matched point pairs, an accuracy of 88.7 percentage points, and an RMSE of 1.21 pixels.

[0066] Experimental results show that this embodiment outperforms the control group in all indicators. It can more stably locate geometric features at the curvature extrema of the spring helical structure, and the weighted local binary mode and gradient direction histogram hybrid representation integrates texture and shape information, which has stronger discriminative power than a single representation or a general representation.

[0067] To verify the superiority of the image registration method, ablation experiments were conducted. The experimental conditions involved using 100 successfully matched image pairs from the previous experiments, generating full-circle mosaic images through registration. The evaluation metrics were the average structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) between the mosaic image and the real reference image.

[0068] The specific experimental data are as follows: Scheme A, which is the mesh generation and local homography transformation method used in this paper, yielded an average SSIM of 0.96 and an average PSNR of 35.2 dB; Scheme B of the ablation group only used the global homography transformation obtained from the initial RANSAC for registration, yielding an average SSIM of 0.81 and an average PSNR of 27.8 dB.

[0069] Experimental results show that the local registration method proposed in this embodiment far surpasses the global transformation method in terms of stitching quality. The improvement lies in the fact that the spring surface is non-planar, and a single global homography transformation cannot model the 3D projection deformation, resulting in ghosting and misalignment in the stitching area; while the mesh and local homography model can decompose the non-rigid deformation into multiple local approximate linear transformations, thereby achieving high-precision seamless stitching of the unfolded surface image.

[0070] To verify the necessity of background texture suppression in the defect detection module, an ablation experiment was conducted. The experiment involved testing on 50 spliced ​​images of springs containing known real defects, with varying background texture intensities. The evaluation metrics were the accuracy and false alarm rate of defect detection.

[0071] The specific experimental data are as follows: Scheme A, i.e., the embodiment, uses a composite band-stop filter for preprocessing before Otsu detection, with a defect detection accuracy of 98.5 percentage points and a false alarm rate of 1.2 percentage points; Scheme B of the ablation group performs Otsu detection on the original spliced ​​image, with a defect detection accuracy of 85.3 percentage points and a false alarm rate as high as 15.8 percentage points.

[0072] Experimental results demonstrate that adding a filtering preprocessing step can improve defect detection performance. The improvement lies in the fact that the inherent periodic machining texture on the spring surface has a high gradient response. Without processing, it is easily misclassified as a defect during Otsu binarization, leading to a surge in false alarm rate. The constructed band-stop filter can filter out periodic interference signals in the frequency domain, making the background area in the residual image flat, thereby enhancing the signal-to-noise ratio of the true defect.

Claims

1. A method for detecting surface defects in springs based on machine vision, characterized in that, Includes the following steps: Acquire a sequence of surface images of the spring to be tested at multiple preset rotation angles; For the image to be registered in the image sequence, the spiral contour of the spring is extracted by edge detection and curve fitting, the curvature is calculated along the spiral contour, and the local extreme points where the absolute value of curvature exceeds the preset curvature threshold are identified as key points. For each keypoint, local binary pattern features and gradient direction histogram features are calculated in the neighborhood of the keypoint, and the two features are weighted and fused according to the local curvature value at the keypoint to generate a hybrid feature representation. Initial matching of key points in adjacent images is performed based on hybrid feature representations, and a random sampling consensus algorithm is used to identify fine matching point pairs from the initial matching pairs; Based on the spatial density distribution of the finely matched point pairs, the overlapping region of the image is divided into a quadrilateral grid; For each quadrilateral mesh, a local homography transformation matrix is ​​calculated to construct a local transformation matrix field. Based on the local transformation matrix field, a local homography transformation is performed on each quadrilateral mesh. All transformed quadrilateral meshes are then merged to generate a full-circumference mosaic image of the spring surface. Based on the geometric parameters of the spiral profile, a composite band-stop filter is constructed in the frequency domain to filter out the periodic background texture in the full-circumference stitched image of the spring surface and obtain the residual image. By setting a threshold and performing connected component analysis on the residual image, surface defects of the spring can be identified and located.

2. The machine vision-based spring surface defect detection method according to claim 1, characterized in that, For the image to be registered in the image sequence, the spiral contour of the spring is extracted through edge detection and curve fitting. The curvature is calculated along the spiral contour, and local extrema points whose absolute curvature exceeds a preset curvature threshold are identified as key points, including: The image is smoothed using Gaussian filtering, and the Canny operator is used to extract the spring edge point set; The least squares method is used to fit the wavy curve model to the edge point set to obtain a smooth and continuous spiral profile. The discrete curvature is calculated point by point along the spiral contour line, and the points that are local maxima or local minima of curvature and whose absolute curvature value exceeds a preset threshold are identified as key points.

3. The machine vision-based spring surface defect detection method according to claim 1, characterized in that, For each keypoint, local binary pattern features and gradient direction histogram features are calculated in the neighborhood of the keypoint. These two features are then weighted and fused based on the local curvature values ​​at the keypoint to generate a hybrid feature representation, including: Define a neighborhood of a predetermined size around each keypoint, and calculate the local binary pattern histogram features and gradient orientation histogram features of the neighborhood respectively. Based on the local curvature values ​​at key points and the statistical distribution of curvature values ​​at all key points, a normalized weighting coefficient is determined. The gradient direction histogram features and local binary pattern features are multiplied by their respective weights and then concatenated to obtain a hybrid feature representation.

4. The machine vision-based spring surface defect detection method according to claim 1, characterized in that, Initial matching of keypoints in adjacent images is performed based on a hybrid feature representation, and a random sampling consensus algorithm is used to identify fine-matching point pairs from the initial matching pairs, including: Set the maximum number of iterations and the distance threshold for interior point detection; In each iteration, a subset of the minimum set of matching point pairs required to compute the geometric transformation model is randomly selected from the initial set of matching point pairs; Candidate global geometric transformation models are calculated based on subsets; The global geometric transformation model is applied to all initial matching point pairs, and the geometric error between the transformed points and the actual corresponding points is calculated. Mark the matching point pairs whose error is less than the distance threshold as inliers in the current iteration; After all iterations are completed, the geometric transformation model that produces the largest set of interior points is selected as the optimal model, and all interior points corresponding to the optimal model are selected as fine-matching point pairs.

5. The machine vision-based spring surface defect detection method according to claim 1, characterized in that, Based on the spatial density distribution of the precisely matched point pairs, the overlapping region of the image is divided into a quadrilateral grid, including: Use the entire overlapping area of ​​the image as the initial root grid; A quadtree splitting strategy is adopted. For any grid cell, if the number of finely matched point pairs contained in the grid cell exceeds the preset upper limit of the number of points, the grid cell is evenly split into four sub-grids. This splitting process is executed recursively until the number of perfectly matched point pairs in all grid cells is no greater than the maximum number of points, or the size of the grid cell is smaller than the preset minimum size.

6. The machine vision-based spring surface defect detection method according to claim 1, characterized in that, Calculate a local homography transformation matrix for each quadrilateral mesh, including: Iterate through all generated quadrilateral meshes; For each quadrilateral grid, collect all perfectly matched point pairs that fall within the grid. If the number of perfectly matched point pairs within the quadrilateral mesh meets the minimum requirement for calculating the homography transformation matrix, then the local homography transformation matrix of the quadrilateral mesh is calculated using the direct linear transformation algorithm. If the number of perfectly matched point pairs within the quadrilateral mesh is insufficient, the nearest ancestor mesh whose transformation matrix has been successfully computed is recursively searched, and the homography transformation matrix of the ancestor mesh is inherited as the transformation matrix of the quadrilateral mesh.

7. The machine vision-based spring surface defect detection method according to claim 1 or 6, characterized in that, Based on the geometric parameters of the spiral profile, a composite band-stop filter is constructed in the frequency domain to filter out the periodic background texture in the full-circumference mosaic image of the spring surface, including: Based on the fitted helical profile geometric parameters, the equivalent texture tilt angle and spatial period of the spring in the full-circumference stitching diagram are calculated. On the two-dimensional Fourier spectrum of the full-circle mosaic, a series of points orthogonal to the texture direction and whose distance from the center of the spectrum is an integer multiple of the product of the reciprocal of the period and the corresponding dimension of the full-circle mosaic are determined as the center positions of the main peak of the texture and each harmonic. Around each peak center position, a Gaussian band-stop filter with preset bandwidth and directionality is constructed; Multiple Gaussian band-stop filters constructed at all main peak and harmonic locations are combined to obtain a matched band-stop filter, which is then applied to the spectrogram to suppress periodic texture components.

8. The machine vision-based spring surface defect detection method according to claim 1, characterized in that, By setting a threshold and performing connected component analysis on the residual image, surface defects of the spring are identified and located, including: The Otsu algorithm is used to binarize the residual image; Apply the opening operation to the binary image to smooth the region contours and eliminate noise; Connected component labeling is performed on the processed binary image; Calculate the area of ​​each connected component, and identify connected components with an area greater than a preset area threshold as real defects, and output the defect location and size information.

9. The machine vision-based spring surface defect detection method according to claim 1, characterized in that, The gradient orientation histogram features are calculated by calling the HOG function of the feature module in the scikit-image library.

10. The machine vision-based spring surface defect detection method according to claim 1, characterized in that, The concatenate function from the NumPy library is used to multiply the gradient direction histogram features and the local binary pattern features by their respective weights and then concatenate them to obtain a hybrid feature representation.

Images