Machine scanning-based self-tapping and self-drilling screw surface defect detection method and system

Abstract

The application is a self-tapping and self-drilling screw surface defect detection method and system based on machine scanning, and relates to the technical field of industrial machine vision, comprising: a multi-angle polarization image array acquisition module; a Stokes polarization parameter solving module; a high-dynamic anti-reflection image reconstruction module; a structural tensor matrix feature extraction module; and a multi-dimensional feature fusion defect recognition module.In the application, based on the polarization optical mechanism and the dichroism reflection model, the specular reflection and the diffuse reflection components of the metal surface are separated from the physical layer, the problem that the overexposure false defects caused by strong reflection eliminate the real defects from being covered is solved, compared with the conventional image enhancement algorithm, the application has rigorous physical mechanism support, no algorithm artifact, and the detection result is real and reliable.

Description

Technical Field

[0001] This invention relates to the field of industrial machine vision technology, and in particular to a method and system for detecting surface defects in self-tapping and self-drilling screws based on machine scanning. Background Technology

[0002] Self-tapping and self-drilling screws are among the most widely used fasteners in construction, automotive manufacturing, 3C electronics, aerospace, and other fields. Their surface quality directly determines the fastening performance, fatigue life, and safety of the connection structure. Screws are prone to surface defects during production and transportation, mainly including physical morphological defects such as mechanical scratches, micro-cracks, missing threads, and burrs, as well as chemical material defects such as rust, oxide layers, and oil stains.

[0003] In current industrial settings, the main problems in detecting surface defects in screws are as follows:

[0004] Self-tapping and self-drilling screws are mostly made of galvanized or Dacromet-treated carbon steel / stainless steel. The metal surface has a strong mirror reflection, which can easily lead to overexposure of highlights during conventional visual inspection, causing defects to be covered up and generating a large number of false defects. The rate of missed detection and false detection remains high.

[0005] Screw threads are continuous helical three-dimensional curved surfaces. Conventional edge detection algorithms (such as Sobel and Canny operators) cannot distinguish between the one-dimensional continuous edges of normal threads and two-dimensional topological abrupt changes caused by defects. They are very prone to misjudging normal thread profiles as defects, resulting in extremely poor robustness.

[0006] In the existing technology, in order to solve the above problems, some solutions use machine learning / deep learning models for defect identification. Such methods require a large number of labeled samples for training, have poor generalization, and the decision-making process of black box models is uninterpretable, which cannot meet the traceability requirements of industrial inspection.

[0007] Some solutions employ polarization optical inspection, but most use a mechanically rotating polarizer architecture, which suffers from problems such as mechanical vibration, image misalignment, and slow inspection speed. These solutions are not suitable for industrial online high-speed inspection scenarios and do not deeply integrate polarization material characteristics with topological features, making it impossible to simultaneously achieve specular removal and thread background interference suppression.

[0008] Therefore, a method and system for detecting surface defects in self-tapping and self-drilling screws based on machine scanning are proposed to address the aforementioned problems. Summary of the Invention

[0009] The purpose of this invention is to provide a method and system for detecting surface defects in self-tapping and self-drilling screws based on machine scanning in order to solve the above-mentioned problems.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] A machine scanning-based surface defect detection system for self-tapping and self-drilling screws includes:

[0012] The multi-angle polarization image array acquisition module is configured to simultaneously capture multi-angle polarization images of the surface of a self-tapping and self-drilling screw using a polarization illumination integrating sphere and a focal plane segmented polarization camera.

[0013] The Stokes polarization parameter calculation module is configured to receive multi-angle polarization images and perform demosaic interpolation to calculate the total light intensity matrix, linear polarization degree matrix, and linear polarization angle matrix that characterize the surface physical material and three-dimensional topological properties.

[0014] The high dynamic range de-reflection image reconstruction module is configured to extract the pure diffuse reflection component based on the total light intensity matrix and the linear polarization degree matrix, and perform anisotropic joint bilateral filtering guided by the linear polarization angle matrix to reconstruct the de-reflection image.

[0015] The structural tensor matrix feature extraction module is configured to: fuse the dereflected image with the linear polarization degree matrix to construct a two-dimensional vector field, and calculate the feature energy map and the consistency divergence map by calculating the local structural tensor matrix;

[0016] The multi-dimensional feature fusion defect identification module is configured to: generate a defect mask based on the feature energy map and the consistency scatter map, extract the geometric features of the connected domain, and complete the classification and determination of the defect by combining the average linear polarization degree within the connected domain.

[0017] Preferably, in the Stokes polarization parameter calculation module:

[0018] An adaptive edge-guided interpolation algorithm based on image spatial gradient is used for demosaic interpolation. The horizontal and vertical image gradients in the local neighborhood are calculated respectively. The interpolation weights of different polarization channels are determined according to the relative magnitudes of the horizontal and vertical image gradients in the local neighborhood, and a light intensity image matrix with multiple polarization angles with full resolution is reconstructed.

[0019] Preferably, the formula for calculating the total light intensity matrix is ​​as follows:

[0020] ;

[0021] in, Represents pixel coordinates The total light intensity at a given location corresponds to the pixel value. , , as well as These represent the full-resolution matrix of the corresponding polarization transmission axis angles in pixel coordinates. The light intensity value at that location.

[0022] Preferably, the calculation of the linear polarization degree matrix and the linear polarization angle matrix specifically involves:

[0023] ;

[0024] ;

[0025] in, This represents the pixel value corresponding to the degree of linear polarization. This represents the pixel value corresponding to the linear polarization angle. This represents the intensity difference of the combined linearly polarized light in the first orthogonal direction. This represents the intensity difference of the combined linearly polarized light in the second orthogonal direction. This represents the arctangent function in the four quadrants.

[0026] Preferably, the formula for extracting the pure diffuse reflection component by the high dynamic range de-reflection image reconstruction module is as follows:

[0027] ;

[0028] in, This represents the intensity of the non-polarized pure diffuse reflection component in the non-polarized light intensity matrix.

[0029] Preferably, the high dynamic range de-glare image reconstruction module, when reconstructing the de-glare image:

[0030] Using the linear polarization angle matrix as a structural guide map for spatial filtering, joint bilateral filtering is performed to eliminate processing texture noise and protect defect edges that produce abrupt changes in three-dimensional normals.

[0031] Preferably, in the structure tensor matrix feature extraction module:

[0032] After fusing the dereflected image with the linear polarization degree matrix to construct a two-dimensional vector field, Gaussian smoothing derivative kernels are used to obtain the horizontal and vertical Gaussian partial derivatives of the two-dimensional vector field, respectively. Then, Gaussian convolution smoothing model is constructed using the horizontal and vertical Gaussian partial derivatives to obtain the local structure tensor matrix.

[0033] Preferably, the solution feature energy map and consistency divergence map are specifically:

[0034] ;

[0035] ;

[0036] in, This represents the pixel value corresponding to the feature energy map. This represents the pixel value corresponding to the consistency scatter plot. and Let represent the first and second eigenvalues ​​of the local structure tensor matrix, respectively, and Greater than or equal to .

[0037] Preferably, the multi-dimensional feature fusion defect identification module filters suspected defect pixels based on the dual threshold judgment logic of feature energy map and consistency divergence map to generate defect mask, uses connected component algorithm to aggregate independent defect connected components, calculates the second central moment of independent defect connected components to obtain aspect ratio features, and calculates the average linear polarization degree in independent defect connected components.

[0038] The judgment rule is as follows:

[0039] When the average linear polarization degree is less than the depolarization effect threshold, it is determined to be a Class I defect;

[0040] When the average linear polarization degree is greater than or equal to the depolarization effect threshold and the aspect ratio characteristic is greater than the narrow shape threshold, it is judged as a second type of defect.

[0041] When the average linear polarization degree is greater than or equal to the depolarization effect threshold and the aspect ratio characteristic is less than or equal to the elongated shape threshold, it is judged as a third type of defect.

[0042] A machine scanning-based method for detecting surface defects in self-tapping and self-drilling screws includes:

[0043] A blue light polarization illumination system with a preset length and a DoFP polarization array camera are used, along with a dual telecentric lens. The light source and camera are triggered synchronously through an encoder. After calibration, the original images of the screw in four polarization states are acquired synchronously.

[0044] Adaptive edge-guided interpolation is performed on the original image to reconstruct the full-resolution polarization intensity matrix. Stokes parameters are calculated to obtain the linear polarization degree DoLP characteristic matrix and the linear polarization angle AoLP characteristic matrix of the morphology.

[0045] Based on the dichroic reflectance model, pure diffuse reflectance components are extracted to remove highlights. The dynamic range is improved by adaptive gamma correction. Then, AoLP-guided joint bilateral filtering is used to reduce noise and preserve edges, resulting in a clean detection substrate image.

[0046] A two-dimensional vector field is constructed by fusing the dereflected image with the DoLP matrix, the composite gradient is calculated and the structure tensor matrix is ​​constructed, and the eigenvalues ​​are solved to obtain the energy map and the consistency divergence map, which distinguishes normal thread and defective areas.

[0047] Defect masks are generated using dual thresholds, connected domain geometric features are extracted, and defect classification is completed by combining the average linear polarization degree. Defect information is then output and linked to the rejection mechanism.

[0048] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0049] 1. This invention separates the specular and diffuse reflection components of a metal surface from a physical level by using polarization optics and a dichroic reflection model. This eliminates the problem of false defects caused by overexposure due to strong reflection and the masking of real defects. Compared with conventional image enhancement algorithms, it has a rigorous physical mechanism support, no algorithm artifacts, and the detection results are true and reliable.

[0050] 2. This invention employs a focal plane array polarization camera in its hardware architecture, eliminating moving mechanical parts and enabling simultaneous acquisition of multiple polarization states. It avoids motion misalignment and temporal artifacts, and its detection speed is perfectly suited to the demands of continuous high-speed production lines, addressing the shortcomings of traditional polarization detection methods, such as slow speed and poor stability. Furthermore, this method integrates two core feature sets: material properties and topological morphology. It can simultaneously cover chemical defects such as rust and oil stains, as well as physical defects such as scratches, cracks, and missing teeth, without requiring multiple detection systems. The solution boasts strong reproducibility, requires no massive sample training, and has a low maintenance threshold. Attached Figure Description

[0051] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0052] Figure 1 This is a system structure diagram of the present invention;

[0053] Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0054] Several embodiments of this application will now be described in more detail with reference to the accompanying drawings to enable those skilled in the art to implement this application. This application may be embodied in many different forms and for various purposes and should not be limited to the embodiments set forth herein. These embodiments are provided to make this application thorough and complete, and to fully convey the scope of this application to those skilled in the art. The embodiments described do not limit this application.

[0055] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having a meaning consistent with their meaning in the relevant field and / or the context of this specification, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0056] Example 1

[0057] Its specific implementation method is combined with the appendix Figure 1 and attached Figure 2 Please provide a detailed explanation.

[0058] Appendix Figure 1 The diagram shows the structural block diagram of the machine scanning-based self-tapping and self-drilling screw surface defect detection system provided in the embodiments of the present invention. It illustrates the connection relationship between the multi-angle polarization image array acquisition module and the multi-dimensional feature fusion defect recognition module, and marks the main functional interaction flow of each module.

[0059] Appendix Figure 2 The flowchart of the machine scanning-based self-tapping and self-drilling screw surface defect detection method provided in the embodiments of the present invention shows the complete steps from acquiring original screw images in four polarization states to outputting defect information and linking the rejection mechanism.

[0060] In this embodiment, it includes:

[0061] Module 1: Multi-angle polarization image array acquisition module, configured to: synchronously capture multi-angle polarization images of the surface of a self-tapping and self-drilling screw by using a polarization illumination integrating sphere and a focal plane segmented polarization camera;

[0062] This module is the physical data input terminal of the entire detection system. Its core purpose is to overcome the strong specular reflection (highlight overexposure) caused by the complex three-dimensional curved surface formed by the thread peaks and the cutting edge of the drill tail of self-tapping and self-drilling screws (especially carbon steel / stainless steel materials with galvanized or Dacromet coating) under conventional light source illumination, and at the same time obtain the surface reflected light flux data under different polarization states.

[0063] Ordinary diffuse dome lights or coaxial lights cannot physically eliminate the specular reflection of bright metals. This system uses a high-frequency flashing blue LED light source array with a center wavelength of 450nm as the illumination basis. Blue light was chosen because its shorter wavelength results in a weaker diffraction effect on the metal surface, allowing it to more sharply reflect micron-level surface scratches and burr edges.

[0064] The outer casing of the light source array features a specially designed polarization illumination integrating sphere structure. The inner wall of the integrating sphere is coated with a barium sulfate layer that has high reflectivity and no polarization degradation effect, ensuring that the internal light undergoes multiple diffuse reflections to form an extremely uniform illumination field. A large-area linear polarizer is installed at the light outlet of the integrating sphere. This polarizer modulates the emitted unpolarized blue light into linearly polarized light with a single vibration direction. The transmission axis of the polarizer is precisely calibrated during system installation and defined as the system's reference polarization angle of 0°.

[0065] To avoid the fatal drawbacks of using traditional mechanically rotating polarizing lenses, such as mechanical vibration, image misalignment, and excessive processing time, this system employs a pixel-level polarization array camera based on focal plane array technology. A layer of microlenses and micropolarizer arrays is directly integrated via photolithography above the CMOS image sensor inside the camera.

[0066] Specifically, the micro-polarizer array uses 2×2 pixels as a macropixel unit. Within a macropixel unit, four adjacent pixels are respectively covered with micro-metal wire grid polarizers with transmission axes of 0°, 45°, 90°, and 135°. This hardware architecture ensures that the system can simultaneously capture image information of four different polarization states on the surface of a self-drilling screw at the same exposure time and from the same viewing angle, eliminating artifacts caused by time differences during the screw's motion scanning process.

[0067] Regarding the lens, a high-resolution, double-telecentric lens is essential. Self-drilling screws exhibit significant depth variations (from the nut to the drill tip), and ordinary lenses produce perspective distortion, causing the screw pitch to appear larger in the foreground and smaller in the background. A double-telecentric lens not only eliminates perspective distortion, ensuring a constant magnification of the screw within the depth of field, but also ensures that the principal ray of light entering the camera sensor is strictly perpendicular to the sensor plane. This is crucial for polarization cameras, as obliquely incident light causes a sharp decrease in the extinction ratio of the micro-polarizer, resulting in severe polarization crosstalk errors.

[0068] To achieve high-speed online inspection for industrial applications, self-tapping and self-drilling screws are placed on a transparent rotating glass disk or a V-groove-based linear conveyor belt. A high-resolution incremental photoelectric rotary encoder is mounted on the drive shaft of the conveyor mechanism. The encoder's A / B phase pulse signals are connected to a field-programmable logic controller (PLC) or a dedicated vision trigger board.

[0069] When the encoder pulse count reaches a preset physical interval (e.g., one pulse for every 0.1 mm advance), the trigger board generates a microsecond-level hard trigger square wave signal, which is simultaneously sent to the blue LED light source controller and the polarization camera. Upon receiving the trigger signal, the light source controller uses an overload constant current drive to cause the LED array to produce an ultra-short, high-intensity flash, typically lasting 10 to 50 microseconds; the camera completes the global shutter exposure within this flash window. This extremely short exposure time not only freezes the screw's movement, eliminating motion blur, but also avoids interference from ambient stray light.

[0070] Before the system is officially put into testing, rigorous system-level calibration must be performed to ensure data reproducibility. Dark current compensation involves triggering the camera to perform multiple exposures in a completely dark environment, calculating the average response value of each pixel, and generating a dark-field image substrate. In actual operation, this dark-field image must be subtracted from each acquired raw image to eliminate sensor thermal noise.

[0071] Secondly, there is the extinction ratio calibration of the micro-polarizer: Due to limitations in semiconductor manufacturing processes, micro-polarizers cannot achieve ideal perfect polarization filtering, and there are transmittance differences and crosstalk between polarizers at different angles. The camera is illuminated using a known perfectly unpolarized uniform light source (the light source inside the integrating sphere), and a uniform light field image is acquired. The average response coefficients of the four channels at 0°, 45°, 90°, and 135° are calculated, thereby generating a full-frame gain flat-field calibration matrix to ensure the physical accuracy of subsequent light intensity calculations.

[0072] Module 2: Stokes polarization parameter calculation module, configured to connect to the multi-angle polarization image array acquisition module, to receive multi-angle polarization images and perform demosaic interpolation, and to calculate the total light intensity matrix, linear polarization degree matrix and linear polarization angle matrix characterizing the surface physical material and three-dimensional topological properties based on Stokes vector theory;

[0073] This module receives the raw mixed polarization image from the "multi-angle polarization image array acquisition module" and, based on Stokes vector theory of physical optics, transforms the raw pixel electrical signals into a polarization parameter matrix reflecting the physical material and three-dimensional topological properties of the screw surface. The processing logic of this module forms the mathematical foundation for subsequent defect identification.

[0074] The raw image output by the DoFP polarization camera is a single-channel image in mosaic form. In this image, each pixel at a spatial coordinate contains only polarization intensity information from one of the following: 0°, 45°, 90°, or 135°. To achieve this at any pixel location... To simultaneously acquire the light intensity of four polarization states, de-mosaic interpolation calculations must be performed.

[0075] Because the threaded edges of self-drilling screws contain a large number of high-frequency details, traditional bilinear interpolation can lead to severe edge blurring and artificial "zipper effect." Therefore, this system adopts an adaptive edge-guided interpolation algorithm based on image spatial gradient.

[0076] For any pixel location lacking a polarization component, the algorithm calculates the image gradients in the horizontal and vertical directions within a 5×5 local neighborhood centered on that pixel. Let... The grayscale values ​​of the original mosaic image, and the horizontal gradient. and vertical gradient The definition is as follows:

[0077] ;

[0078] ;

[0079] The algorithm then determines the interpolation weights based on the relative magnitudes of the gradients. If This indicates the presence of vertical edges in the local region. During interpolation, adjacent pixels in the same phase along the horizontal direction should be given greater weight, and vice versa. For diagonal directions (such as interpolation of 45° and 135° polarization components), the gradients in both diagonal directions are calculated additionally. Through this adaptive edge-guided interpolation, the system calculates the gradients at each pixel coordinate. Four light intensity image matrices with complete full resolution were reconstructed from the above, denoted as: , , and .

[0080] The polarization state of a light wave can be determined by the four components of the Stokes vector. Full description. In natural and industrial visual environments, reflected light from metal surfaces typically does not produce a significant circular polarization component, i.e., the circular polarization parameter. The values ​​approach zero. Therefore, this system only calculates the first three Stokes parameters that can adequately characterize the linear polarization properties.

[0081] The system performs pixel-by-pixel calculations on the four full-resolution intensity matrices obtained after resolution interpolation:

[0082] First Stokes parameter This represents the total intensity of the reflected light, equivalent to a grayscale image from a traditional unpolarized camera. Its calculation formula is:

[0083] ;

[0084] To reduce the impact of interpolation errors, more rigorous equivalent formulas are usually used in practical engineering:

[0085] ;

[0086] Alternatively, you can use:

[0087] ;

[0088] This system uses the arithmetic mean of the two to maximize the reduction of high-frequency spatial noise:

[0089] ;

[0090] Second Stokes parameter It characterizes the intensity difference of linearly polarized light in the 0° and 90° directions, reflecting the horizontal or vertical geometric edge features of the screw surface:

[0091] ;

[0092] Third Stokes parameter It characterizes the intensity difference of linearly polarized light in the 45° and 135° directions, and it is extremely sensitive to changes in the diagonal slope of the screw's helical tooth surface:

[0093]

[0094] After this calculation, the system transforms the original one-dimensional mosaic image into three floating-point matrices of equal size. , , This forms the mathematical basis for all subsequent de-reflection and surface morphology analysis.

[0095] After obtaining the Stokes parameters, this module further calculates two crucial derivative physical quantities: the degree of linear polarization and the angle of linear polarization. These two parameters are the decisive characteristics that distinguish screws from "normal metallic high-gloss surfaces," "scratches / fracture defects," and "rust / oil stains and non-metallic deposits."

[0096] linear polarization degree This represents the proportion of linearly polarized light in the total intensity of the reflected light, and its value ranges from 0 to 1. The formula for its calculation is:

[0097] ;

[0098] Physical meaning mapping: According to Fresnel's law of reflection, when illumination light with a specific polarization state shines on the perfectly galvanized metal surface of a self-drilling screw, whether it is specular reflection or directional diffuse reflection, the reflected light will largely retain its original polarization characteristics. At this time, the reflected light in this area... The value is relatively high. However, when rust, oxide layers, or rough fracture burrs are present on the surface, light undergoes strong multiple scattering (depolarization effect) in these microscopic roughness or non-metallic media, causing the reflected light to degenerate into unpolarized light, corresponding to the area The value will drop sharply. Therefore, The matrix is ​​equivalent to a "material distribution map" that can separate chemical defects such as rust and oxidation from the normal physical surface of metal.

[0099] linear polarization angle This represents the rotation angle of the polarization axis of the linearly polarized component in the reflected light relative to the system's reference coordinate system (0°), and its value typically ranges from... arrive Between. The calculation formula is:

[0100] ;

[0101] In order to avoid Phase reversal of the function and To prevent division by zero overflow errors, the actual algorithm strictly calls the four-quadrant arctangent function atan2(y,x) from computer graphics, specifically:

[0102] ;

[0103] Physical meaning mapping: The value of is closely related to the direction of the microscopic local spatial normal vector on the screw surface. The threads of self-tapping and self-drilling screws exhibit a complex continuous spatial helical structure, and their surface normals change continuously in three-dimensional space, leading to... The image also exhibits a gradient pattern highly consistent with the thread morphology. If a minute mechanical scratch or dent appears on the thread tooth surface, the microscopic three-dimensional normal vector at that location will undergo a drastic change, causing the pixel to... Values ​​relative to surrounding normal continuous thread surfaces The value undergoes a significant jump. Therefore, The matrix is ​​a high-precision "three-dimensional surface topological gradient map" that can be used to identify even in grayscale images with varying light intensity. Micrometer-level morphological damage that is difficult to detect in )

[0104] In summary, the Stokes polarization parameter calculation module reconstructs the original photoelectric signal into a representation of the total light intensity through rigorous image spatial differential interpolation and physical formula calculation. Matrix, representing material purity and roughness The matrix, and the three-dimensional topological normals representing the surface. matrix.

[0105] Module 3: High Dynamic Range De-reflection Image Reconstruction Module, configured to connect with the Stokes polarization parameter calculation module, used to extract the pure diffuse reflection component based on the total light intensity matrix and the linear polarization degree matrix, and to perform anisotropic joint bilateral filtering guided by the linear polarization angle matrix to reconstruct the de-reflection image;

[0106] After obtaining the Stokes polarization parameter matrix output by module two ( ) and derived parameter matrix ( After that, the core task of this module is to use the polarization physics characteristics to remove the strong specular reflection (highlight overexposure) on the surface of the self-drilling screw from both mathematical and physical levels, and reconstruct a high-quality "diffuse reflection substrate image" that contains only the real surface texture and has no light spot interference.

[0107] According to the dichroic reflection model, the reflected light after a light source shines on a metal surface can be decomposed into two parts: specular reflection component and diffuse reflection component.

[0108] For the galvanized or stainless steel surface of self-tapping and self-drilling screws, when illuminated at a specific angle, the specularly reflected light will maintain a high degree of polarization of the incident light (manifesting as an extremely high degree of linear polarization). This portion of light is the culprit behind image overexposure and the masking of defects. However, the diffuse reflected light scattered after penetrating the microscopic rough structure of the metal surface has its polarization state completely disrupted, transforming into unpolarized light. This portion of light carries the true morphological information such as scratches, microcracks, and rust spots.

[0109] Therefore, based on the aforementioned physical mechanism, this module constructs a non-polarized light intensity matrix. The solution model. Due to This represents the total reflected light intensity, while This represents the proportion of linearly polarized light, therefore the intensity of unpolarized light (i.e., the pure diffuse component) can be accurately calculated using the following formula:

[0110] ;

[0111] Through this physical mechanism-based multiplication operation, the system transforms the original image... middle The light intensity of overexposed reflective areas with extremely high reflectivity (such as flares at the tip of thread teeth) was drastically suppressed, while the light intensity of... Low-value dark texture areas and rust spots are preserved, fundamentally achieving physical-level specular removal.

[0112] Extracted non-polarized diffuse reflection component image While highlights are eliminated, the overall image's absolute brightness decreases significantly due to the stripping of a large amount of specular reflection energy, resulting in lower contrast and hindering subsequent extraction of minute edges. Conventional global histogram equalization also amplifies sensor thermal noise in dark areas.

[0113] Therefore, this module designs a method based on Stokes total light intensity. A priori-guided adaptive local gamma correction remapping algorithm. For each pixel in the image... Dynamically calculate its corresponding gamma exponent :

[0114] ;

[0115] in, Adjustment coefficient (range of values) arrive ), This is a constant used to control the sensitivity of light intensity mapping.

[0116] Subsequently, the diffuse image is remapped using this dynamic gamma exponent to obtain an enhanced de-reflected image. :

[0117] ;

[0118] in, This is the maximum quantization value for system pixels (e.g., 255 for an 8-bit camera and 4095 for a 12-bit camera). This algorithm can intelligently stretch details in the original highlight areas while maintaining robust contrast in the original shadow areas, thereby reconstructing a clear physical substrate image with extremely high dynamic range.

[0119] After high-dynamic stretching, the micro-grooves (non-defect noise) that were originally hidden in the metal texture are amplified because the dominant specular reflection light is removed. If ordinary Gaussian filtering is used for noise reduction, the critical defect edges (such as tiny scratches) of the self-drilling screw will inevitably be blurred.

[0120] This system introduces the joint bilateral filtering theory and uses the linear polarization angle matrix calculated by module two. As a structural guide diagram for spatial filtering. Due to Representing the microscopic three-dimensional normal direction, the actual defect (crack) is... There must be a sudden change in the normal line, while normal machining tool marks are present. The surface is smooth and continuous.

[0121] Define the filter window size as The filtered de-reflected image The calculation formula is:

[0122] ;

[0123] in, The Gaussian kernel function for spatial distance. This is the Gaussian kernel function with a range (here, the polarization angle difference). Normalized weighting coefficients. Ensure energy conservation:

[0124] ;

[0125] Through this Guided joint bilateral filtering allows the system to smooth out minor machining texture noise on the screw surface while perfectly protecting and sharpening real defect edges that cause abrupt changes in 3D normals, providing a perfectly clean data source for subsequent tensor feature extraction.

[0126] Module 4: Structure Tensor Matrix Feature Extraction Module, configured to connect with the high dynamic range de-reflection image reconstruction module, used to fuse the de-reflection image with the linear polarization degree matrix to construct a two-dimensional vector field, and calculate the feature energy map and consistency divergence map by calculating the local structure tensor matrix;

[0127] Traditional industrial vision inspection often relies on first-order gradients (such as Sobel and Canny operators) to extract defect edges. However, in the context of complex threads in self-drilling screws, simple first-order gradients will misjudge all normal thread surface undulations as defects.

[0128] To address this pain point, this module introduces the structure tensor, a high-order differential tool used in computer vision to process complex textures. By calculating the local geometric anisotropy of the image, it mathematically isolates the "normal continuous thread topology" from the "abrupt micro-defects".

[0129] This module no longer simply processes single-channel grayscale images, but instead processes the final de-reflected image output by the previous module. and the linear polarization degree matrix Merged into a two-dimensional vector field .

[0130] in This is a weighting balancing factor used to align dimensions. The physical significance of this fusion is that it enables subsequent differential operations to not only perceive spatial changes in light intensity but also synchronously perceive spatial changes in material purity (roughness), which plays a decisive role in distinguishing between "black oil stains" and "black pit defects."

[0131] Because partial derivatives are extremely sensitive to noise, the system uses a scale of Gaussian smoothing derivative kernels are used to obtain the horizontal direction of the vector field. and vertical direction The partial derivatives of . Let . For a two-dimensional Gaussian function, the partial derivative is defined as:

[0132] ;

[0133] ;

[0134] The result here and It is a composite gradient matrix containing multidimensional physical information.

[0135] Construction of local structure tensor matrix at continuous integral scale

[0136] The gradient of a single pixel is unstable. The core idea of ​​the structure tensor is to determine the geometric structure of a region (whether it is a flat region, an edge region, or a region with abrupt corner changes) by statistically analyzing the distribution direction of the gradient within a local integral neighborhood.

[0137] The system at each pixel coordinate At this point, a composite gradient is used to construct a positive semidefinite initial tensor matrix :

[0138] ;

[0139] In order to reflect local structural features, it is necessary to... Integral scale is Gaussian convolution smoothing (usually requires) Thus, the final structure tensor matrix is ​​obtained. :

[0140] ;

[0141] This matrix It contains the covariance information of the magnitude and direction of all micro-gradients in the local neighborhood of the pixel. For a self-drilling screw, the gradient direction in the local neighborhood of a normal continuous thread is highly consistent; however, if there are scratches or gaps on the thread, the gradient direction in the local neighborhood will show multiple orthogonal branches.

[0142] For the 2×2 structure tensor matrix at each pixel Since it is a real symmetric matrix, it must have two non-negative real eigenvalues, denoted as . and (and set) ).

[0143] The system calculates the feature value of each pixel in parallel using the formula for solving the eigenvalues ​​of algebraic equations.

[0144] ;

[0145] ;

[0146] These two eigenvalues ​​have extremely profound geometric and physical significance:

[0147] It represents the energy of the direction (principal gradient direction) where the gray level and polarization change is strongest in a local area.

[0148] It represents energy in a direction orthogonal to the main direction.

[0149] Based on the eigenvalues, this module further calculates two crucial detection feature maps: the feature energy map and the consistency divergence map.

[0150] Characteristic energy map That is, the trace of the matrix:

[0151]

[0152] Energy diagram Highlight all physical edges, including normal thread crests and abnormal scratches.

[0153] Consistency scatter plot This is the core indicator for distinguishing between normal threads and defects:

[0154] ;

[0155] The values ​​of are strictly distributed in between.

[0156] For normal thread tooth surfaces and continuous cutting edges at the drill shank, due to their typical one-dimensional linear characteristics, there is a strong gradient in only one direction. Consistency at this point It approaches 1 (exhibiting strong anisotropy).

[0157] For microscopic pits, cross-shaped scratches, fracture gaps, and rust spots with abrupt material changes, these defects, being two-dimensional topological structures with strong variations in all directions, lead to… and All have large and relatively close values, indicating consistency. It will drop sharply to near 0 (becoming isotropic).

[0158] For a defect-free, smooth cylindrical body region, and All approach , It is extremely small, so there is no need to calculate the divergence.

[0159] Through this series of extremely rigorous advanced partial differential and linear algebra operations, the structure tensor matrix feature extraction module successfully transformed the extremely complex problem of "screw surface defect detection" into an analysis of the eigenvalue matrix. and The problem of determining the combined state fundamentally avoids the drawback of template matching failure in existing technologies, providing a perfect, dimensionless mathematical criterion for the final defect classification and judgment.

[0160] Module 5: Multidimensional feature fusion defect identification module, configured to connect with the structure tensor matrix feature extraction module, used to generate defect masks based on feature energy maps and consistency divergence maps and extract geometric features of connected domains, and combine the average linear polarization degree within the connected domains to complete the classification and determination of defects;

[0161] The preceding "Structure Tensor Matrix Feature Extraction Module" has already output the feature energy map representing the local edge energy. And the consistency divergence plot characterizing the consistency of local geometric topological orientation. Meanwhile, the "Stokes polarization parameter calculation module" retains the linear polarization degree matrix characterizing the material purity. The core function of this module is to establish decision boundaries in three independent high-dimensional physical fields, bridging the gap between "low-level pixel features" and "macroscopic defect categories," thereby enabling the classification and extraction of minute scratches, fractures, burrs, and rust spots. The specific execution steps of this module are as follows:

[0162] The system needs to isolate all "potentially defective" areas from the complex background of screws. Normal screw machining textures have low energy, while thread peaks, although high in energy, also exhibit high consistency (strong anisotropy). Only true defects simultaneously exhibit both high-energy abrupt changes and low consistency (isotropy).

[0163] Therefore, the system constructs a binary initial defect mask matrix. For any pixel in the image Its decision logic is strictly defined by the following piecewise function:

[0164] ;

[0165] in, The characteristic energy background rejection threshold is used to filter out minute sensor noise and extremely shallow normal machining marks. A consistency divergence threshold (typically set between 0.2 and 0.4) is used to distinguish one-dimensional continuous edges (such as normal threads). Approaching 1) and two-dimensional mutation regions (such as scratches or notches, their (Approaching 0).

[0166] Through the above logical AND operation, the mask matrix... Median The pixels constitute all the candidate points for suspected defects.

[0167] Initial mask matrix Defects in the image typically appear as discrete clusters of pixels. To perform morphological classification of defects, the system employs an eight-neighborhood region growing algorithm to aggregate spatially adjacent suspected defect pixels into independent sets of connected regions. ,in The total number of connected regions. Indicates the first An independent defective connected domain.

[0168] For each connected component The system calculates its higher-order topological parameters based on the spatial geometric moment theory.

[0169] Calculate the zeroth moment (i.e., the area of ​​the defective pixel) of the connected component. :

[0170] ;

[0171] Next, calculate the first-order space moments. and :

[0172] ;

[0173] ;

[0174] This allows for the precise location of the geometric centroid coordinates of the connected domain of the defect. :

[0175]

[0176] To distinguish the shape of defects (such as elongated scratches versus circular pits), it is necessary to further calculate the second-order central moments. And construct the covariance matrix of the region:

[0177] ;

[0178] By solving for the eigenvalues ​​of the covariance matrix, the length of the equivalent ellipse major axis of the defective connected domain can be obtained. and minor axis length Based on this, the aspect ratio characteristics of the defect were calculated. : ;

[0179] After acquiring the geometric features, the system introduces the linear polarization parameter of the material dimension. To complete the final classification. Geometric features can only reflect the destruction of shape, while It can reflect changes in chemical composition.

[0180] The system extracts each defective connected component. All pixels within The value is calculated, and the average linear polarization degree of the region is determined. :

[0181]

[0182] Combining geometric aspect ratio and material polarization The system designed the following non-machine learning absolute physical classification criteria:

[0183] Category 1 defects (rust spots, oxide layers, or oil stains):

[0184] Judgment condition: when hour( The threshold for depolarization effect is usually 0.15.

[0185] Physical basis: Regardless of its shape and area, if the average linear polarization degree of the region is extremely low, it indicates that the optical properties of the region have changed from directional reflection by the metal to multi-body scattering (depolarization) by non-metallic impurities. The system directly classifies it as surface chemical contamination or corrosion defects.

[0186] Type II defects (mechanical scratches, micro-cracks):

[0187] Judgment condition: when And geometric aspect ratio hour( For elongated shapes, the threshold is typically set to 3.0 or higher.

[0188] Physical basis: The high degree of polarization indicates that the region is still a clean metallic material, but its geometry is extremely elongated (the major axis is much larger than the minor axis), and it causes an isotropic abrupt change in the structural tensor, which perfectly matches the physical behavior of microcracks caused by tool scratches or stress.

[0189] Category 3 defects (impact dents, missing teeth, fractures, burrs, and flash):

[0190] Judgment condition: when And geometric aspect ratio At that time, and the defect area Greater than the lower limit of tolerance for small noise area.

[0191] Physical basis: The material is still metal, but the shape is not elongated or strip-shaped; instead, it presents as a blocky or polygonal area abrupt change. If it appears at the edge of a thread, it is a missing tooth or burr; if it appears on a flat surface, it is an impact dent. The system will output the exact centroid coordinates of this type of defect. Based on the size and area, the joint rejection mechanism executes the rejection of waste products.

[0192] Example 2

[0193] Please see Figure 2 A machine scanning-based method for detecting surface defects in self-tapping and self-drilling screws includes the following components:

[0194] A blue light polarization illumination system with a preset length and a DoFP polarization array camera are used, along with a dual telecentric lens. The light source and camera are triggered synchronously through an encoder. After calibration, the original images of the screw in four polarization states are acquired synchronously.

[0195] Adaptive edge-guided interpolation is performed on the original image to reconstruct the full-resolution polarization intensity matrix. Stokes parameters are calculated to obtain the linear polarization degree DoLP characteristic matrix and the linear polarization angle AoLP characteristic matrix of the morphology.

[0196] Based on the dichroic reflectance model, pure diffuse reflectance components are extracted to remove highlights. The dynamic range is improved by adaptive gamma correction. Then, AoLP-guided joint bilateral filtering is used to reduce noise and preserve edges, resulting in a clean detection substrate image.

[0197] A two-dimensional vector field is constructed by fusing the dereflected image with the DoLP matrix, the composite gradient is calculated and the structure tensor matrix is ​​constructed, and the eigenvalues ​​are solved to obtain the energy map and the consistency divergence map, which distinguishes normal thread and defective areas.

[0198] Defect masks are generated using dual thresholds, connected domain geometric features are extracted, and defect classification is completed by combining the average linear polarization degree. Defect information is then output and linked to the rejection mechanism.

[0199] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0200] It should be noted that, in this document, the use of relational terms such as "first" and "second" is merely for distinguishing one entity or operation from another, and does not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0201] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0202] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0203] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A surface defect detection system for self-tapping and self-drilling screws based on machine scanning, characterized in that, include: The multi-angle polarization image array acquisition module is configured to simultaneously capture multi-angle polarization images of the surface of a self-tapping and self-drilling screw using a polarization illumination integrating sphere and a focal plane segmented polarization camera. The Stokes polarization parameter calculation module is configured to receive multi-angle polarization images and perform demosaic interpolation to calculate the total light intensity matrix, linear polarization degree matrix, and linear polarization angle matrix that characterize the surface physical material and three-dimensional topological properties. The high dynamic range de-reflection image reconstruction module is configured to extract the pure diffuse reflection component based on the total light intensity matrix and the linear polarization degree matrix, and perform anisotropic joint bilateral filtering guided by the linear polarization angle matrix to reconstruct the de-reflection image. The structural tensor matrix feature extraction module is configured to: fuse the dereflected image with the linear polarization degree matrix to construct a two-dimensional vector field, and calculate the feature energy map and the consistency divergence map by calculating the local structural tensor matrix; The multi-dimensional feature fusion defect identification module is configured to: generate a defect mask based on the feature energy map and the consistency scatter map, extract the geometric features of the connected domain, and complete the classification and determination of the defect by combining the average linear polarization degree within the connected domain.

2. The machine scanning-based surface defect detection system for self-tapping and self-drilling screws according to claim 1, characterized in that, In the Stokes polarization parameter calculation module: An adaptive edge-guided interpolation algorithm based on image spatial gradient is used for demosaic interpolation. The horizontal and vertical image gradients in the local neighborhood are calculated respectively. The interpolation weights of different polarization channels are determined according to the relative magnitudes of the horizontal and vertical image gradients in the local neighborhood, and a light intensity image matrix with multiple polarization angles with full resolution is reconstructed.

3. The machine scanning-based surface defect detection system for self-tapping and self-drilling screws according to claim 2, characterized in that, The specific formula for calculating the total light intensity matrix is ​​as follows: ； in, Represents pixel coordinates The total light intensity at a given location corresponds to the pixel value. , , as well as These represent the full-resolution matrix of the corresponding polarization transmission axis angles in pixel coordinates. The light intensity value at that location.

4. The machine scanning-based surface defect detection system for self-tapping and self-drilling screws according to claim 3, characterized in that, The specific steps for solving the linear polarization degree matrix and linear polarization angle matrix are as follows: ； ； in, This represents the pixel value corresponding to the degree of linear polarization. This represents the pixel value corresponding to the linear polarization angle. This represents the intensity difference of the combined linearly polarized light in the first orthogonal direction. This represents the intensity difference of the combined linearly polarized light in the second orthogonal direction. This represents the arctangent function in the four quadrants.

5. The machine scanning-based surface defect detection system for self-tapping and self-drilling screws according to claim 4, characterized in that, The formula for extracting the pure diffuse reflection component in the high dynamic range de-reflection image reconstruction module is as follows: ； in, This represents the intensity of the non-polarized pure diffuse reflection component in the non-polarized light intensity matrix.

6. The machine scanning-based surface defect detection system for self-tapping and self-drilling screws according to claim 5, characterized in that, The high dynamic range de-reflection image reconstruction module, when reconstructing a de-reflection image: Using the linear polarization angle matrix as a structural guide map for spatial filtering, joint bilateral filtering is performed to eliminate processing texture noise and protect defect edges that produce abrupt changes in three-dimensional normals.

7. The machine scanning-based surface defect detection system for self-tapping and self-drilling screws according to claim 1, characterized in that, In the structure tensor matrix feature extraction module: After fusing the dereflected image with the linear polarization degree matrix to construct a two-dimensional vector field, Gaussian smoothing derivative kernels are used to obtain the horizontal and vertical Gaussian partial derivatives of the two-dimensional vector field, respectively. Then, Gaussian convolution smoothing model is constructed using the horizontal and vertical Gaussian partial derivatives to obtain the local structure tensor matrix.

8. The machine scanning-based surface defect detection system for self-tapping and self-drilling screws according to claim 7, characterized in that, The specific steps for solving the eigenenergy map and the consistency divergence map are as follows: ； ； in, This represents the pixel value corresponding to the feature energy map. This represents the pixel value corresponding to the consistency scatter plot. and Let represent the first and second eigenvalues ​​of the local structure tensor matrix, respectively, and Greater than or equal to .

9. The machine scanning-based surface defect detection system for self-tapping and self-drilling screws according to claim 1, characterized in that, The multi-dimensional feature fusion defect identification module uses a dual threshold judgment logic based on feature energy map and consistency divergence map to filter suspected defect pixels and generate defect masks. It uses a connected component algorithm to aggregate independent defect connected components, calculates the second central moment of the independent defect connected components to obtain aspect ratio features, and calculates the average linear polarization degree within the independent defect connected components. The judgment rule is as follows: When the average linear polarization degree is less than the depolarization effect threshold, it is determined to be a Class I defect; When the average linear polarization degree is greater than or equal to the depolarization effect threshold and the aspect ratio characteristic is greater than the narrow shape threshold, it is judged as a second type of defect. When the average linear polarization degree is greater than or equal to the depolarization effect threshold and the aspect ratio characteristic is less than or equal to the elongated shape threshold, it is judged as a third type of defect.

10. A method for detecting surface defects in self-tapping and self-drilling screws based on machine scanning, and a system for detecting surface defects in self-tapping and self-drilling screws based on machine scanning according to any one of claims 1-9, characterized in that, include: A blue light polarization illumination system with a preset length and a DoFP polarization array camera are used, along with a dual telecentric lens. The light source and camera are triggered synchronously through an encoder. After calibration, the original images of the screw in four polarization states are acquired synchronously. Adaptive edge-guided interpolation is performed on the original image to reconstruct the full-resolution polarization intensity matrix. Stokes parameters are calculated to obtain the linear polarization degree DoLP characteristic matrix and the linear polarization angle AoLP characteristic matrix of the morphology. Based on the dichroic reflectance model, pure diffuse reflectance components are extracted to remove highlights. The dynamic range is improved by adaptive gamma correction. Then, AoLP-guided joint bilateral filtering is used to reduce noise and preserve edges, resulting in a clean detection substrate image. A two-dimensional vector field is constructed by fusing the dereflected image with the DoLP matrix, the composite gradient is calculated and the structure tensor matrix is ​​constructed, and the eigenvalues ​​are solved to obtain the energy map and the consistency divergence map, which distinguishes normal thread and defective areas. Defect masks are generated using dual thresholds, connected domain geometric features are extracted, and defect classification is completed by combining the average linear polarization degree. Defect information is then output and linked to the rejection mechanism.

Images