A method for recognizing surface defects based on illumination disturbance consistency

Abstract

The application discloses a curved surface defect recognition method based on illumination disturbance consistency, and belongs to the technical field of curved surface defect recognition, and solves the problem that real linear defects and reflection pseudo-defects in a high-reflective curved surface are difficult to distinguish. The method comprises the following steps: acquiring a plurality of illumination images of a same high-reflective curved surface detection area under a plurality of illumination conditions; pre-processing and spatially registering the plurality of illumination images to obtain a plurality of registered illumination images; constructing a direction response tensor; calculating an illumination disturbance consistency index map based on the direction response tensor; fusing the illumination disturbance consistency index map and the direction response intensity to generate a defect confidence map, and extracting a candidate defect area according to the defect confidence map; performing skeleton thinning and topological relationship construction on the candidate defect area to obtain a defect skeleton topological map; extracting defect structure attributes based on the defect skeleton topological map, and outputting a structured defect recognition result.

Description

Technical Field

[0001] This invention belongs to the field of surface defect recognition technology, specifically relating to a surface defect recognition method based on the consistency of illumination perturbation. Background Technology

[0002] Highly reflective curved surfaces are prone to producing bright spots, shadows, and specular reflection boundaries under ring light, oblique incident light, or polarized light conditions. Traditional grayscale thresholding, edge detection, and ordinary neural network segmentation can easily misjudge reflective areas as scratches, grooves, or cracks.

[0003] A single-illumination image can only reflect the grayscale or texture differences under a certain imaging state, and cannot determine whether a suspected linear structure remains stable under different illumination disturbances. Therefore, single-image defect identification is easily affected by illumination angle, surface curvature, and local reflection state.

[0004] Existing methods mostly output defect masks or defect boxes, which are difficult to further express the length, direction, continuity, fracture connection relationship and risk level of defects, and cannot meet the structured evaluation requirements in the precision inspection of highly reflective curved surfaces. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention aims to provide a surface defect identification method based on illumination perturbation consistency. By modeling illumination perturbation consistency and expressing the skeleton topology, it achieves defect identification, structural understanding, and grade evaluation, thereby solving the problem of difficulty in distinguishing between real linear defects and reflective pseudo-defects in highly reflective surfaces.

[0006] The objective of this invention can be achieved using the following specific technical solutions: The surface defect identification method based on illumination perturbation consistency includes the following steps: Step 1: Obtain a set of multi-illumination images of the same highly reflective curved surface detection area under multiple illumination conditions; Step 2: Preprocess and spatially register the multi-illumination image set to obtain the registered multi-illumination image set; Step 3: Perform multi-directional linear structure response calculations on each registered illumination image to construct the directional response tensor; Step 4: Calculate the illumination perturbation consistency index map based on the directional response tensor. The illumination perturbation consistency index map is obtained by fusing response intensity stability, directional stability and structural continuity stability. Step 5: Fuse the illumination perturbation consistency index map with the directional response intensity to generate a defect confidence map, and extract candidate defect regions based on the defect confidence map; Step 6: Refine the skeleton and construct the topological relationship of the candidate defect region to obtain the defect skeleton topology map; Step 7: Extract the defect structure attributes based on the defect skeleton topology map and output the structured defect identification results.

[0007] Further, in step 2, the preprocessing includes at least one of grayscale processing, noise reduction processing, brightness normalization processing, contrast enhancement processing, and geometric distortion correction processing; the spatial registration adopts feature point matching, phase correlation, affine transformation, perspective transformation, or sub-pixel level registration methods, and the registration process is as follows: select an image as a reference image, extract image feature points of the reference image and the image to be registered, match the image feature points, estimate the spatial transformation matrix based on the matching result, and perform interpolation transformation on the image to be registered based on the spatial transformation matrix.

[0008] Furthermore, in step 3, the directional response tensor is constructed as follows: Let N be the number of illuminated images, K be the number of directions, H be the image height, and W be the image width. For the registered image I′n(x,y) under the nth illumination condition, the linear structure enhancement operator in the kth direction is used. Calculate the directional response: ,in, This represents the linear structural response intensity of pixel (x, y) under the nth illumination condition and the kth direction; By combining the directional responses under all lighting conditions and all directions, we obtain the directional response tensor: , Where R represents the orientation response tensor, K represents the number of orientations, N represents the number of illumination images, and H and W represent the image height and image width, respectively.

[0009] Furthermore, in step 4, the illumination perturbation consistency index is calculated according to the following formula: C(x,y)=αS a (x,y)+βS b (x,y)+γS c (x,y), Where C(x,y) represents the illumination perturbation consistency index at pixel (x,y), and S a (x,y) represents the response intensity stability, used to characterize the degree of fluctuation in the linear response intensity of the same pixel or local area under different lighting conditions; S b (x,y) represents directional stability, used to characterize the consistency of the main response direction of the same pixel or local region under different lighting conditions; S c (x,y) represents the structural continuity stability, used to characterize the connectivity and continuity of the suspected linear structure in the local neighborhood; α, β, and γ are the weighting coefficients, respectively.

[0010] Specifically, the response intensity stability is obtained by comparing the relative deviation between the maximum directional response value under the nth illumination condition and the reference maximum response value under all illumination conditions and all directions; the directional stability is obtained by comparing the consistency between the main response direction and the reference main direction under different illumination conditions; and the structural continuity stability is obtained by performing threshold segmentation and connected component analysis on the maximum directional response map under different illumination conditions to calculate the local structural connectivity of the connected region where the pixel is located.

[0011] Furthermore, in step 5, the defect confidence map is generated as follows: First, the directional response intensity of the k-th direction under all illumination conditions is averaged to obtain the directional average response. ; Then, extract the maximum value of the pixel (x,y) in the average response across all directions to obtain the average response intensity in the principal direction. ; Finally, the average response intensity in the principal direction is... By fusing the illumination perturbation consistency index C(x,y), an initial defect confidence map is obtained: ,in, λ represents the initial defect confidence of pixel (x,y), and λ represents the response intensity adjustment parameter.

[0012] Further, the initial defect confidence map is normalized to obtain a defect confidence map P(x,y). The defect confidence map P(x,y) is then segmented according to a confidence threshold T to obtain a candidate defect region mask M(x,y). .

[0013] Furthermore, in step 6, the defect skeleton topology map is constructed as follows: 1) Perform morphological cleaning on the candidate defect area to remove isolated small noise areas and repair local fracture areas; 2) Refine the skeleton of the candidate defect region and extract the single-pixel skeleton map to characterize the direction of the center of the linear defect; 3) Count the number of eight-neighbor skeleton adjacencies of skeleton pixels, and identify skeleton endpoints, ordinary points and branching points based on the number of eight-neighbor skeleton adjacencies; 4) Use the skeleton endpoints and bifurcation points as topological nodes, and use the skeleton endpoints or bifurcation points as boundaries to divide the skeleton into several independent skeleton segments. Each skeleton segment represents a suspected linear defect structure. 5) Construct a defect skeleton topology diagram based on the topology nodes and the independent skeleton segments.

[0014] Further, the defect skeleton topology graph is represented as: G=(V,E,A), where V represents the set of topological nodes composed of skeleton endpoints and bifurcation points, E represents the set of connecting edges formed by skeleton segments, and A represents the set of skeleton segment attributes; the skeleton segment attributes include at least one of skeleton segment length, main direction, average width, continuity, and average illumination perturbation consistency confidence.

[0015] Specifically, in step 7, the defect structure attributes include at least one of defect length, defect direction, average width, average confidence level, average consistency, continuity, curvature, number of bifurcations, number of endpoints, and connection relationship; the structured defect identification result includes at least one of defect location, defect type, defect length, defect direction, defect width, confidence level, and risk level.

[0016] Compared with the prior art, the present invention has the following advantages: (1) This invention sets up multiple illumination disturbance conditions to image the same detection area. Instead of relying on grayscale or edge features in a single illumination image for defect judgment, it uses the stability difference between real linear defects and reflective pseudo-defects under multiple illumination conditions for identification, thereby improving the reliability of high reflective surface defect detection.

[0017] (2) The present invention constructs a directional response tensor that includes spatial location, illumination conditions and directional response, which can uniformly express the linear structural response of the same detection area under different illumination and different directions, providing a data basis for defect stability modeling across illumination and direction.

[0018] (3) The present invention proposes an illumination perturbation consistency index, which integrates response intensity stability, directional stability and structural continuity stability to characterize the stability of suspected defects under multiple illumination perturbations, thereby effectively suppressing reflective bright spots, shadow boundaries and random texture pseudo responses that only appear under specific illumination conditions.

[0019] (4) The present invention integrates the illumination perturbation consistency index and the directional response intensity to generate a defect confidence map, so that the extraction of candidate defect regions no longer directly depends on the gray level of the original image or a single response intensity, but simultaneously considers the linear response intensity and cross-illumination stability, thereby improving the authenticity of candidate defect regions.

[0020] (5) The present invention converts the candidate defect region from a pixel-level mask into a structured graph representation through a skeleton topology graph, which can extract structural attributes such as defect length, direction, width, continuity and connection relationship, and can restore the fracture defect structure caused by local reflection or segmentation fracture.

[0021] (6) This invention can output not only the defect area, but also the defect type, defect size, defect direction, confidence level and risk level, realizing the extension from defect detection to defect understanding and level assessment. It is applicable to the surface quality detection of highly reflective curved surfaces such as metal polished parts, optical curved surfaces, bright injection molded parts, and medical implant surfaces. Attached Figure Description

[0022] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of multi-light image capture according to the present invention; Figure 3 This is a schematic diagram illustrating the changes in images after preprocessing and spatial registration according to the present invention; Figure 4 This is a flowchart of the calculation process for the illumination perturbation consistency index of the present invention; Figure 5 This is a schematic diagram of the process of obtaining a single-pixel skeleton diagram from the skeleton topology diagram of the present invention; (a) represents the skeleton topology diagram, and (b) represents the single-pixel skeleton diagram. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0024] like Figure 1 As shown, this invention provides a method for identifying surface defects based on the consistency of illumination perturbation. The identification method includes the following steps: (a) Step 1: Obtain a set of multi-light images of the same highly reflective curved surface detection area under multiple lighting conditions.

[0025] like Figure 2 As shown, for the same highly reflective curved surface detection area, multiple light sources of different angles or types are set at a fixed detection station to image the same detection area and obtain a multi-illumination image set.

[0026] Different illumination conditions include, but are not limited to, left-side oblique incidence light, right-side oblique incidence light, ring light, coaxial light, polarized light, or combinations thereof. By perturbing the same detection area under various illumination conditions, real linear defects and reflective pseudo-defects exhibit different response characteristics under different illumination conditions.

[0027] (ii) Step 2: Preprocess and spatially register the multi-illumination image set to obtain the registered multi-illumination image set.

[0028] Preprocessing includes one or more of the following: denoising, brightness normalization, contrast enhancement, geometric distortion correction, and image registration. Denoising methods may include BM3D denoising, nonlocal mean denoising, bilateral filtering, or other image denoising methods; brightness normalization methods may include adaptive histogram equalization, grayscale normalization, or other illumination correction methods.

[0029] Spatial registration methods can employ feature point matching, phase correlation, affine transformation, perspective transformation, or sub-pixel level registration. The registration process is as follows: An image is selected as a reference image; image feature points are extracted from both the reference image and the image to be registered; these feature points are matched; the spatial transformation matrix is ​​estimated based on the matching results; and interpolation transformation is performed on the image to be registered based on the spatial transformation matrix. Specifically, the registration process can be represented as: ,in, Let n represent the spatial transformation function of the nth image relative to the reference image.

[0030] (III) Step 3: Perform multi-directional linear structure response calculations on each registered illumination image to construct the directional response tensor.

[0031] Specifically, for each illumination image, multiple directional line structure enhancement operators are used to extract potential line structure responses. These line structure enhancement operators can be Gabor guided filter banks, directional convolution kernels, Hessian line structure enhancement operators, Frangi filters, or other directional operators capable of enhancing line structures. The line structure enhancement operator can be denoted as... The directional response calculation can then be expressed as: , in, This represents the image after registration under the nth illumination condition; Indicates direction as Linear structure enhancement operator; This represents the response intensity of a pixel under the nth lighting condition and the kth direction.

[0032] To calculate directional stability, it is necessary to find the optimal orientation for each pixel. The optimal orientation can be determined based on the maximum response of the pixel among all orientations under different illumination conditions n, or by other criteria. This invention takes the maximum response as an example: The direction of maximum response corresponds to the k-direction corresponding to the maximum response under illumination condition n, denoted as . .

[0033] By combining the response results under all lighting conditions and all directions, a directional response tensor is constructed: , In the formula, K represents the number of directions, N represents the number of illuminated images, and H and W represent the image height and width, respectively. This orientation response tensor simultaneously includes orientation, illumination, and spatial dimensions, and is used to characterize the linear structural response of each pixel in the same detection region under different illumination conditions and orientations. This tensor provides a unified feature representation for subsequent calculations of response intensity stability, orientation stability, and structural continuity stability.

[0034] (iv) Step 4: Calculate the illumination perturbation consistency index map based on the directional response tensor. The illumination perturbation consistency index map is obtained by fusing response intensity stability, directional stability and structural continuity stability.

[0035] The uniformity index of illumination perturbation can be expressed as: C(x,y)=αS a (x,y)+βS b (x,y)+γS c (x,y). The illumination perturbation consistency index is used to characterize the stability of suspected defects under multiple illumination conditions. This index includes at least the following three stability dimensions.

[0036] First, the stability of the response intensity S a (x,y) response intensity stability is used to characterize the degree of fluctuation in the linear response intensity of the same pixel or local area under different lighting conditions: ,in, To prevent extremely small positive numbers with a denominator of zero, This represents the maximum directional response value of pixel (x, y) under the nth illumination condition. It has already been calculated in step three; The reference maximum response value of pixel (x,y) under all lighting conditions and all directions.

[0037] Response intensity stability is used to characterize the degree of fluctuation in the linear response intensity of the same pixel or local area under different lighting conditions. When a region maintains a relatively stable linear response under multiple lighting conditions, its response intensity stability is high; when a region only shows a strong response under a specific lighting condition, while the response is significantly weakened or disappears under other lighting conditions, its response intensity stability is low.

[0038] Second, directional stability S b (x,y) is used to characterize the consistency of the main response direction of the same pixel or local region under different lighting conditions: , in, This represents the principal response direction of pixel (x, y) under the nth illumination condition. The result has already been calculated in step three; This represents the reference principal direction or average principal direction of a pixel (x, y) under multiple illumination conditions.

[0039] The geometric orientation of genuine linear defects typically does not shift significantly with changes in illumination conditions, while the principal response direction of reflective pseudo-defects tends to change abruptly with variations in illumination angle. Therefore, by comparing the similarity of principal response directions under different illumination conditions, genuine linear defects can be enhanced while reflective pseudo-defects can be suppressed.

[0040] Third, structural continuity stability S c (x,y) is used to characterize the connectivity and continuity of the suspected linear structure in its local neighborhood: , in, It represents the local structural connectivity index of pixel (x,y) under the nth illumination condition.

[0041] The calculation involves: First, based on the maximum linear response intensity under the nth illumination condition... Perform threshold segmentation: ,in, This is the binary response diagram under the nth illumination condition; The response threshold under the nth illumination condition is determined based on experience and actual conditions; then, for By labeling connected components, several connected regions are obtained: .

[0042] If a pixel (x, y) belongs to a connected region, then the area of ​​the connected region containing that pixel is: .

[0043] To make images comparable across different lighting conditions, normalization can be performed using the area of ​​the largest connected component under the current lighting condition. Therefore, the local structural connectivity index of pixel (x, y) under the nth lighting condition can be expressed as: , in, The area of ​​the connected region containing the pixel. Let the maximum area of ​​the connected region be the nth illumination condition. To prevent extremely small positive numbers with a denominator of zero.

[0044] Real linear defects typically exhibit structures with a certain length and continuity, while random reflective noise, local bright spots, or isolated textures usually lack stable local connectivity. Therefore, by statistically analyzing the connectivity of linear structures within a local area, we can further distinguish between real defects and random pseudo-responses.

[0045] The weighting coefficients α, β, and γ are used to adjust the contribution of the three types of stability indices to the consistency index of illumination perturbation. These weighting coefficients can be set based on the reflectivity of the surface to be detected, the type of defect, the number of illumination conditions, and the sample calibration results. For the identification of linear defects on highly reflective surfaces, since real linear defects typically maintain relatively stable response intensity and main response direction under different illumination conditions, while reflective pseudo-defects are more prone to response intensity fluctuations and directional drift, the weights of response intensity stability and directional stability can be prioritized.

[0046] In this way, the present invention does not judge defects based solely on the grayscale difference or edge intensity of a single image, but rather judges their authenticity based on the stability of the response intensity, main direction, and structural continuity of the suspected defect under various lighting disturbances, thereby improving the reliability of high reflectivity surface defect identification.

[0047] (v) Step 5: Fuse the illumination perturbation consistency index map with the directional response intensity to generate a defect confidence map, and extract candidate defect regions based on the defect confidence map.

[0048] The specific process of generating the defect confidence map and extracting candidate defect regions is as follows: 1) Average the response intensity of the k-th direction under all illumination conditions to obtain the directional average response. .

[0049] 2) Extract the maximum value of the average response of the pixel in all directions, and use it as the average response intensity in the principal direction of the pixel: Where (x,y) represents a pixel. The average linear response intensity in all directions.

[0050] 3) The uniformity index of illumination disturbance C(x,y) and the average response intensity of the principal direction are compared. The initial defect confidence map is obtained by fusion: , in, C(x,y) represents the initial defect confidence level of pixel (x,y), and C(x,y) represents the illumination perturbation consistency index, which has already been calculated in this step. This is a response intensity adjustment parameter used to adjust the contribution of directional response intensity to the defect confidence map.

[0051] Response intensity adjustment parameters This can be determined through empirical pre-setting, sample calibration, or adaptive optimization. Specifically, labeled samples containing both real linear defects and reflective pseudo-defects can be selected, and different calculations can be performed respectively. The defect detection rate, false detection rate, and false negative rate are calculated under various values, and the value that optimizes the overall evaluation index is selected. As a parameter for adjusting the response intensity under the current detection conditions.

[0052] 4) Normalize the initial defect confidence map: , in, This represents the normalized defect confidence plot. and These represent the minimum and maximum values ​​in the initial defect confidence plot, respectively. To prevent extremely small positive numbers with a denominator of zero, normalization is performed. The range of values ​​for is (0, 1).

[0053] 5) Segment the defect confidence map according to the threshold T to obtain the candidate defect region mask: ,in, The candidate defect region is a mask, and T is the confidence threshold, which can be determined by a fixed threshold, an adaptive threshold, or a sample calibration threshold.

[0054] In this way, only regions that simultaneously satisfy high consistency of illumination perturbation and strong linear response can obtain high confidence in the defect confidence map, thereby suppressing unstable reflective pseudo-defects that change with illumination and enhancing real linear defects that exist stably under multiple illumination conditions.

[0055] (vi) Step 6: Refine the skeleton and construct the topological relationship of the candidate defect region to obtain the defect skeleton topology map.

[0056] Specifically, the candidate defect region is first morphologically cleaned to remove isolated small noise areas and repair local fracture areas. Then, skeleton thinning is performed on the candidate defect region to extract a single-pixel skeleton representing the center direction of the linear defect. Skeleton thinning can employ the Zhang-Suen thinning algorithm, median transform, distance transform skeleton extraction algorithm, or other skeleton extraction methods.

[0057] After obtaining the defect skeleton, the neighborhood connectivity of the skeleton pixels is statistically analyzed to identify skeleton endpoints, ordinary points, and bifurcation points. The distinction between skeleton endpoints, ordinary points, and bifurcation points is determined by counting the number of pixels in the eight-neighborhood of each skeleton pixel. , in, This represents the set of eight neighboring pixels of the skeleton pixel (p). This indicates the number of skeleton neighbors around the skeleton pixel.

[0058] Endpoints are used to characterize the start and end positions of linear defects, bifurcation points are used to characterize the intersection or split positions of linear defects, and ordinary points are used to form continuous skeleton paths. Endpoints and bifurcation points are used as a set of topological nodes, and the skeleton is divided into several independent skeleton segments with the endpoints or bifurcation points as boundaries. Each skeleton segment represents a suspected linear defect structure.

[0059] After obtaining the skeleton segments and topological nodes, a defect skeleton topology graph is constructed. The defect skeleton topology graph is represented as: G=(V,E,A), where V represents the set of topological nodes consisting of skeleton endpoints and branching points, E represents the set of connecting edges formed by skeleton segments, and A represents the set of skeleton segment attributes; the skeleton segment attributes include at least one of skeleton segment length, main direction, average width, continuity, and average illumination perturbation consistency confidence.

[0060] The skeleton segment length is calculated by summing the distances between all pixels in the skeleton segment. ,in, This represents the path length of the i-th skeleton segment. This represents the Euclidean distance between adjacent skeleton pixels.

[0061] The main direction of the skeleton segment is calculated using the coordinates of its first and last endpoints: ,in, Indicates the main direction of the (i)th skeleton segment, , )and( , ) represent the starting and ending coordinates of the skeleton segment, respectively.

[0062] The average width of the skeleton segment can be determined by bidirectionally searching for candidate defect regions in the local neighborhood along the direction perpendicular to the main direction, statistically analyzing the width near each skeleton point, and then calculating the average width of all skeleton points. : ,in, and These represent the forward search distance and the reverse search distance of the j-th skeleton point along the direction perpendicular to the main direction of the skeleton segment, respectively.

[0063] The continuity of a skeleton segment is determined based on the type of its connecting nodes. The continuity value for a skeleton segment is: , where K i For the continuity of the i-th skeleton, V e For the set of endpoints, V j For the set of bifurcation points, L min This is the minimum effective length.

[0064] The calculation process for the average illumination perturbation consistency information is as follows: ,in, This represents the average illumination perturbation consistency confidence level of the (i)th skeleton segment. Represents skeleton points The consistency index of illumination disturbance at the location.

[0065] By constructing a skeleton topology graph, this invention transforms the candidate defect region from a pixel-level mask into a topological representation with nodes, edges, and structural attributes, and obtains relevant information for each skeleton, providing a structured basis for subsequent defect type identification and grade evaluation.

[0066] (vii) Step 7: Extract the defect structure attributes based on the defect skeleton topology map and output the structured defect identification results.

[0067] Based on the skeleton topology map, the attributes of each skeleton segment or each structured defect object are extracted, and the defect type identification and risk level assessment are completed according to the attributes.

[0068] Defect attributes include, but are not limited to, defect length, defect direction, average width, average confidence, average consistency, continuity, curvature, number of bifurcations, number of endpoints, connectivity, and weight of the region.

[0069] Based on the numerical values ​​of the aforementioned defect attributes, defect classification and grading rules can be established to categorize defects into continuous groove defects, short scratch defects, isolated spot defects, complex defects, or other preset types, and output the corresponding risk level. The risk level can be determined based on a combination of one or more of the following: defect length, width, consistency confidence of illumination disturbance, continuity, and region weight.

[0070] Example 1

[0071] Figure 1 This diagram illustrates a first preferred embodiment of the method of the present invention. This embodiment focuses on the identification of linear defects in polished metal surfaces, optical surfaces, or glossy injection-molded surfaces. Linear defects include, but are not limited to, scratches, grooves, cracks, and residual polishing marks. Since highly reflective surfaces are prone to strong specular reflections, bright spots, shadow boundaries, and localized reflective artifacts under different lighting conditions, this embodiment distinguishes between real linear defects and reflective pseudo-defects through consistent multi-lighting perturbation modeling, and further achieves a structured understanding of the defects through a skeleton topology diagram.

[0072] Figure 2This is a diagram of a multi-light imaging device. In a fixed inspection position, the highly reflective curved surface to be inspected is placed within the camera's field of view, maintaining a constant relative position between the camera and the surface. Multiple light sources are arranged around the same inspection area, including one or more of the following: left-side oblique incidence light source, right-side oblique incidence light source, ring light source, and polarized light source.

[0073] Multi-light images are stored in the same image folder according to lighting conditions or file names, and common image format files, including jpg, jpeg, png, bmp, or tif, are read. Color images are converted to grayscale images during reading to obtain a collection of multi-light images in grayscale format.

[0074] like Figure 3 The diagram illustrates the image preprocessing and spatial registration process. Preprocessing and spatial registration are performed on a set of images under multiple illuminations to obtain a registered image set. In this embodiment, the first image is used as the reference image. First, the reference image is denoised, then brightness normalization is performed to obtain a normalized reference image. In one specific implementation, the denoising process uses the BM3D denoising method, and the brightness normalization process uses an adaptive histogram equalization method.

[0075] During spatial registration, feature points are extracted from the reference image and the image to be registered, feature matching is performed, and the affine transformation matrix is ​​estimated based on the matching points. Subsequently, the affine transformation matrix is ​​used to perform a geometric transformation on the image to be registered, aligning it with the reference image in spatial coordinates. The registration process can be represented as: , Among them, T n Let n represent the spatial transformation function of the nth image relative to the reference image.

[0076] In one specific implementation, image feature points are extracted using the ORB feature point detection method, feature matching relationships are obtained using the Hamming distance matching method, the transformation matrix is ​​estimated using affine transformation, and the registered image is obtained using interpolation. Through this process, the same pixel coordinates in a multi-illumination image set correspond to the same physical surface location, providing a basis for subsequent comparisons of the responses of the same pixel under different illumination conditions.

[0077] Multi-directional linear structure response extraction is performed on the registered image set. In this embodiment, eight linear structure enhancement operators are set. Each direction corresponds to a directional filtering kernel. As a specific implementation, the directional filtering kernel can be a Gabor filter kernel. It should be noted that the Gabor filter is only one specific implementation in this embodiment, and the present invention does not limit the directional response extraction to Gabor filters; directional convolution kernels, Hessian linear structure enhancement operators, Frangi filters, or other linear structure enhancement operators can also be used. The image is filtered in eight directions to obtain the directional response, and then the directional response results of all directions and all illumination conditions are combined to obtain the directional response tensor. For the nth registered image, its Kth direction... The linear structure response is represented as follows: The directional response tensor contains directional, illumination, and spatial dimensions, and is used to fully express the linear structural response of the same detection area under different illuminations and directions.

[0078] This embodiment calculates the illumination perturbation consistency index from three dimensions: response intensity stability, directional stability, and structural continuity stability. Figure 4 As shown.

[0079] The response intensity stability calculation process includes: calculating the maximum response of a pixel under all illumination and all directions, and then calculating the maximum directional response under the nth illumination condition, thus determining the response intensity stability. Defined as: , in, To prevent extremely small positive numbers with a denominator of zero.

[0080] When a pixel exhibits a stable linear response under multiple lighting conditions Larger; when the pixel shows a strong response only under a certain lighting condition and a weak response under other lighting conditions. The value is relatively small. The directional stability calculation process includes calculating the principal response direction index of a pixel for a given direction, and the reference principal direction of the same pixel under all illumination conditions. Directional stability is defined as: When real linear defects maintain similar principal directions under different lighting conditions, The magnitude is relatively large; when the reflective pseudo-defect changes with the direction of illumination, causing a jump in the principal direction, Smaller.

[0081] The structural stability calculation process includes: obtaining the maximum directional response map under a certain illumination condition; then, thresholding the obtained response map to obtain a binary response map; subsequently, performing connected component analysis on the binary response map to obtain the area of ​​each connected component; and for the connected region containing the pixel (x, y), its normalized connectivity index is defined as: ,in, Represents pixels The area of ​​the connected region, It represents the maximum area of ​​all connected regions under a certain lighting condition.

[0082] Structural continuity stability Defined as: When a certain region exhibits a continuous linear structure under multiple lighting conditions, Larger; when the area exhibits random noise, isolated bright spots, or discontinuous texture, Smaller.

[0083] Based on the above three stability indices, calculate the illumination disturbance consistency index: In specific implementation cases, take Subsequently, on After normalization, a uniformity index of illumination perturbation in the range of [0,1] is obtained.

[0084] Using the above method, regions with stable response intensity, stable direction, and continuous structure obtain higher consistency indices; unstable regions caused only by local reflections, noise, or changes in illumination obtain lower consistency indices. (Consistency index based on illumination perturbation) Using the directional response tensor, a defect confidence map is generated.

[0085] The specific defect confidence map calculation first involves calculating the average response in the k-th direction under all illumination conditions. Then, the maximum value of the average response of each pixel in all directions is calculated. By fusing the illumination perturbation consistency index with the average response in the maximum direction, a defect confidence map is obtained: , in, This is a response intensity adjustment parameter. In one specific implementation, it is taken as... .

[0086] Then on After normalization, candidate defect regions are extracted based on the confidence threshold T: , in, This serves as a mask for the candidate defect region. In one specific implementation, T = 0.5.

[0087] A skeleton topology map is constructed for the candidate defect region mask M(x,y) to convert the pixel-level candidate region into a structured defect representation. Specifically, the defect region is first morphologically processed and the skeleton is refined to obtain a single-pixel skeleton map. Examples of skeleton topology diagrams and skeleton refinement results are as follows: Figure 5 As shown. Then, the skeleton diagram... Neighborhood analysis is performed on each skeleton pixel. For a skeleton pixel (p), its neighborhood is statistically analyzed. Number of skeleton pixels in the neighborhood .when When = 2, it means that the skeleton pixel has only one skeleton neighbor besides itself, and it is determined as an endpoint; when When the value is 0, it indicates that the skeleton pixel has at least three skeleton neighbors besides itself, and it is determined as a branch point; the remaining skeleton pixels are ordinary skeleton points.

[0088] The endpoints and bifurcation points are temporarily removed from the skeleton graph to obtain a skeleton segment graph. Connectivity analysis is then performed on the remaining skeleton pixels to obtain several skeleton segments. Each skeleton segment consists of a set of continuous skeleton pixel coordinates.

[0089] Thus, the candidate defect region is transformed from the original pixel-level mask into a topological graph structure containing node, edge, and skeleton segment attributes. This structure can characterize the endpoints, bifurcation points, connectivity, breakage, and continuity of the defect.

[0090] Based on the skeleton segment set S and the skeleton topology graph G, defect attributes are extracted for each skeleton segment, and structured defect results are output. For the i-th skeleton segment... Its length The path length of the skeleton segment is calculated as the sum of the distances between all pixels in the skeleton segment. ,in, This represents the path length of the i-th skeleton segment. This represents the Euclidean distance between adjacent skeleton pixels.

[0091] The main direction of the skeleton segment is obtained by calculating the coordinates of the first and last endpoints. : ,in, Indicates the main direction of the i-th skeleton segment, ( , )and( , ) represent the starting and ending coordinates of the skeleton segment, respectively.

[0092] To determine the average width of the skeleton segment, a bidirectional search for candidate defect regions can be performed within the local neighborhood along the direction perpendicular to the main direction. The width near each skeleton point is then statistically analyzed, and finally, the average width of all skeleton points is calculated. : ,in, and These represent the forward search distance and the reverse search distance of the j-th skeleton point along the direction perpendicular to the main direction of the skeleton segment, respectively.

[0093] To determine the consistency confidence of the average illumination perturbation of a skeleton segment, read the values ​​of all skeleton pixels in that segment in the consistency index map and calculate the average value. : ,in, This represents the confidence level of the average illumination perturbation consistency for the i-th skeleton segment. Represents skeleton points The consistency index of illumination disturbance at the location.

[0094] The continuity of a skeleton segment is determined based on the type of its connecting nodes. The continuity values ​​for a skeleton segment are: , Among them, K i For the continuity of the i-th skeleton, V e For the set of endpoints, V j For the set of bifurcation points, L min This is the minimum effective length.

[0095] In one specific implementation, the hierarchical rules are established as follows: First, filter skeleton segments whose average confidence score is below a threshold. Set the confidence filtering threshold to [value]. ,when When this happens, the skeleton segment is identified as a low-confidence pseudo-defect and filtered out. In one specific implementation, the following is taken: .

[0096] Subsequently, defect type identification and grade assessment were performed based on skeleton segment length, continuity, and average confidence level. Specifically: when And K i When =1, the skeleton segment is identified as a groove-type defect; if If the risk level is high, then the risk level is determined to be high; if If the risk level is positive, the risk level is determined to be medium; otherwise, it is determined to be low.

[0097] when and At that time, the skeleton segment was identified as a short scratch-type defect; if If the risk level is positive, the risk level is determined to be medium; otherwise, it is determined to be low.

[0098] when At that time, the skeleton segment is identified as an isolated point or spot-type defect, and the risk level is determined to be low.

[0099] The remaining cases can be identified as complex defects, and the risk level is determined to be medium.

[0100] Finally, a structured defect result table is output, which includes at least the defect number, defect type, defect length, defect direction, consistency confidence level, and risk level.

[0101] In one specific implementation, the structured results described above are saved as a table file for easy subsequent quality traceability, defect statistics, and visual annotation. The results of this implementation case are shown in the table below.

[0102]

[0103] This embodiment demonstrates how to distinguish between real linear defects and reflective pseudo-defects by utilizing the response stability of the same detection area under different illumination perturbation conditions. Compared to single image thresholding or single edge detection methods, this embodiment does not directly determine defects based on grayscale abrupt changes. Instead, it calculates response intensity stability, directional stability, and structural continuity stability using the directional response tensor, and generates an illumination perturbation consistency index and a defect confidence map accordingly.

[0104] Meanwhile, this embodiment further converts the candidate defect region into a skeleton topology map, and completes defect type identification and risk level assessment through attributes such as the length, direction, width, continuity and consistency confidence of the skeleton segments, realizing the transformation from pixel-level defect segmentation to structured defect understanding.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying surface defects based on the consistency of illumination perturbation, characterized in that, Includes the following steps: Step 1: Obtain a set of multi-illumination images of the same highly reflective curved surface detection area under multiple illumination conditions; Step 2: Preprocess and spatially register the multi-illumination image set to obtain the registered multi-illumination image set; Step 3: Perform multi-directional linear structure response calculations on each registered illumination image to construct the directional response tensor; The directional response tensor is constructed as follows: Let N be the number of illuminated images, K be the number of directions, H be the image height, and W be the image width. For the registered image I′n(x,y) under the nth illumination condition, the linear structure enhancement operator in the kth direction is used. Calculate the directional response: ,in, This represents the linear structural response intensity of pixel (x, y) under the nth illumination condition and the kth direction; By combining the directional responses under all lighting conditions and all directions, we obtain the directional response tensor: , Where R represents the orientation response tensor, K represents the number of orientations, N represents the number of illumination images, and H and W represent the image height and image width, respectively; Step 4: Calculate the illumination perturbation consistency index map based on the directional response tensor. The illumination perturbation consistency index map is obtained by fusing response intensity stability, directional stability and structural continuity stability. Step 5: Fuse the illumination perturbation consistency index map with the directional response intensity to generate a defect confidence map, and extract candidate defect regions based on the defect confidence map; Step 6: Refine the skeleton and construct the topological relationship of the candidate defect region to obtain the defect skeleton topology map; Step 7: Extract the defect structure attributes based on the defect skeleton topology map and output the structured defect identification results.

2. The surface defect identification method based on illumination perturbation consistency according to claim 1, characterized in that, In step 2, the preprocessing includes at least one of grayscale processing, noise reduction processing, brightness normalization processing, contrast enhancement processing, and geometric distortion correction processing; the spatial registration adopts feature point matching, phase correlation, affine transformation, perspective transformation, or sub-pixel level registration method, and the registration process is as follows: select an image as a reference image, extract image feature points of the reference image and the image to be registered, match the image feature points, estimate the spatial transformation matrix according to the matching result, and perform interpolation transformation on the image to be registered according to the spatial transformation matrix.

3. The surface defect identification method based on illumination perturbation consistency according to claim 1, characterized in that, In step 4, the illumination disturbance consistency index is calculated according to the following formula: C(x,y)=αS a (x,y)+βS b (x,y)+γS c (x,y), Where C(x,y) represents the illumination perturbation consistency index at pixel (x,y), and S a (x,y) represents the response intensity stability, used to characterize the degree of fluctuation in the linear response intensity of the same pixel or local area under different lighting conditions; S b (x,y) represents directional stability, used to characterize the consistency of the main response direction of the same pixel or local region under different lighting conditions; S c (x,y) represents the structural continuity stability, used to characterize the connectivity and continuity of the suspected linear structure in the local neighborhood; α, β, and γ are the weighting coefficients, respectively.

4. The surface defect identification method based on illumination perturbation consistency according to claim 3, characterized in that, The response intensity stability is obtained by comparing the relative deviation between the maximum directional response value under the nth illumination condition and the reference maximum response value under all illumination conditions and all directions; the directional stability is obtained by comparing the degree of consistency between the principal response direction and the reference principal direction under different illumination conditions. The structural continuity stability is obtained by thresholding and connecting component analysis of the maximum directional response maps under different lighting conditions, and calculating the local structural connectivity of the connected regions where the pixels are located.

5. The surface defect identification method based on illumination perturbation consistency according to claim 1, characterized in that, In step 5, the defect confidence map is generated as follows: First, the directional response intensity of the k-th direction under all illumination conditions is averaged to obtain the directional average response. ; Then, the maximum value of pixel (x, y) in the average response across all directions is extracted to obtain the average response intensity in the principal direction. ; Finally, the average response intensity in the principal direction is... By fusing the illumination perturbation consistency index C(x,y), an initial defect confidence map is obtained: ,in, λ represents the initial defect confidence of pixel (x,y), and λ represents the response intensity adjustment parameter.

6. The surface defect identification method based on illumination perturbation consistency according to claim 5, characterized in that, The initial defect confidence map is normalized to obtain a defect confidence map P(x,y). The defect confidence map P(x,y) is then segmented according to a confidence threshold T to obtain a candidate defect region mask M(x,y). 。 7. The surface defect identification method based on illumination perturbation consistency according to claim 1, characterized in that, In step 6, the defect skeleton topology map is constructed as follows: 1) Perform morphological cleaning on the candidate defect area to remove isolated small noise areas and repair local fracture areas; 2) Refine the skeleton of the candidate defect region and extract the single-pixel skeleton map to characterize the direction of the center of the linear defect; 3) Count the number of eight-neighbor skeleton adjacencies of skeleton pixels, and identify skeleton endpoints, ordinary points and branching points based on the number of eight-neighbor skeleton adjacencies; 4) Use the skeleton endpoints and bifurcation points as topological nodes, and use the skeleton endpoints or bifurcation points as boundaries to divide the skeleton into several independent skeleton segments. Each skeleton segment represents a suspected linear defect structure. 5) Construct a defect skeleton topology diagram based on the topology nodes and the independent skeleton segments.

8. The surface defect identification method based on illumination perturbation consistency according to claim 7, characterized in that, The defect skeleton topology is represented as: G=(V,E,A), where V represents the set of topological nodes consisting of skeleton endpoints and bifurcation points, E represents the set of connecting edges formed by skeleton segments, and A represents the set of skeleton segment attributes; the skeleton segment attributes include at least one of skeleton segment length, main direction, average width, continuity, and average illumination perturbation consistency confidence.

9. The surface defect identification method based on illumination perturbation consistency according to claim 1, characterized in that, In step 7, the defect structural attributes include at least one of defect length, defect direction, average width, average confidence level, average consistency, continuity, curvature, number of bifurcations, number of endpoints, and connection relationship; the structured defect identification result includes at least one of defect location, defect type, defect length, defect direction, defect width, confidence level, and risk level.

Images