A machine vision method and system for non-standard product defect image processing

Abstract

The application relates to the field of image data processing and discloses a machine vision method and system for non-standard product defect image processing, which comprises the following steps: according to an initial concave-convex feature map, a glossy reflection separation process is carried out, a shadow area compensation technology is combined, texture detail distribution is extracted, and a surface texture map after removing light interference is generated; according to a smoothed texture distribution map, a local texture smoothing and defect edge reservation technology is used to separate out a suspected defect area and determine a potential defect point distribution map; according to a defect confirmation map, high-precision line laser scanning is adopted to obtain three-dimensional detail data of a high-risk area, a texture detail extraction result is combined, and defect depth and morphological feature distribution are determined; through the defect depth and morphological feature distribution, an adaptive focal length imaging technology is adopted to carry out image acquisition on different depth areas, and a post-filtering noise control technology is fused to obtain a clear defect image of the whole area.

Description

Technical Field

[0001] This invention relates to the field of image data processing, and in particular to a machine vision method and system for processing defective images of non-standard products. Background Technology

[0002] In the industrial manufacturing sector, surface defect detection of irregularly shaped metal parts is a crucial technology, directly impacting product quality and production safety. This is especially true in high-precision industries such as aerospace and automotive, where even minute defects can lead to serious consequences. Research in this field not only improves production efficiency but also represents the ultimate pursuit of technical precision, bearing the heavy responsibility of ensuring product reliability.

[0003] However, some existing testing methods often fall short when faced with complex environments. Many traditional methods struggle to adapt to the diverse shapes and complex surface characteristics of irregularly shaped metal parts, especially when multiple factors such as light reflection, texture interference, and spatial constraints are combined, often resulting in biased test results or even failing to accurately identify key issues. This limitation does not stem solely from the inadequacy of the technology itself, but rather from the inability to effectively address the challenges posed by the interaction of multiple environmental variables.

[0004] Focusing on specific technical challenges, the high reflectivity of irregularly shaped metal parts becomes a core issue. Because metal surfaces easily produce strong specular reflections, light at different angles can create halos or shadows. These phenomena can mask actual micro-defects on the surface, such as cracks or pores, making it difficult for inspection equipment to distinguish genuine flaws. At a deeper level, this high reflectivity characteristic is also closely related to the complex geometry of the part's surface, such as curved surfaces and chamfers. Changes in shape make the light distribution more uneven, further increasing the difficulty of imaging and forming the core contradiction in inspection.

[0005] Taking a specific business scenario as an example, when inspecting a metal part with deep holes and multiple curved surfaces, the high reflectivity of the surface makes it almost impossible for light to illuminate the inside of the deep holes evenly. The resulting image is either an overexposed spot or a completely invisible shadow area, where the real defects are often hidden. Even adjusting the angle of the light source cannot completely solve the problem of uneven light distribution, and the inspection equipment often misses critical flaws as a result, affecting subsequent assembly or safe use.

[0006] Therefore, how to accurately capture the real defects on the surface of irregularly shaped metal parts under the dual interference of high reflectivity and complex geometry has become a key problem that this study urgently needs to overcome. Summary of the Invention

[0007] The purpose of this invention is to provide a machine vision method and system for processing defect images of non-standard products, in order to solve the technical problems of reflection interference, misjudgment of periodic tool marks, and identification of defects in blind areas in narrow spaces on the surface of complex metal parts under multiple illumination conditions.

[0008] The technical solution of the present invention is as follows:

[0009] This invention provides a machine vision method for processing defect images of non-standard products, mainly including:

[0010] By illuminating the surface of an irregularly shaped metal part with multiple angle light sources, an image sequence is obtained, the surface normal vector is calculated, a reflection model is constructed, and the surface height information is reconstructed to obtain an initial concave-convex feature map.

[0011] Based on the initial bump feature map, gloss reflection separation and shadow area compensation are performed to extract texture details and generate a surface texture map after removing lighting interference.

[0012] Frequency domain feature analysis is performed on the surface texture map to identify periodic patterns. When periodic tool marks are detected, frequency suppression processing is performed to obtain a smoothed texture distribution map.

[0013] Based on the smoothed texture distribution map, local texture smoothing and defect edge preservation techniques are used to separate suspected defect areas and determine the distribution map of potential defect points.

[0014] Based on the distribution map of potential defect points, and combined with the surface height reconstruction data, the potential defect points are compared with three-dimensional geometric features. Those that match the geometric characteristics of the real defects are marked as high-risk areas, and a defect confirmation map is obtained.

[0015] Based on the defect confirmation map, line laser scanning is performed on high-risk areas to obtain three-dimensional detailed data and determine the distribution of defect depth and morphological features.

[0016] By analyzing the distribution of defect depth and morphological features, adaptive focal length imaging is used to acquire images of different depth regions, and the images are then fused and filtered to obtain a clear defect image of the entire region.

[0017] Based on clear defect images of the entire area, multi-path image acquisition is performed for blind spots in narrow spaces. Combined with periodic pattern recognition results, potential risk points in the blind spots are identified, and a final defect distribution report is generated.

[0018] This invention provides a machine vision system for processing defect images of non-standard products, mainly comprising:

[0019] The image sequence acquisition module is used to acquire image sequences under different lighting conditions by illuminating the surface of an irregularly shaped metal part with multi-angle light sources.

[0020] The surface height reconstruction module is used to calculate the surface normal vector and construct the reflection model to reconstruct the surface height information and obtain the initial concave and convex feature map.

[0021] The texture extraction module is used to perform gloss reflection separation and shadow area compensation based on the initial concave-convex feature map, extract texture details, and generate a surface texture map after removing lighting interference.

[0022] The periodic pattern processing module is used to perform frequency domain feature analysis on the surface texture map, identify periodic patterns, and perform frequency suppression processing when periodic tool marks are detected to obtain a smoothed texture distribution map.

[0023] The defect region separation module is used to separate suspected defect regions and determine the distribution map of potential defect points based on the smoothed texture distribution map and using local texture smoothing and defect edge preservation techniques.

[0024] The defect confirmation module is used to compare the three-dimensional geometric features of potential defect points with surface height reconstruction data based on the distribution map of potential defect points. Points that match the geometric characteristics of real defects are marked as high-risk areas, thus obtaining a defect confirmation map.

[0025] The 3D detail acquisition module is used to perform line laser scanning on high-risk areas based on the defect confirmation map to acquire 3D detail data and determine the depth and morphological feature distribution of defects.

[0026] The defect image generation module is used to acquire images of different depth regions by using adaptive focal length imaging based on the distribution of defect depth and morphological features, and then fuse and filter them to obtain a clear defect image of the entire region. Based on the clear defect image of the entire region, multi-path image acquisition is performed for blind areas in narrow spaces. Combined with the results of periodic pattern recognition, potential risk points in the blind areas are identified, and a final defect distribution report is generated.

[0027] The beneficial effects of this application are as follows: A machine vision method and system for processing defect images of non-standard products, in use, acquires image sequences through multi-angle light source illumination, combines surface normal vector calculation and reflection model construction to initially reconstruct surface height information and generate an initial concave-convex feature map, then extracts texture details through gloss and reflection separation and shadow compensation techniques, and uses frequency domain analysis and filter kernel adjustment to suppress tool mark interference, obtaining a smooth texture map. This invention further utilizes local texture smoothing and three-dimensional geometric feature comparison to accurately separate and confirm defect areas, combines high-precision line laser scanning and adaptive focal length imaging to acquire clear defect images of the entire area, and finally solves the problem of identifying potential risk points in blind areas through multi-path image acquisition and pattern recognition. This invention significantly improves the accuracy and comprehensiveness of surface defect detection for complex metal parts, especially in terms of light interference suppression and blind area analysis, providing an efficient and reliable solution for industrial inspection. Attached Figure Description

[0028] Figure 1 This is a flowchart of a machine vision method for processing defect images of non-standard products according to the present invention; Figure 2 This is a schematic diagram of the process for determining the distribution map of concave and convex features in a machine vision method for processing defective images of non-standard products according to the present invention. Figure 3 This is a flowchart illustrating the process of determining the optimized texture detail map in a machine vision method for processing defective images of non-standard products according to the present invention. Figure 4 This is a schematic diagram of the structure of a machine vision system for processing defect images of non-standard products according to the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described embodiments are merely some embodiments of the invention, and not all embodiments. The components of the embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0030] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0031] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0032] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0033] A specific embodiment of the machine vision method and system for processing defect images of non-standard products according to the present invention:

[0034] like Figures 1-3 As shown, the machine vision method for processing defect images of non-standard products in this embodiment may specifically include:

[0035] Step S101: Illuminate the surface of the irregular metal part with multi-angle light sources to obtain image sequences under different lighting conditions. Use the surface normal vector calculation method combined with the light intensity distribution data to construct a reflection model, initially reconstruct the surface height information, and obtain the initial concave-convex feature map.

[0036] The irregularly shaped metal part is surface-illuminated by multiple light sources to obtain image sequences under different lighting conditions, which are stored as the original image dataset. The original image dataset is processed using a surface illumination method, and combined with changes in lighting conditions, the light intensity value of each image is extracted to generate a light intensity distribution matrix. In one possible implementation, a photometric stereo vision configuration is first used to sequentially illuminate light sources from different directions, capturing N grayscale images with identical viewing angles but different lighting directions, with an image resolution of W×H. The acquired images are radiometrically calibrated, linearly mapping the grayscale values ​​to relative light intensity. Then, each image is treated as a W×H grayscale matrix, and the grayscale values ​​of the N images are arranged along the third dimension at the same coordinates (x, y) to construct a W×H×N three-dimensional light intensity distribution matrix; alternatively, an N×1 light intensity observation vector is extracted for each pixel. This is used for subsequent surface normal and height reconstruction.

[0037] For the illumination intensity distribution matrix, the surface normal vector is calculated, and the corresponding reflection model is constructed to obtain the set of surface reflection parameters. In one possible implementation, firstly, the reflection equation is constructed based on the Lambert reflection model: ,in To observe grayscale values, For surface albedo, For light source intensity, For the surface normal vector, Let be the direction vector of the light source. To simplify calculations, the albedo and normal vector are combined into . And let the calibrated light source vector Then the reflection model simplifies to ,in, Indicates the first The luminous intensity of each light source; For the first The observed grayscale values ​​under each light source. For each pixel, using... The observation vector is composed of gray values ​​under different lighting conditions. ,by The light source matrix is ​​composed of several light source vectors. .when When the light sources are not coplanar, the solution is obtained using the least squares method. Solve Then, calculate The surface albedo is obtained from the modulus length. and will Normalization yields the surface normal vector This yields a set of surface reflection parameters, including surface albedo and surface normal vectors. The above calculation is performed pixel-by-pixel to obtain the full-image normal vector field. Albedo map .

[0038] Based on the set of surface reflection parameters and the reconstruction algorithm, a preliminary surface height map is calculated to generate initial height distribution data. In one possible implementation, the surface normal vector for each pixel is calculated based on the set of surface reflection parameters. Then calculate the surface along the normal vector. and Gradient field of partial derivatives in the direction: , For the gradient matrix , Performing Fast Fourier Transform on each yields the following results: , Substituting into the frequency domain integral formula:

[0039]

[0040] in, , , They are , , The Fourier transform result, , For frequency domain coordinates, The unit is the imaginary unit. Calculate the frequency domain height. The spatial domain height map is then obtained through inverse fast Fourier transform. The output is a single-channel floating-point grayscale matrix with the same resolution as the original image, where each element represents the relative physical height of the corresponding pixel.

[0041] Initial height distribution data is acquired, the data distribution pattern is analyzed, and outlier areas in the surface height map are identified and marked as areas requiring correction. Specifically, the probability distribution of height values ​​across the entire map is statistically analyzed using a height histogram to identify isolated outliers at both ends; simultaneously, the gradient magnitude of the height map is calculated. Analyze its spatial distribution. Based on The principle is to calculate the mean of the gradient magnitude across the entire image. and standard deviation , will satisfy Pixels whose height values ​​exceed the preset physical range are identified as height abrupt change anomalies. Connected component marking is applied to these anomalies to form closed anomaly regions, generating a mask matrix where anomaly regions are assigned a value of 1 and normal regions are assigned a value of 0, thus determining the data for the region to be corrected.

[0042] By combining the data of the region to be corrected with the initial convexity map, the surface height map is locally adjusted to generate the final convexity feature distribution map. Specifically, based on the mask corresponding to the data of the region to be corrected, abnormal regions in the initial height map are erased; using the normal height and gradient data of the edges of the abnormal regions as boundary conditions, bicubic interpolation or thin-plate spline functions are used for macroscopic surface fitting to obtain the reference bottom surface. Then extract the micro-texture details of the initial bump map within the corresponding area. ,according to By fusing microscopic convex and concave features and smoothing the repaired boundaries using Gaussian mixture filtering, a final convex and concave feature distribution map is generated. Used to adjust the blending strength of micro-details This represents the final height value after local adjustments. The final convexity / concave feature distribution map is stored in a structured data format, and a complete dataset of surface features is output.

[0043] Step S102: Based on the initial bump feature map, perform gloss reflection separation processing, combine shadow area compensation technology, extract texture detail distribution, and generate a surface texture map after removing illumination interference.

[0044] From the input concave-convex feature map, image decomposition techniques are used to separate the gloss reflection component. Preliminary segmentation of the reflection regions yields the base image after reflection separation. In one possible implementation, a dual-color reflection model is applied to the input concave-convex feature map, decomposing the total reflected light intensity into diffuse and gloss reflection components. The image is converted from RGB to HSV space, and the specular reflection coefficient within the local window is calculated based on the minimum saturation prior. Guided filtering is used to separate the gloss reflection component, resulting in a gloss reflection component map. The Otsu algorithm is used to calculate a segmentation threshold for this component map. Pixels exceeding the threshold are marked as reflection regions, and a morphological dilation operation with a kernel size of 3×3 is performed to expand the boundaries of the reflection regions, generating a binary mask for the reflection regions, thus obtaining the base image after removing gloss reflection interference.

[0045] For the base image after reflection separation, shadow region compensation data is acquired. Brightness in dark areas is adjusted using a preset shadow compensation technique to determine the intermediate image after shadow compensation. In one possible implementation, for the base image after reflection separation, a shadow mask is extracted by setting a low brightness threshold, and a large-kernel Gaussian filter is used to extract the background illuminance map as shadow region compensation data. Brightness adjustment is performed based on adaptive gamma correction, and the correction index for each pixel is dynamically calculated. ,in, As a control factor, the darker the pixel... The smaller the value, the better. Perform pixel-level non-linear remapping on dark areas. Then, the local area after compensation is subjected to contrast-limited adaptive histogram equalization, and the intermediate image after shadow compensation is output.

[0046] Based on the shadow-compensated intermediate image, surface texture details are extracted, and gradient analysis is used to enhance the distribution of texture details, identifying the sharp areas of texture details. In one possible implementation, bandpass filtering is used to extract surface texture details from the shadow-compensated intermediate image: the intermediate image is smoothed using a Gaussian filter with a small kernel to obtain... Subtracting the original image from the high-frequency components yields the high-frequency components. Then, using gradient analysis, taking the Scharr operator as an example, we calculate... , First-order partial derivatives in the direction , The comprehensive gradient magnitude matrix is ​​obtained. Multiply the gradient magnitude by the gain factor Then overlay it onto the texture details: By setting Sliding window calculation of local standard deviation ,Will Greater than the resolution baseline threshold The regions are identified and marked as areas with clear texture details. The intermediate image after shadow compensation in pixel coordinates The grayscale value at that location; The image obtained after Gaussian smoothing the intermediate image is in The grayscale value at that location; The extracted surface texture details (high-frequency components) represent features such as micro-defects, scratches, and tool marks. This is the gain coefficient, and its value typically ranges from 0.5 to 1.5. To enhance the texture details after processing, the original texture details are added to the weighted gradient magnitude. This refers to the size of the sliding window.

[0047] If lighting interference exists in areas of sharp texture detail, local contrast adjustment techniques are used to remove the lighting effect, resulting in a texture image with lighting interference eliminated. Specifically, for local pixels with lighting interference, the average value of their neighborhood is used... and standard deviation Combined with the preset target mean relative to target standard deviation Pixel remapping is performed using the following formula to obtain the texture image after illumination interference removal. :

[0048]

[0049] in, The enhanced image pixel values; A coefficient to limit noise amplification, used to suppress excessive noise enhancement, with a value ranging from 0.2 to 2.0, preferably... .

[0050] For the texture image after illumination interference removal, the surface feature distribution is reconstructed to obtain a complete mapping of texture detail distribution, thus determining the final surface texture distribution map. In one possible implementation, the two-dimensional image pixel coordinates are first established using camera intrinsic parameters and pre-reconstructed 3D macroscopic height map data. Three-dimensional physical space coordinates of the workpiece surface The geometric mapping relationship between them is used to perform coordinate transformation and geometric distortion correction; secondly, a bilinear interpolation method is used to transform the texture image after eliminating illumination interference. As a texture map, it is mapped to the corresponding surface position of the 3D curved surface model; finally, a multi-channel feature matrix is ​​constructed, in which each feature point contains the mapped texture gray value and its corresponding physical space coordinates, so as to obtain a surface texture distribution map without lighting interference and eliminating surface distortion in the 3D unfolded coordinate system as the final output.

[0051] Based on the final surface texture distribution map, the surface feature reconstruction is refined by fusing the results of the initial feature extraction to obtain an optimized texture detail map. Specifically, a multi-scale, multi-channel feature pyramid fusion method is used, with the final concave-convex feature distribution map extracted from the initial feature extraction as the bottom spatial reference and the final surface texture distribution map as the high-frequency detail layer. Pixel-level weighted superposition is performed on the corresponding frequency bands using Laplacian pyramid decomposition. Then, a refinement process is performed: the high-frequency texture matrix and the concave-convex gradient map are multiplied by their spatial positions. Only when the same location appears as a high-contrast line segment on the texture map and as a microscopic groove on the concave-convex feature map is it identified as a real physical defect, and false textures are filtered out. A non-maximum suppression algorithm is used to find local peak points along the gradient direction, refining wider textures into precise skeletons of single-pixel width. Morphological closing operations and broken-line connection algorithms are used to geometrically close and stitch together the tiny breaks in long strip defects. The final output is an optimized texture detail map with sharp edges, free from illumination interference and false positive noise.

[0052] Step S103: For the surface texture map, frequency domain feature analysis and directional texture separation method are used to identify whether there is a periodic pattern. If periodic tool marks are detected, the texture distribution map is obtained by adjusting the filter kernel parameters and tool mark frequency suppression technology.

[0053] Initial data acquisition of the surface texture map is performed, and image preprocessing techniques are used to remove noise interference, resulting in a preliminarily cleaned texture image. In one possible implementation, bilateral filtering is used for denoising: a sliding window is set, and the spatial Gaussian weight and gray-level Gaussian weight of each pixel within the window relative to the center pixel are calculated respectively; the gray-level Gaussian weight is determined based on the gray-level difference between pixels, and when the gray-level difference exceeds a preset threshold, the weight approaches zero to preserve edge features such as periodic knife marks; the two sets of weights are combined to perform a weighted average on the pixels within the window to reconstruct the image; after the above processing, randomly distributed Gaussian noise and salt-and-pepper noise are smoothly removed, while directional and continuous knife mark edges are completely preserved, thus obtaining a preliminarily cleaned texture image.

[0054] Based on the initially cleaned texture image, Fourier transform is used to extract frequency features, analyze the presence of periodic patterns, and identify significant periodic signals in the frequency domain. In one possible implementation, a two-dimensional discrete Fourier transform is first performed on the initially cleaned texture image to transform it from the spatial domain to the frequency domain, and the spectrum is centered to calculate the amplitude spectrum as a frequency feature. Next, integral projection analysis is performed on the spectrum along different radial angles to calculate the total energy distribution at each angle. If prominent energy peaks appear at specific angles and frequency radii, a periodic pattern is identified. Finally, a signal-to-noise ratio threshold method is used to determine significant periodic signals: the average energy of the entire spectrum excluding the DC component is calculated. and standard deviation Detecting all local energy maxima will satisfy The spectral points were determined to be significant periodic signals, among which... The significance factor is usually set to 4 to 6 to distinguish periodic knife marks from random noise background. For a local maximum point in the frequency domain The amplitude spectrum value at that location.

[0055] If a significant periodic signal is detected, directional separation processing is performed on the frequency characteristics to obtain texture components in different directions and determine whether a knife-mark-related periodic pattern exists. In one possible implementation, firstly, based on the center frequency of the significant periodic signal, a fan-shaped bandpass filter mask corresponding to the direction is constructed in the frequency domain. The original spectrum matrix is ​​multiplied by the mask, and then an inverse Fourier transform is performed to obtain the spatial domain texture component in that direction. By rotating the mask angle, multiple texture components in different directions are extracted sequentially. Secondly, the texture components in each direction are judged: the geometric features of the lines in the components are extracted. If the aspect ratio is greater than a preset threshold and the spacing between multiple line segments is equal, it meets the parallelism and equal spacing characteristics of knife marks. At the same time, the presence of collinear harmonic points at the fundamental frequency and harmonics is verified in the frequency domain. When a texture component in a certain direction is represented by highly parallel dense long line segments in the spatial domain and has collinearly distributed multi-level harmonic peaks in the frequency domain, it is determined that a knife-mark-related periodic pattern exists.

[0056] By adjusting the filter kernel parameters of the separated texture components, the frequency components related to the tool marks are suppressed to obtain the filtered texture data.

[0057] In one possible implementation, the coordinates of the significant periodic signal corresponding to the knife mark in the frequency domain are first determined based on the frequency spectrum obtained from the Fourier transform. and its symmetrical points The center parameter of the filter is used as the basis for analysis; secondly, the diffusion radius of the energy point of the knife mark in the spectrum is analyzed, and the cutoff radius of the notch filter is dynamically set. Using a Butterworth notch stopband filter, the gain coefficient at each frequency point is calculated using the following formula:

[0058]

[0059] in, and Frequency points to and distance, This represents the filter order. Finally, the original spectrum matrix... With the filter kernel matrix Element-wise multiplication yields the spectrum after suppressing the knife mark frequency components. Then, the data is returned to the spatial domain by inverse Fourier transform to obtain the filtered texture data.

[0060] Based on the filtered texture data, the image is reconstructed using the inverse Fourier transform method to obtain a smooth texture distribution map and determine the final texture distribution state. In one possible implementation, the filtered, centered spectrum matrix is ​​first inversely centered, shifting the DC component back to the four corners of the matrix; then, a two-dimensional discrete inverse Fourier transform is performed to map the spectrum matrix from the frequency domain back to the spatial domain, obtaining a complex matrix. :

[0061]

[0062] Finally, the real part of the complex matrix is ​​taken and normalized, and its value is mapped to a grayscale range of 0-255 to obtain a smooth texture distribution map; at this point, the periodic tool mark background has been suppressed, and only the metal substrate and non-periodic suspected defect features are retained. This is the filtered spectrum matrix. and These are the width and height of the image, respectively; The spatial domain complex matrix obtained after the inverse transformation has its real part forming the reconstructed image; , : Coordinate variables in the frequency domain , ; , Coordinate variables in the spatial domain , ; Imaginary unit, satisfying ; : The kernel function of the inverse Fourier transform, used to convert frequency domain data back to the spatial domain.

[0063] For the smooth texture distribution map, local region comparative analysis is performed to determine whether there are residual periodic interferences, resulting in the final optimized texture distribution result. Data storage and formatting processing is then performed on the final optimized texture distribution result to obtain standardized texture distribution data suitable for subsequent analysis.

[0064] In one possible implementation, the smoothed texture distribution map is divided into multiple overlapping local windows. Within each window, a line integral projection is performed along the main direction of the identified tool marks to obtain a one-dimensional grayscale waveform. The autocorrelation function of this waveform is calculated. If a secondary peak exceeding a preset threshold appears at the tool mark period step, it is determined that there is residual periodic interference in the local area. For the local areas where residual interference is determined to exist, an anisotropic diffusion model is used for directional optimization: a larger diffusion coefficient is set along the tangent direction of the tool marks to smooth the residual ripples, and a smaller diffusion coefficient is set along the direction perpendicular to the tool marks to retain the defect edges, thereby smoothing out the residual interference and blending it with the background to obtain the final optimized texture distribution result.

[0065] Step S104: Based on the smoothed texture distribution map, use local texture smoothing and defect edge preservation techniques to separate the suspected defect areas and determine the potential defect point distribution map.

[0066] Step 1: Obtain the smoothed texture distribution map data. Based on the texture distribution information, use a local processing method to divide the image into blocks to obtain preliminary texture analysis results. Specifically, based on the predicted size of the texture and potential defects, set the block window size to... For each pixel, the overlap step size between adjacent blocks is set, with an overlap rate of 25% to 50%. A double loop is used to traverse the entire smooth texture distribution map, cropping it into multiple spatially continuous and overlapping local image blocks. For each local image patch, calculate the texture analysis index: local mean. Local variance Information entropy, as well as contrast and energy based on the local gray-level co-occurrence matrix; these indicators are output as the feature vector set of each block to obtain preliminary texture analysis results.

[0067] Step Two: Based on the preliminary texture analysis results, smoothing techniques are applied to each region to suppress details, highlighting key texture features and determining the distribution range of suspected defects. Specifically, anisotropic diffusion filtering is used to process each image block: local pixel grayscale is treated as heat, and the grayscale gradient of each pixel in four directions is calculated; when the gradient value is less than a preset control threshold... When the gradient value is greater than a certain value, the diffusion coefficient reaches its maximum value, enabling rapid and smooth suppression of fine background details; when the gradient value is greater than a certain value... When the diffusion coefficient approaches zero, defect edges are preserved, thus highlighting the main texture features while suppressing background texture; then, the difference map of the image patches before and after smoothing is calculated: ,in, For the difference map at pixel positions The absolute value of the grayscale difference at a given point reflects the degree of change of that pixel before and after smoothing. Before smoothing, local image patches are in pixels The grayscale value at that location; After anisotropic diffusion filtering, the same local image patch at the pixel level... The grayscale value at each location. Finally, based on the local mean of each image patch. With local standard deviation Dynamically set a binarization threshold: ,in, The preset scaling factor is used to adjust the threshold relative to the standard deviation to control the defect detection sensitivity. Pixels in the difference map that are above the threshold are marked as 1, forming a mask matrix; the area covered by all connected pixel blocks with a value of 1 in the mask is determined as the distribution range of suspected defects.

[0068] Step 3: For the distribution range of suspected defects, the edge preservation method is used to enhance the region boundary, separate the clearly defined defect region, and obtain the image data after defect extraction.

[0069] In one possible implementation, a narrow band region is formed by extending a preset pixel width (e.g., 3 pixels) outward from the edge of the suspected defect distribution range. An adaptive bilateral Laplacian operator is used for boundary enhancement: noise-free local edge grayscale trends are extracted, the second derivative of the edge is calculated, and a weight coefficient proportional to the local contrast is applied. The original boundary is superimposed to narrow the transition zone between the defect and the background, making the edges steeper. Then, a watershed algorithm is used for defect separation: the enhanced strong edges are used as insurmountable boundaries, and the gray-level minimum points inside the suspected defect are used as seed points for region diffusion, so that the diffusion boundary stops at the outer contour of the enhanced defect. Then, noise with an area smaller than a preset threshold is removed through geometric topology to obtain a clear defect region. Finally, a binary mask matrix with the same size as the original image is constructed. The pixels within the defective area are assigned a value of 1, and the background is assigned a value of 0, and compared with the source image. Perform point-to-point multiplication: Output image data after defect extraction ,in, This is the grayscale value at the same pixel location in the original high-resolution texture map (or initial input image). In the output image, normal areas are displayed as black, while defective areas retain the original texture, grayscale, and edge information.

[0070] Step 4: Analyze the distribution characteristics of potential defects from the extracted image data. Using a preset threshold, if the texture anomaly value in a certain area exceeds the threshold, it is marked as a potential defect point, resulting in a preliminary set of potential defect locations. Specifically, for each independent closed connected component in the extracted image data, calculate its geometric features, including area, perimeter, aspect ratio, and circularity; simultaneously, extract the grayscale dynamic range of pixels within the region and perform statistical distribution analysis; calculate the texture anomaly value for each pixel using the following formula. :

[0071]

[0072] in, For pixels grayscale value, This represents the average gray level of the normal base points within the current image patch. The grayscale standard deviation of the base point is used; finally, a preset dynamic threshold is applied. (Values ​​range from 3.5 to 6.0, corresponding to the base standard deviation) (multiple), will satisfy The pixels or regions are marked as potential defect points, and the initial set of potential defect points is obtained by summarizing them.

[0073] Step 5: Based on the initial set of potential defect locations and the density information of the location distribution, cluster the locations using distribution construction techniques to determine the concentrated areas of location distribution and construct a distribution map of potential defects. One possible implementation uses the DBSCAN density clustering algorithm, setting a neighborhood radius. and the minimum number of points required to form the core point ; Traverse all potential defect points, if a certain point In radius Number of other defects included in the range Then Mark it as the core point. Starting from the core point, search for its... Points with reachable density within a neighborhood are grouped into a defect cluster. Clusters with overlapping neighborhoods are merged, and isolated points that cannot be assigned to any cluster are filtered out as noise, resulting in several defect clusters. The region enclosed by each defect cluster is the concentrated area of ​​point distribution. Finally, the convex hull of each defect cluster is calculated to obtain the minimum convex polygon boundary line enclosing all outer points of the cluster. These polygonal regions are then filled with 1s in a binary matrix of the same size as the original image, thus constructing a distribution map of potential defects.

[0074] Step Six: For the distribution map of potential defects, apply the region separation technique to divide the concentrated area into two sub-regions. If the point density in the sub-region after division is higher than the preset threshold, the sub-region is confirmed as a high-risk defect area, and the final defect distribution result is determined.

[0075] In one possible implementation, a quadtree decomposition method is used to perform a secondary division of each concentrated region in the potential defect distribution map: taking the center of the geometrically circumscribed square of the region as the origin, the region is vertically divided into four equal-sized sub-regions; for each sub-region, the uniformity of the distribution of points within it is recursively checked. If it is still not uniform and the size of the sub-region is larger than a preset minimum unit (e.g., 8×8 pixels), then the division into four equal parts continues until uniformity is achieved or the minimum unit is reached. For each sub-region after division... Calculate the point density using the following formula:

[0076]

[0077] in, This represents the total number of potential defect points contained within this sub-region. This represents the total area (number of pixels) of the sub-region. The calculated density... With preset density threshold (Values ​​ranging from 30% to 50%) are compared, if If the sub-region is identified as a high-risk defect region, then all high-risk sub-regions are aggregated as the final defect distribution result.

[0078] Step S105: Based on the potential defect point distribution map and surface height reconstruction data, the suspected defect points are compared using three-dimensional geometric features to determine whether they conform to the geometric characteristics of real defects. If they do, they are marked as high-risk areas, and a defect confirmation map is obtained.

[0079] Initial suspected defect point data is obtained from the defect distribution map, and the coordinate information of these points is extracted using an automated scanning tool to obtain a preliminary set of suspected defect points. Specifically, the automated scanning tool performs a line-by-line raster scan of the binarized defect distribution map to identify all connected regions with a pixel value of 1. Then, for each independent closed contour region... Calculate its zeroth moment and first moment , According to the formula , The geometric centroid coordinates are obtained. If the defect area is large, the coordinate sequence of all pixels on the contour edge is extracted and stored sequentially. Finally, the tool outputs a structured array of point coordinates as a preliminary set of suspected defect points.

[0080] Among them, automated scanning tools refer to contour analysis and geometric feature extraction operator modules encapsulated based on computer vision graphics libraries (such as OpenCV, Halcon, or MIL). Examples include the contour finder and connected component state scan analyzer in OpenCV, or the Blob analysis commonly used in computer vision.

[0081] Based on the set of suspected defect points, the corresponding surface height data is obtained, and a three-dimensional geometric model is generated by combining the reconstructed data. Spatial mapping technology is used to construct the correlation structure between points and heights to determine the three-dimensional geometric shape.

[0082] In one possible implementation, the pixel coordinates of each point in the initially extracted set of suspected defect points are used. Using it in conjunction with the final concave-convex feature distribution map output from step S101 (i.e., the surface height matrix in the structured dataset), The pixel-level spatial alignment between the pixels allows for direct indexing and reading of the corresponding height scalar value. The reconstructed data includes the surface height map of the metal workpiece, the normal vector field, and the camera intrinsic and extrinsic parameter calibration matrices calculated using photometric stereo vision. Then, the coordinates and height data of all pixels in the entire image are converted into a 3D point cloud in the physical world coordinate system through inverse projection using the camera intrinsic parameters. The point cloud is triangulated using Delaunay triangulation or Poisson surface reconstruction algorithms to generate a seamless 3D mesh model. Next, a spatial association tree is constructed: all vertices of the 3D mesh model are organized into a KD-tree or octree spatial index structure; simultaneously, each 2D point in the preliminary suspected defect point set is... Mapping to a 3D point cloud space yields the corresponding 3D defect points. Finally, with each Centered on the defect point, a KD tree is used to search for 3D points on the surrounding normal metal substrate and fit a local plane. The vertical cross-sectional distance between the defect point and the fitted plane is then calculated. As a relative depth feature, it further calculates the solid geometric volume, three-dimensional curvature and physical opening width of all three-dimensional points in the defect connected domain, thereby transforming the two-dimensional suspected defect into a three-dimensional geometric shape with real physical dimensions.

[0083] For the three-dimensional geometric morphology, a feature comparison method is used to extract geometric characteristic parameters. These parameters are then matched and analyzed against preset geometric characteristic standards. If the extracted parameters and their standard deviations are within a preset threshold, the point is determined to possess true defect characteristics, resulting in a set of true defect points. Specifically, for each suspected defect point's three-dimensional geometric morphology, the following geometric characteristic parameters are extracted: maximum concavity depth. (Maximum vertical distance between the defect point and the reference plane of the adjacent normal metal), physical opening width (Cross-sectional span of the 3D defect mesh profile on the normal plane), aspect ratio and the rate of change of three-dimensional curvature Then, a feature comparison method is used: centering on the suspected defect point, a local tangent plane of the surrounding normal area is fitted using the least squares method as a reference plane. The perpendicular distance from each vertex on the defect mesh to this reference plane is calculated to extract the depth distribution. Simultaneously, a cross-sectional profile is drawn along the minor axis of the defect to extract the opening width. Next, the extracted parameters are matched and analyzed with preset geometric characteristic standards. These preset standards are pre-set according to industrial product quality acceptance specifications; for example, a true defect must meet a depth requirement. And the included angle of the edge normal For each parameter, calculate the deviation between the actual value and the lower limit of the standard, such as depth deviation. ,in, This is the minimum permissible depth in the preset standard. Finally, it is determined whether the deviation is within a preset threshold. The preset threshold is a measurement uncertainty or tolerance threshold, typically set to... Or 5% of the standard value. If the deviation is... If the deviations of other parameters do not exceed the corresponding thresholds, then the suspected defect is determined to have the characteristics of a real defect and is included in the set of real defect points.

[0084] From the set of real defect points, data on points with real defect characteristics are obtained. Combined with a three-dimensional geometric model, the degree of clustering of points in space is analyzed. If the degree of clustering is higher than a preset threshold, it is marked as a high-risk area, and the range of the high-risk area is determined.

[0085] In one possible implementation, points exhibiting realistic defect characteristics are projected onto the surface of a 3D mesh model. A spatial neighborhood analysis method based on geodesic distance is employed: the mesh topological path distance is calculated along the curved surface of the irregularly shaped component, rather than linear Euclidean distance, to avoid misclassifying points located on different curved surfaces as interconnected. Then, the degree of clustering is quantified: a 3D spherical window (with a radius of, for example, 2 mm) is set centered on each defect point, and the number of real defect points contained within the window is calculated. The aggregation degree is calculated using the following formula:

[0086]

[0087] in, For the volume of the spherical window, This represents the local area of ​​the corresponding 3D curved surface. Finally, the calculated aggregation degree is... Compare with a preset threshold. The preset threshold is set according to engineering requirements, for example, a value of... .when If the risk level exceeds the threshold, the local area is determined to be a high-risk area, and the spatial range of the high-risk area is output.

[0088] For high-risk areas, the surface height differences between their boundary points and surrounding areas are obtained. Boundary stability is calculated using a difference analysis tool. If the difference exceeds a preset threshold, it is determined to be an expansion trend of the high-risk area, and the expansion trend distribution is obtained. Specifically, first, the closed boundary lines of the high-risk area are traversed, and a pair of conjugate sampling loops are constructed by extending 1-2 pixels inside and outside the boundary lines. From the height map... Read the height scalar values ​​of the corresponding coordinates of the two sampling loops, and then use the formula... Calculate the surface height difference at each point along the boundary, where, This represents the height value corresponding to the sampling loop inside the boundary. The height value corresponding to the sampling loop outside the boundary. Then calculate the boundary stability: arc length along the boundary profile. High differences Find the first derivative or calculate the local standard deviation This characterizes the intensity of spatial fluctuations in the height difference gradient. Smaller fluctuations indicate a more stable boundary, while larger fluctuations suggest a risk of microscopic collapse or continuous tearing of the boundary. Next, the difference values ​​at each boundary point are... The value is compared with a preset threshold. This preset threshold is set based on the upper limit of the workpiece material roughness, typically three times the upper limit, and generally set to 0.02mm to 0.04mm for precision metal parts. If the height difference of a certain boundary segment exceeds this threshold, the boundary structure is determined to be unstable and has a tendency to expand and deteriorate outwards. Finally, for boundary segments determined to have an expansion trend, the tearing path is predicted by extending outwards along the direction of maximum principal curvature or the stress release normal, and the expansion trend distribution is output. The output format is a two-dimensional vector field map or an expansion probability heatmap with probability weights, where pixels on the expansion trend path are assigned a probability value of 0 to 1.

[0089] Based on the expansion trend distribution and the defect distribution map, a convolutional neural network algorithm is used to extract deep features from high-risk areas, analyze the correlation between potential expansion paths and suspected defect points, and determine the final defect confirmation range. Using the final defect confirmation range, a defect confirmation map is generated, and visualization tools are used to color-code and annotate high-risk areas and expansion trends, resulting in a complete defect confirmation image.

[0090] In one possible implementation, a multi-channel feature tensor is constructed as the input to the convolutional neural network: channel 1 is the original grayscale texture map, channel 2 is the surface height map, and channel 3 is the expansion trend probability heatmap. The network adopts the U-Net++ architecture (or DeepLabv3+), the encoder uses a pre-trained ResNET-34 as the backbone network, and the decoder introduces a self-attention module. The training method is as follows: 2000 sets of sample images of irregularly shaped metal parts are collected, and three senior quality inspection engineers manually label the real defect areas, expansion trend boundaries, and non-defect interference areas to generate pixel-level label maps (background 0, core defect 1, expansion trend boundary 2). The network model employs a joint training approach using cross-entropy loss and Dice loss, with an initial learning rate of 0.001 and a momentum of 0.9. Training lasts for 20 epochs. The input image patch size is 256×256 pixels. The output is the confidence score of each pixel belonging to one of three classes (background, core defect, and expansion trend). The final defect confirmation range is defined as a contiguous group of pixels with a core defect confidence score greater than 0.85 and an expansion trend confidence score greater than 0.75. After five-fold cross-validation, the network model achieves an average intersection-union ratio (IUU) of 0.89 on the validation set, meeting the requirements for engineering applications.

[0091] As another feasible implementation, the DeepLabv3+ architecture can also be used, which extracts multi-scale contextual features through the hollow spatial pyramid pooling module, and can also achieve the analysis of defect expansion trends. Those skilled in the art will understand that any convolutional neural network with pixel-level multi-classification and multi-scale feature fusion capabilities (such as SegNet, PSPNet, etc.) can be used to implement the defect confirmation range determination function of the present invention.

[0092] Step S106: Based on the defect confirmation map, use high-precision line laser scanning to obtain three-dimensional detail data of high-risk areas, and combine the texture detail extraction results to determine the defect depth and morphological feature distribution.

[0093] High-precision line laser scanning technology is used to comprehensively scan high-risk areas and acquire original 3D detailed data. By preprocessing the original 3D detailed data to filter out noise interference, clear 3D structural data is obtained. In one possible implementation, the high-precision line laser scanner is activated, and a microscopic 3D scan is performed on the high-risk core area identified in the defect confirmation map generated in step S105 to obtain high-resolution original point cloud or height map data. Then, noise filtering preprocessing is performed: 1. A 3D statistical outlier removal algorithm is used: the laser scan point cloud is traversed, and the average distance from each point to its k nearest neighbors is calculated. Assuming that the distance distribution of the entire image conforms to a Gaussian distribution, points with an average distance exceeding "mean + 3 times standard deviation" are identified as flying point noise and removed; 2. Adaptive bilateral point cloud filtering or moving least squares method is used to smooth high-frequency burrs: when fitting local surfaces, the smoothing weight is controlled according to the change in the angle between the surface normal vectors, thereby smoothing out laser line jump burrs caused by uneven reflection while completely preserving steep edges such as cracks and pits. After the above preprocessing, clear and noise-free three-dimensional structural data are output.

[0094] Based on clear 3D structural data and combined with texture detail extraction technology, surface texture information is analyzed to determine defect depth information. For example, 3D surface topological curvature difference extraction technology (i.e., microscopic digital surface high-pass filtering technology) is used to filter out the macroscopic 3D curved surface of irregularly shaped parts, obtaining the microscopic undulation features remaining on the surface, including the spatial height amplitude distribution of roughness, micro-tool marks, linear crack indentations, and point pits. Then, the defect depth information is analyzed and determined: using the normal laser 3D data around the high-risk area as the boundary, nonlinear least squares fitting is performed using low-order polynomial surfaces or B-spline surfaces to reconstruct the macroscopic 3D ideal geometric bottom surface of the area in its intact state. ; to obtain clear 3D height data from actual scanning Align with the ideal bottom surface and make the difference: Extract all The spatially continuous connected domain is used as a microscopic physical depth distribution. Based on the above depth distribution, the following defect depth information is determined: maximum depth. Average depth and depth variance / gradient field .

[0095] Based on the defect depth information, morphological feature analysis methods are used to identify the specific shape and boundary of the defect and obtain morphological feature data. For example, a three-dimensional differential geometric morphological feature analysis method is employed. During boundary identification, the first-order partial derivatives of the three-dimensional data are combined with the Canny-Sollin operator to calculate the rate of change of the angle between the normal vectors at each point on the surface. When the abrupt change in angle exceeds a preset threshold (e.g., greater than 25°), the envelope at that point is determined as the precise physical boundary of the defect. Then, shape identification is performed: using the eigenvalues ​​of the Hessian matrix... , Classify, if It is identified as dot-like pits or sand holes; if Identified as linear cracks or scratches; if and The defects were identified as burrs caused by surface extrusion. Finally, morphological feature data were obtained: spatial integration was performed on the 3D mesh patches within the boundary using Green's formula to calculate the true surface area, physical principal axis length, maximum width, aspect ratio, roundness, and principal extension direction vector of the defects; the volume of the 3D convex hull was calculated by performing volume integration on the 3D cylinder formed by all triangular patches inside the defects and the reference plane.

[0096] If there are outliers in the morphological feature data, the outliers are compared with a preset threshold range to determine whether they belong to the defect range, thus obtaining the defect distribution data.

[0097] In one possible implementation, within a defined defect boundary closure region, a 3×3 median differencer is used for detection: if the feature value (e.g., depth value) of a pixel deviates from the median of its 8 neighboring pixels by more than 4 times the standard deviation, or if the normal vector of the pixel is spatially isolated and reversed, then the pixel is determined to be a morphological or depth anomaly. The anomaly is then compared with a preset threshold range. The preset threshold range includes: a physical scale reasonableness threshold range (e.g., a depth tolerance range [0.005mm, 1.500mm]) and a statistical continuity threshold range (the maximum abrupt change in variance allowed between the anomaly and the neighborhood mean). Finally, a multi-spatial neighborhood consensus voting and geometric constraint region diffusion mechanism is used to determine whether anomalies belong to the defect range: if an anomaly is completely surrounded by confirmed real defect pixels, and its original grayscale appearance is dark or reflective, it is determined to be an extreme damage point inside the defect and included in the defect range; if an anomaly is located outside the defect boundary, but its depth difference is greater than the upper limit of surface roughness, and its distance from the topological geodesic of the defect core is within one processing step, it is determined to be a boundary tear burr or sub-pixel micro-crack and included in the defect range; if an anomaly exceeds the reasonable range of physical scale and there is no defect response around it, it is determined to be a hardware specular interference noise point and is removed. Through the above determination, the final defect distribution data is obtained.

[0098] Based on defect distribution data, the scan results of high-risk areas are integrated to generate a description of defect distribution, enabling accurate judgment. By archiving these distribution descriptions, a defect feature database is constructed, providing foundational data support for subsequent analysis.

[0099] In one possible implementation, the defect distribution data obtained from the high-precision line laser scanning and texture detail fusion calculation in step S106 is integrated into a defect distribution description report according to the following dimensions: 1. Basic identification and location: Assign a globally unique asset ID to each defect, mark the spatial surface area label of the irregular part to which it belongs (such as surface A, deep hole chamfer area), and output the high-precision physical space three-dimensional coordinates of the defect centroid. 2. Three-dimensional morphological description: Outputs the defect topological shape classification conclusion (cracks, scratches, pores, spalling, etc.), microscopic physical absolute boundary line equation (polygon vertex sequence), actual physical length, maximum opening width, and actual surface area (unit: mm²); 3. Depth and volume quantification: Outputs the three-dimensional absolute maximum depth of the defect. Average depth (Accurate to the micrometer level), and the actual physical volume of the peeling or cracking loss at the defect (unit: mm³); 4. Evolution trend and hazard rating: Output edge geometric stability index (high / medium / low), potential deterioration cracking direction vector obtained by combining the expansion path, and quality risk rating based on a comprehensive assessment of depth, area, and volume.

[0100] Step S107: Based on the distribution of defect depth and morphological features, adaptive focal length imaging technology is used to acquire images for different depth regions. After fusing filtering and noise control technology, a clear defect image of the entire region is obtained.

[0101] For the defect depth and depth region, adaptive focal length imaging technology is used to perform layered scanning of different depth regions to obtain initial regional image data. Specifically, based on the absolute depth field distribution of the defect output in step S106, the depth region is divided in combination with the depth of field range of the industrial camera. The camera optical parameters are retrieved: allowable circle of confusion diameter δ, aperture value F, lens focal length f, and current object distance L, to calculate the current depth of field ΔL; using ΔL as the step size, the total defect depth is divided into multiple continuous depth regions from top to bottom. Then, adaptive focal length imaging technology is used for layered scanning: the motion control system reads the physical position of the current depth region, calculates the corresponding target object distance, and inputs a voltage pulse signal to the electric liquid lens or servo zoom lens to make the focusing plane accurately focus on the depth region; the camera is triggered to acquire the image of the region; this process is repeated layer by layer, changing the focal length and acquiring the layered image sequence of each depth region to obtain the initial regional image data.

[0102] From the acquired regional image data, morphological features are extracted, and the features are classified using a preset threshold to determine whether there are significant defect areas in the image. If the classification results show the presence of significant defect areas, the image data of that area is locally magnified to obtain a high-resolution local defect image.

[0103] In one possible implementation, a 15×15 sliding window is run on the layered image, and the Laplacian-of-Gaussian operator is used to extract high-frequency edge energy values. The gray-level co-occurrence matrix is ​​then used to extract local entropy and spatial heterogeneity indices as morphological features. A binary tree decision tree or a shallow support vector machine classifier is then used, combined with a preset threshold, to classify the features. The preset threshold includes an edge energy threshold. and connected region area threshold (Calibrated based on background material roughness and noise). The classifier divides each image window into a normal background area, a benign texture area, or a suspected real damage area. If the area of ​​a connected pixel marked as a suspected real damage area is greater than... If the local contrast of this region deviates extremely from the average background, then a significant defect area is identified in the image. Next, the coordinates of the minimum bounding rectangle of the significant defect area are calculated. First, lock the ROI boundary; then use a high-magnification zoom lens to increase the optical magnification, or use a deep learning-based single-image super-resolution algorithm (such as RCAN or ESRGAN) to perform 2x or 4x lossless digital interpolation magnification to obtain a high-resolution local defect image.

[0104] For high-resolution images with local defects, filtering methods are applied to control noise, eliminate interference signals in the image, and obtain a denoised defect image. The filtering method can be nonlocal mean filtering or 3D block matching filtering. In one possible implementation, nonlocal mean filtering is used as an example: around the current pixel... Define a 3×3 neighborhood block Search within the large window of the entire map for all blocks with similar structures. Calculate the gray-level Euclidean distance between two blocks; the smaller the distance, the higher the similarity weight is assigned. ; pixel The denoised grayscale value is obtained by weighted averaging of the grayscale values ​​of the center pixels of all similar blocks. This method can eliminate high-frequency interference signals such as random speckle and white noise generated by metal reflection, while preserving real continuous physical boundaries such as cracks, thus obtaining a denoised defect image.

[0105] By comparing the denoised defective image with the initial region image data, the sharpness distribution of each depth region in the entire image is determined. Specifically, using the initial region image data as a spatial coordinate reference, affine transformations and pixel-level alignment are performed on local images captured at different depths and denoised to eliminate the image breathing effect during zooming. Then, for each depth layer image... The Laplacian gradient operator is used to calculate the gradient of each pixel in the entire image. Gradient energy in the neighborhood As a local sharpness assessment value:

[0106]

[0107] For each spatial coordinate position By comparing the sharpness values ​​of all depth layer images laterally, we can find the one that makes the image sharpness worse. The depth index that reaches the maximum value: Finally, a two-dimensional resolution mapping matrix is ​​constructed, in which coordinates... The value at each point indicates the depth layer at which the point is sharpest, and its maximum sharpness energy score is recorded, thus obtaining the sharpness distribution across all depth regions in the entire area. Among these, For the first Layer depth image in pixel coordinates The local sharpness assessment value (focus metric) at that point indicates that the edge of that point is sharper and the focus is more accurate. The larger the value, the sharper the edge of that point and the more accurate the focus. The index number of the depth layer; These are the pixel coordinates in the image; For the first Layer Image The response value obtained by applying the Laplace operator (second derivative) is used to characterize the degree of drastic change in local gray level. , This is the offset of the neighborhood window; For the first Denoising grayscale image of layer depth image; For the first In a depth layer image, with Centered, Offset The pixel intensity is the location.

[0108] Based on the sharpness distribution, the parameters of the adaptive focal length imaging technique are adjusted, and a second acquisition is performed on the low-resolution areas to obtain an optimized full-area defect image. The low-resolution areas are defined by setting a sharpness threshold. Traverse the spatial sharpness mapping matrix. If the maximum sharpness score of a certain region across all depth layers satisfies... If the area is low resolution, it will be marked as a low-resolution area. The core control parameters of the adaptive focal length imaging technology mainly include: the driving pulse voltage value of the liquid lens (or the target pulse number of the servo motor), the current focal plane depth of the camera lens, and the overlapping depth of field span.

[0109] In one possible implementation, the parameters of the adaptive focal length imaging technique are adjusted as follows: Known high-resolution depth values ​​are extracted from the periphery of the low-resolution area. One or more secondary compensation depth layers are inserted between adjacent depth layers (e.g., a 0.3mm layer between 0.2mm and 0.4mm). The compensated depth values ​​are converted into the driving current of the liquid lens or the pulse signal of the servo motor using a focusing and zoom formula, causing the camera's focus plane to move to the gap depth. Next, secondary point acquisition is performed on the low-resolution area: Under the adjusted focal length parameters, secondary exposure acquisition is specifically performed on the marked low-resolution area to obtain high-contrast supplementary image fragments; the local sharpness energy of the new fragments is calculated, and the sharpness score of the corresponding area in the original sharpness mapping matrix is ​​updated. Finally, a multi-focus image fusion algorithm based on the Laplacian pyramid is used: all depth layer images (including secondary acquisition fragments) are decomposed into high-frequency and low-frequency pyramids. Based on the updated sharpness mapping matrix, the sharpness score of each pixel coordinate is calculated. The high-frequency and low-frequency coefficients of the most perfectly focused layer are extracted, preserved and synthesized at the pixel level, and reconstructed through inverse pyramid transformation to output an optimized full-area defect image.

[0110] Image stitching technology is employed to integrate the optimized full-area defect image with local defect images to generate a complete and clear defect image. Specifically, ORB feature points and binary descriptor vectors are extracted from the global defect image and the high-resolution local defect image, respectively. Hamming distance is used for feature matching, and the RANSAC algorithm is used to remove mismatched points. The homography matrix between the two images is then calculated. Then, based on the homography matrix... Perspective transformation and bicubic interpolation resampling are performed on the high-resolution local defect image to ensure its spatial alignment with the corresponding defect ROI region in the global defect image. Finally, a gradient weight mask is constructed. The image is fused using a combination of fade-in and fade-out multi-band image blending technology: the low-frequency part is smoothly faded over a wide range to eliminate stitching breaks, and the high-frequency part significantly increases the retention weight of local high-resolution images, thereby obtaining a complete and clear defect image with seamless geometric alignment and smooth transition of brightness and sharpness.

[0111] Step S108: Based on the clear defect image of the entire area, multi-path image acquisition is performed for the blind area in the narrow space. Combined with the periodic pattern recognition results, it is determined whether there are potential risk points related to the tool marks in the blind area, and the final defect distribution report is generated.

[0112] By employing multi-path acquisition devices, defect image data from multiple angles is acquired for blind areas within confined spaces, completing preliminary data collection. One possible implementation utilizes a miniature snake-like robotic arm mounted on the end effector of a six-axis industrial robot, a multi-branch fiber optic endoscope, or a ring-distributed array of multi-angle miniature CMOS cameras as the multi-path acquisition device. For confined blind areas on irregularly shaped metal parts (including deep hole inner wall grooves, micro-thread roots, internal cavity dead angles, and T-slots) where macroscopic geometric structures obstruct direct illumination by the main camera and line laser, the device is controlled to move at multiple angles or switch viewing angles to acquire defect image data from different directions within the blind area, thus completing multi-angle image data collection for the blind area.

[0113] Based on the collected defect image data, a pre-established image processing module is used to denoise and enhance the image, resulting in a clear processed image. In one possible implementation, the specific algorithm flow of the pre-established image processing module is as follows: First, the input blind zone image is subjected to block adaptive histogram equalization (CLAHE), with parameters set as follows: block size of 8×8 pixels, clipping threshold of 2.0, and after limiting the contrast amplification factor, the image is converted from RGB space to Lab space, processing only the L channel, and then converted back to RGB. Then, three filtering steps are executed sequentially: First, a 3×3 median filter is used to remove isolated salt grain noise; second, an anisotropic diffusion filter is applied, with an iteration count of 10, a diffusion coefficient of 0.25, and a gradient threshold dynamically calculated based on the image grayscale (e.g., taking the 70th percentile of the image gradient magnitude) to smooth background noise while preserving knife marks and crack edges; third, a bilateral filter is used, with spatial domain sigma=1.5 and grayscale domain sigma=25.0, to further eliminate residual high-frequency interference. All filtering operations are accelerated using CUDA, achieving a processing time of less than 30ms for 1024×1024 images on the NVIDIA Jetson AGX Xavier platform. This module loads a pre-compiled dynamic link library during system initialization. The interface function is `process Blind Image(unsigned char* input, unsigned char* output, int width, int height)`, outputting an 8-bit grayscale or color image. After this processing, background noise in the original endoscopic images with low illumination and narrow field of view is effectively suppressed, and defect edges are clearly discernible.

[0114] For the processed image, a periodic pattern analysis method is applied to identify whether there are repetitive or regular knife mark features in the image, and to determine potential abnormal regions. One possible implementation uses a spatial directional frequency domain topological texture analysis technique based on a two-dimensional Gabor filter bank. First, a set of filters with different directions is constructed... (e.g., 0°, 45°, 90°, 135°) and different center frequencies The Gabor kernel function is used to perform a two-dimensional convolution between the denoised and enhanced image and the filter bank to obtain energy response maps in each direction. Then, the mean, variance, and second moment of the energy in the response maps are calculated. If in a specific direction... If the energy response exhibits a high and narrow peak, and the autocorrelation function shows periodic peaks with equal spacing, then repetitive or regular knife-mark features are identified in the image. Finally, the local energy outlier detection method is used: within the normal knife-mark region, the Gabor response is continuous and uniform; if the periodic signal is interrupted, the texture is disordered, or the energy amplitude suddenly increases within a certain local block, then the boundary of that local block is defined as a potential anomalous region.

[0115] If an abnormal area is identified, the spatial correlation between the abnormal area and known risk points is calculated by combining the distribution of knife mark features to determine whether there is a potential risk.

[0116] In one possible implementation, known risk points are retrieved from a shared 3D defect feature database. These risk points are the centroid coordinates or edge skeleton points of high-risk real defects that have been precisely located and confirmed on the outer surface of the metal part in steps S104 to S107, and have a unified camera world coordinate system. Then, using the calibration matrix of the multipath acquisition device, the two-dimensional coordinates of the abnormal areas detected in the blind zone image are transformed to the same world coordinate system to obtain the center point of the blind zone anomaly. Next, the anomaly center point of the blind zone and the nearest known external surface risk point are calculated. Euclidean distance between And extract the three-dimensional potential expansion path direction vector of known risk points. Calculate the cosine of the angle between the spatial vector pointing from the center of the blind zone anomaly to the known risk point and the vector of the expansion direction. Calculate the correlation score using the following formula:

[0117]

[0118] Finally, set a spatial correlation threshold. ,like If this is confirmed, it indicates that external visible defects have spread inward through the material's interior or blind areas, thus confirming a potential risk. Spatial relevance score; Abnormal center point of blind zone Compared with the most recently known risk points The Euclidean distance between them; The denominator should be a very small positive number to avoid being zero.

[0119] Based on the analysis results of potential risks, the system maps the location distribution of abnormal areas within the blind zone, generating corresponding risk distribution data. In one possible implementation, the system employs coordinate projection technology to map the three-dimensional points of the blind zone identified as potentially risky through correlation analysis. The probability field is projected back onto the digital 3D mesh skeleton of the workpiece. Risk probabilities are then filled into the corresponding locations on the blind zone CAD topology map, and scores are assigned based on correlation. The risk weights are assigned from 0.0 to 1.0, and a structured risk distribution dataset is generated. The dataset includes at least the blind spot number, the abnormal area, and the probability of potential tearing risk.

[0120] After acquiring the risk distribution data, the overall information of the full-area defect image is combined to generate a complete risk analysis result, and the final distribution report is output. The overall information of the full-area defect image refers to the complete 2D / 3D composite defect confirmation image data finally synthesized and output in step S107, which includes a macroscopic view of the workpiece, clear multi-layer focus, high-definition grayscale texture, and high-risk core 3D features.

[0121] In one possible implementation, the integrated risk analysis results are obtained by fusing external visible defect data (including the area, depth, and volume of confirmed cracks) with blind zone hidden risk data (including blind zone number, abnormal area, and risk probability) through a multi-source information association matrix. The data is then combined based on geometric adjacency. If external cracks and internal blind zone risk points are determined to be caused by the same stress concentration zone, they are classified as belonging to the same "fatal failure chain." The final output distribution report is structured data, containing at least the following: 1. Basic workpiece information: workpiece batch number, serial number, inspection timestamp, and pass status; 2. Visible defect area data: the number of actual defects on the outer surface and the physical coordinates of each defect. 1. Maximum depression depth, opening width, 3D surface area and volume; 2. Description of potential risks in blind zones: a list of abnormal areas within the blind zone, description of abnormal tool marks, and risk deterioration paths and probabilities of internal and external overlap; 3. Visualized comprehensive risk map: a clear fusion map of the entire area overlaid with a panoramic defect heat map, with color markings according to hazard level; 4. Automated quality handling recommendations: outputting final failure decisions (scrapping, rework, or reprocessing, etc.) based on depth, volume, and internal and external collaborative expansion trends.

[0122] like Figure 4As shown, this invention provides a machine vision system for processing defect images of non-standard products, mainly comprising:

[0123] The image sequence acquisition module is used to acquire image sequences under different lighting conditions by illuminating the surface of an irregularly shaped metal part with multi-angle light sources.

[0124] The surface height reconstruction module is used to calculate the surface normal vector and construct the reflection model to reconstruct the surface height information and obtain the initial concave and convex feature map.

[0125] The texture extraction module is used to perform gloss reflection separation and shadow area compensation based on the initial concave-convex feature map, extract texture details, and generate a surface texture map after removing lighting interference.

[0126] The periodic pattern processing module is used to perform frequency domain feature analysis on the surface texture map, identify periodic patterns, and perform frequency suppression processing when periodic tool marks are detected to obtain a smoothed texture distribution map.

[0127] The defect region separation module is used to separate suspected defect regions and determine the distribution map of potential defect points based on the smoothed texture distribution map and using local texture smoothing and defect edge preservation techniques.

[0128] The defect confirmation module is used to compare the three-dimensional geometric features of potential defect points with surface height reconstruction data based on the distribution map of potential defect points. Points that match the geometric characteristics of real defects are marked as high-risk areas, thus obtaining a defect confirmation map.

[0129] The 3D detail acquisition module is used to perform line laser scanning on high-risk areas based on the defect confirmation map to acquire 3D detail data and determine the depth and morphological feature distribution of defects.

[0130] The defect image generation module is used to acquire images of different depth regions by using adaptive focal length imaging based on the distribution of defect depth and morphological features, and then fuse and filter them to obtain a clear defect image of the entire region. Based on the clear defect image of the entire region, multi-path image acquisition is performed for blind areas in narrow spaces. Combined with the results of periodic pattern recognition, potential risk points in the blind areas are identified, and a final defect distribution report is generated.

[0131] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. The scope of patent protection of the present invention shall be determined by the claims. Similarly, any equivalent structural changes made based on the description and drawings of the present invention shall also be included within the scope of protection of the present invention.

Claims

1. A machine vision method for processing defect images of non-standard products, characterized in that, The method includes: By illuminating the surface of an irregularly shaped metal part with multiple angle light sources, an image sequence is obtained, the surface normal vector is calculated, a reflection model is constructed, and the surface height information is reconstructed to obtain an initial concave-convex feature map. Based on the initial bump feature map, gloss reflection separation and shadow area compensation are performed to extract texture details and generate a surface texture map after removing lighting interference. Frequency domain feature analysis is performed on the surface texture map to identify periodic patterns. When periodic tool marks are detected, frequency suppression processing is performed to obtain a smoothed texture distribution map. Based on the smoothed texture distribution map, local texture smoothing and defect edge preservation techniques are used to separate suspected defect areas and determine the distribution map of potential defect points. Based on the distribution map of potential defect points, and combined with the surface height reconstruction data, the potential defect points are compared with three-dimensional geometric features. Those that match the geometric characteristics of the real defects are marked as high-risk areas, and a defect confirmation map is obtained. Based on the defect confirmation map, line laser scanning is performed on high-risk areas to obtain three-dimensional detailed data and determine the distribution of defect depth and morphological features. By analyzing the distribution of defect depth and morphological features, adaptive focal length imaging is used to acquire images of different depth regions, and the images are then fused and filtered to obtain a clear defect image of the entire region. Based on clear defect images of the entire area, multi-path image acquisition is performed for blind spots in narrow spaces. Combined with periodic pattern recognition results, potential risk points in the blind spots are identified, and a final defect distribution report is generated.

2. The method according to claim 1, characterized in that, The process of obtaining the initial concave-convex feature map includes: Image sequences under multi-angle light sources are acquired, the illumination intensity distribution matrix is ​​extracted, the surface normal vector is calculated, and the reflection parameters are obtained. The surface height map is calculated based on the reflection parameters, and local adjustments are made to abnormal areas to output a distribution map of concave and convex features.

3. The method according to claim 1, characterized in that, The process of generating a surface texture map after removing illumination interference includes: separating the gloss reflection component from the bump feature map, performing brightness compensation on the shadow area, extracting and enhancing surface texture details, and then removing residual illumination interference through local contrast adjustment.

4. The method according to claim 1, characterized in that, The obtained smoothed texture distribution map includes: The surface texture map is denoised and subjected to Fourier transform to analyze frequency characteristics in order to identify periodic tool marks. The filter kernel parameters are adjusted to suppress the frequency components of the knife marks, and the image is then reconstructed through inverse transformation. Local residual interference is also optimized.

5. The method according to claim 1, characterized in that, The determination of the potential defect point distribution map includes: The smoothed texture distribution map is divided into blocks to suppress background details and preserve defect edges; Suspected defect areas are extracted by segmentation using a preset threshold. Then, texture outlier judgment and density clustering are performed on the extracted defect areas to construct a distribution map of potential defect points.

6. The method according to claim 1, characterized in that, The obtained defect confirmation map includes: The coordinates of potential defect points are obtained from the distribution map of potential defect points, and a three-dimensional geometric model is generated by combining the surface height data. Compare the geometric characteristic parameters of the defect points to determine whether they conform to the characteristics of a real defect; Spatial clustering analysis is performed on the matching points to mark high-risk areas, and the expansion trend is judged by combining the differences in boundary height. Finally, the defect confirmation range is determined by using a deep learning model.

7. The method according to claim 1, characterized in that, The determination of defect depth and morphological feature distribution includes: High-precision line laser scanning is performed on high-risk areas to obtain three-dimensional structural data after noise is filtered out. By combining texture details, defect depth information is obtained through surface fitting; Then, the shape and boundary of the defect are identified through three-dimensional differential geometry analysis to obtain morphological feature data.

8. The method according to claim 1, characterized in that, The process of obtaining a clear defect image across the entire area includes: Based on the depth of the defect, hierarchical regions are divided, and adaptive focal length technology is used to perform layered scanning and image acquisition. Local magnification and filtering are performed on areas with significant defects to reduce noise. By comparing and determining the sharpness distribution of each depth region, adjusting the parameters of the low-sharp areas and acquiring them a second time, the global and local defect images are finally fused to generate a complete and clear defect image.

9. The method according to claim 1, characterized in that, The generation of the final defect distribution report includes: Multi-angle image acquisition of the blind area, followed by noise reduction and enhancement, is performed using periodic pattern analysis to identify abnormal knife mark areas; The spatial correlation between abnormal areas and known risk points is calculated to determine the risk, generate blind zone risk distribution data, and then integrate it with the full-area defect image information to output a complete risk analysis report.

10. A machine vision system for processing defect images of non-standard products, characterized in that, The system includes: The image sequence acquisition module is used to acquire image sequences under different lighting conditions by illuminating the surface of an irregularly shaped metal part with multi-angle light sources. The surface height reconstruction module is used to calculate the surface normal vector and construct the reflection model to reconstruct the surface height information and obtain the initial concave and convex feature map. The texture extraction module is used to perform gloss reflection separation and shadow area compensation based on the initial concave-convex feature map, extract texture details, and generate a surface texture map after removing lighting interference. The periodic pattern processing module is used to perform frequency domain feature analysis on the surface texture map, identify periodic patterns, and perform frequency suppression processing when periodic tool marks are detected to obtain a smoothed texture distribution map. The defect region separation module is used to separate suspected defect regions and determine the distribution map of potential defect points based on the smoothed texture distribution map and using local texture smoothing and defect edge preservation techniques. The defect confirmation module is used to compare the three-dimensional geometric features of potential defect points with surface height reconstruction data based on the distribution map of potential defect points. Points that match the geometric characteristics of real defects are marked as high-risk areas, thus obtaining a defect confirmation map. The 3D detail acquisition module is used to perform line laser scanning on high-risk areas based on the defect confirmation map to acquire 3D detail data and determine the depth and morphological feature distribution of defects. The defect image generation module is used to acquire images of different depth regions by using adaptive focal length imaging based on the distribution of defect depth and morphological features, and then fuse and filter them to obtain a clear defect image of the entire region. Based on the clear defect image of the entire region, multi-path image acquisition is performed for blind areas in narrow spaces. Combined with the results of periodic pattern recognition, potential risk points in the blind areas are identified, and a final defect distribution report is generated.

Images