Material surface defect detection system based on industrial visual inspection

Abstract

The invention relates to the field of industrial visual inspection, and discloses a material surface defect detection system based on industrial visual inspection, which comprises a laser light source module, a light beam distribution assembly, a dark field illumination structure, a near field irradiation structure, an industrial visual inspection module, a signal processing module and a data processing module. The laser light source module is used for generating a modulated laser beam, and the laser beam distribution assembly guides the laser beam to the surface of a material; and the dark field illumination structure is matched with the near field irradiation structure to form an irradiation path with an inclined direction and a sub-wavelength scale. The industrial visual inspection module comprises an industrial camera, an image acquisition device and an image processing module and is used for acquiring and processing images of the defect area. The signal processing module is used for performing frequency domain analysis on the scattered signals; and the data processing module is used for analyzing spatial positions and forms of defects based on image and signal data. The system is suitable for fine detection and structure identification of tiny defects on the surface of a material.

Description

Technical Field

[0001] The present invention relates to the field of industrial visual inspection, and in particular to a material surface defect detection system based on industrial visual inspection. Background Art

[0002] In industries such as high-end manufacturing, electronic packaging, semiconductor wafers, photovoltaic modules, and precision parts, the quality of material surfaces directly impacts product functionality and reliability. Consequently, automated, high-precision detection of minute surface defects (such as scratches, pits, bubbles, and foreign particles) has become a key application area for industrial vision inspection technology.

[0003] In the existing technology, two main methods are commonly used for surface defect detection: one is the traditional optical scattering detection method, which identifies micro defects by irradiating the material surface with laser and analyzing the scattered light signal; the other is the imaging-based visual inspection method, which uses industrial cameras to capture images under specific lighting conditions and analyze them through image algorithms.

[0004] However, traditional scattering inspection methods are sensitive to factors such as system stability, alignment accuracy, and background noise, and are difficult to adapt to large-area scanning or materials with complex surface structures. Existing visual inspection solutions have limited resolution for identifying submicron defects and are often limited by single lighting methods, insufficient image contrast, and the accuracy of defect image extraction algorithms.

[0005] In addition, most scattered signal detection schemes rely on single-photon detectors or avalanche photodiodes, which have complex installation positions and detection angles, making them difficult to integrate and control with high-speed assembly lines, which is not conducive to actual industrial deployment. Summary of the Invention

[0006] The purpose of the present invention is to provide a material surface defect detection system based on industrial visual inspection, which has the advantages of high sensitivity, high spatial resolution and good industrial integration, and can achieve stable identification and real-time feedback of tiny defects in complex manufacturing environments.

[0007] The above technical objectives of the present invention are achieved through the following technical solutions:

[0008] A material surface defect detection system based on industrial visual inspection, comprising:

[0009] A laser light source module, comprising a plurality of wavelength-tunable lasers for generating a modulated laser beam;

[0010] A beam distribution assembly, connected to the laser light source module, for directing the modulated laser beam to the surface of the material in multiple directions;

[0011] A dark field illumination structure is provided on the emission path of the modulated laser beam, the dark field illumination structure comprising an annular light source and an annular aperture, for illuminating the material surface at an oblique angle and blocking axial reflected light;

[0012] A near-field illumination structure, disposed between the dark-field illumination structure and the material surface, for limiting the size of the illumination beam to a sub-wavelength scale;

[0013] Industrial visual inspection module, which is set above or to the side of the material surface to collect material surface images and identify image abnormal areas caused by surface defects;

[0014] An image processing module, electrically connected to the industrial visual inspection module, for performing enhancement processing, defect feature extraction and spatial modeling on the collected images;

[0015] A data processing module is electrically connected to the image processing module. The data processing module includes a programmable logic device and a graphics processing unit. The programmable logic device is used for synchronous timing and control of image data. The graphics processing unit is used to analyze the position, size and morphology of material surface defects based on the spatial distribution, grayscale characteristics and time characteristics of abnormal image areas.

[0016] It is further provided that: the laser light source module includes a laser array structure composed of a plurality of laser emitting units, the laser emitting units are linearly arranged along the width direction of the material surface and are configured to generate parallel modulated laser beams to irradiate multiple partitions of the material surface;

[0017] The laser light source module further includes an automatic light intensity compensation control unit, which is electrically connected to each laser emitting unit and the image processing module, and is used to adjust the output power of each laser emitting unit according to the regional image brightness information provided by the image processing module;

[0018] The automatic light intensity compensation control unit independently controls the power output of different laser emitting units based on a preset brightness threshold and partition parameters.

[0019] By adopting the above technical solution, a laser array structure consisting of multiple laser emitting units is provided in the laser light source module and arranged linearly along the width direction of the material surface, thereby achieving multi-zone parallel illumination of the measured area, providing independently controllable light source input for different surface areas. By configuring an automatic light intensity compensation control unit, electrically connected to the image processing module, and dynamically adjusting the output power of each laser emitting unit based on image brightness information, the laser illumination intensity is evenly distributed among multiple detection areas on demand.

[0020] By combining the above structure and functions, it is possible to maintain consistency in lighting conditions in various areas when the reflectivity of the material surface changes or the local structure is uneven, thereby improving the lighting uniformity and imaging stability of the system in complex surface environments.

[0021] It is further configured that: the beam distribution component includes a multi-channel beam splitting structure and a micromirror array structure, the multi-channel beam splitting structure includes a plurality of beam splitting optical elements for splitting the modulated laser beam generated by the laser light source module into a plurality of sub-beam channels;

[0022] The micromirror array structure is arranged on the emission path of each sub-beam channel, and includes a plurality of independently drivable micro-electromechanical mirror units for adjusting the emission direction and illumination angle of each sub-beam respectively.

[0023] The multi-channel beam splitting structure is connected to the micromirror array structure. Each reflector unit of the micromirror array structure corresponds one-to-one to the corresponding sub-beam channel, forming a controllable multi-path beam output system for guiding multiple modulated laser beams to multiple detection areas on the material surface respectively.

[0024] By adopting the above technical solution and setting up a multi-channel beam splitting structure, the modulated laser beam output by the laser light source module is divided into multiple sub-beam channels, so that a single light source can cover multiple detection areas at the same time, thereby improving the spatial lighting multiplexing capability of the system.

[0025] By setting up a micromirror array structure on each sub-beam channel and using independently driven micro-electromechanical mirror units to accurately adjust the beam direction of each channel, each sub-beam can illuminate the corresponding material surface area at a set angle, forming a multi-path, controllable illumination system;

[0026] The system achieves refined control of the beam direction and distribution, enabling the system to flexibly adjust the irradiation path when performing multi-area or multi-station surface defect detection to adapt to changes in the surface geometry of different materials, thereby enhancing the system's coverage capability, adaptability, and optical path configuration flexibility.

[0027] Further configuration: the industrial visual inspection module includes an industrial camera, an image acquisition device and an image processing terminal. The industrial camera is used to acquire images of the material surface. The image acquisition device is used to receive and cache data output by the camera, and perform format conversion and time marking. The image processing terminal is used to decode, preprocess and extract defect features of the received images.

[0028] It is further configured that the image acquisition device includes an image cache unit and a data transmission interface, wherein the image cache unit is used to temporarily store image frame data transmitted by the industrial camera, and the data transmission interface is used to transmit the cached image data to the image processing terminal in a preset frame sequence structure;

[0029] The image processing terminal includes an image decoding module, which is used to parse image frame sequence information and time stamp identifiers, and decode image data into a processable image format.

[0030] By adopting the above technical solution and setting up an image cache unit and a data transmission interface in the image acquisition device, the image data collected by the industrial camera can be intermediately cached and formatted before being transmitted to the image processing terminal, effectively alleviating the impact of image stream data rate fluctuations on the bandwidth and stability of the processing unit;

[0031] The image decoding module in the image processing terminal is used to parse the image frame sequence and timestamp information, and decode the image data to ensure the timing consistency and format integrity of the image input;

[0032] It is further provided that: the image processing module includes an image calculation module, the image calculation module is used to perform image preprocessing operations based on the decoded image data, and extract and encode regional defect features contained in the material surface image;

[0033] The image calculation module is configured with multiple functional sub-units, including an image pre-processing sub-module, a defect feature extraction sub-module, and a defect classification and identification sub-module;

[0034] The image preprocessing submodule is used to perform image noise suppression and detail enhancement processing, the defect feature extraction submodule performs edge detection and morphological analysis on the preprocessed image; the defect classification and recognition submodule uses a lightweight convolutional neural network to classify the extracted defect areas.

[0035] By adopting the above technical solution, the image preprocessing submodule is used to improve image quality and edge contrast, suppress background noise, and provide a stable image foundation for subsequent analysis. The defect feature extraction submodule extracts the contours of the defect area in the image and parameterizes it to achieve quantitative expression of the defect morphology and structural characteristics. The defect classification and identification submodule analyzes and compares the extracted defect features and outputs defect type labels and confidence information by integrating machine learning or deep learning models.

[0036] The above-mentioned sub-modules form a continuous image processing and reasoning link, which can effectively improve the system's defect recognition accuracy, adaptability and classification automation capabilities under complex background conditions, realize the full-process analysis function of the image from preliminary enhancement processing to defect structure recognition and intelligent classification, and enhance the intelligence level and practicality of the industrial visual inspection system.

[0037] It is further configured that a joint modeling interface is provided between the image processing module and the data processing module.

[0038] The image processing module includes a defect 3D reconstruction unit, which is used to generate a corresponding defect depth map or point cloud model based on the defect area image output by the image processing module and the spatial position information provided by the data processing module;

[0039] The data processing module includes a three-dimensional analysis unit, which is used to calculate geometric morphology parameters of the defect depth map or point cloud model and output the spatial size, surface inclination angle and volume estimation results of the defect.

[0040] By adopting the above technical solution, a joint modeling interface is set up between the image processing module and the data processing module to achieve the coordinated call of image information and spatial data, supporting the simultaneous acquisition of image features and three-dimensional spatial information during the defect recognition process;

[0041] The defect 3D reconstruction unit in the image processing module can combine the defect area extracted from the 2D image with the structural information of the material surface to generate a point cloud model or depth map containing depth and morphology;

[0042] The data processing module performs geometric calculations on the model through a three-dimensional analysis unit, extracting the spatial shape parameters of the defect, such as depth value, edge tilt angle, volume estimation, etc., to form a complete spatial quantitative output result;

[0043] This structure enhances the system's ability to identify three-dimensional defects such as pits and protrusions, and is suitable for surface inspection tasks of complex materials that are sensitive to depth, such as microstructure surfaces, wafers, and thin films.

[0044] Further configuration: the defect 3D reconstruction unit further includes a multi-angle image reconstruction module,

[0045] The multi-angle image reconstruction module is connected to multiple industrial cameras set at different shooting angles, and is used to receive defect area images collected at multiple angles, and construct corresponding three-dimensional point cloud models or depth map data based on the parallax information between the images.

[0046] The inter-image disparity information includes the displacement relationship between pixel-level corresponding points in images of different viewing angles.

[0047] By adopting the above technical solution, by introducing a multi-angle image reconstruction module into the defect 3D reconstruction unit and configuring multiple industrial cameras installed at different shooting angles, the system can collect multi-view image information of the defect area on the material surface;

[0048] Pixel-level parallax information is formed between images of different perspectives. By calculating the spatial displacement relationship between corresponding points between images, the 3D geometric morphology model of the defect area is reconstructed.

[0049] This structure improves the spatial accuracy and reconstruction integrity of three-dimensional modeling, and can restore the concave-convex structure, edge contour and depth changes of defects. It is suitable for defect modeling tasks on smooth surfaces, complex contours or reflective materials, and expands the system's ability to recognize spatial structural defects.

[0050] Further configuration: the data processing module further includes a defect level assessment unit and an early warning control unit,

[0051] The defect level assessment unit is used to classify defects according to parameters such as the area, depth, and type of the defect area; the early warning control unit determines whether to trigger an alarm instruction based on a preset level threshold and transmits the judgment result to the upper control system or display terminal.

[0052] By adopting the above technical solution and setting up a defect level assessment unit and an early warning control unit in the data processing module, the system can quantitatively score and grade the detected defects;

[0053] Once a certain type of defect exceeds the set threshold, an alarm signal can be issued in real time to automatically identify the severity of the defect and quickly respond to processing instructions;

[0054] This structure enhances the system's intelligent judgment capabilities and closed-loop control capabilities of the production site, preventing seriously defective products from flowing into subsequent links and improving overall quality control efficiency.

[0055] In summary, the present invention has the following beneficial effects:

[0056] The present invention organically combines a laser light source module, a beam distribution assembly, a dark-field illumination structure, a near-field illumination structure, an industrial vision inspection module, an image processing module, and a data processing module to form a highly sensitive, high-resolution material surface micro-defect detection system suitable for industrial assembly line environments. Specifically, the present invention effectively solves the key problems of the prior art in the following aspects:

[0057] 1. Dark-field illumination eliminates strong axial reflected light, and near-field illumination compresses the spot size to improve spatial resolution. Combined with the industrial vision inspection module, it can capture scattering or topographic features caused by tiny defects with high contrast and clarity, achieving high-precision defect detection in complex backgrounds. This solves the problem of traditional optical inspection systems being prone to specular reflection interference when dealing with high-reflectivity materials or complex textured surfaces, which can cause defect signals to be obscured or blurred.

[0058] 2. Industrial cameras can quickly capture high-resolution images. The image processing module performs edge detection and region recognition. The programmable logic device and graphics processing unit in the data processing module implement high-speed image decoding, data modeling, and classification output, thereby achieving intelligent, structured defect detection and real-time feedback. This solves the problem that traditional photoelectric signal processing paths rely on dedicated detectors, making it difficult to perform structured identification and analysis of defect images, and resulting in low processing efficiency.

[0059] 3. Near-field illumination provides a high-spatial-resolution illumination foundation, while high-resolution cameras enhance detail acquisition capabilities. Image enhancement algorithms improve edge sharpness and grayscale gradient contrast, making submicron defects more prominent in images and thus improving detection sensitivity and accuracy. This solves the problem of tiny defects (such as pinholes and fine cracks) being difficult to form distinct features in traditional visual images due to their small size and low contrast, leading to false detection and missed detection.

[0060] 4. The defect recognition algorithm automatically extracts and classifies defect areas, the 3D modeling unit restores the defect geometry, and the graphics processing unit implements parallel reasoning and real-time calculations, effectively outputting defect depth, volume, and type information to support subsequent quality analysis and process closed-loop optimization. This solves the problem that ordinary image processing systems can only detect planar features and cannot obtain spatial information such as defect shape and depth.

[0061] 5. The multi-wavelength laser source adapts to the spectral response of different materials, the flexible angle lighting realizes adjustable direction and variable incident angle illumination, and the multi-angle visual acquisition enhances the image dimension and information coverage, thereby improving the versatility and robustness of the system in the inspection of multi-category, multi-batch, and multi-morphology workpieces; it solves the problem that a single wavelength or fixed lighting method cannot adapt to all inspection requirements under different materials, different colors, and different surface texture conditions, resulting in poor detection stability and versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of the system architecture of the present invention. DETAILED DESCRIPTION

[0063] The present invention will be further described in detail below with reference to the accompanying drawings.

[0064] Example:

[0065] A material surface defect detection system based on industrial visual inspection, such as Figure 1 As shown, including:

[0066] A laser light source module, comprising a plurality of wavelength-tunable lasers for generating a modulated laser beam;

[0067] In a preferred embodiment, the laser light source module includes multiple laser emitting units Li (where i = 1, 2, ..., N), and each emitting unit is linearly arranged along the width direction of the material surface to form a one-dimensional laser array for respectively irradiating the detection area Ri corresponding to the material surface.

[0068] In order to achieve dynamic and balanced adjustment of the lighting intensity of each detection area, the laser light source module is provided with an automatic light intensity compensation control unit, which is electrically connected to each laser emitting unit and the image processing module, and adjusts the power output of each emitting unit based on the image feedback information.

[0069] The image processing module calculates the brightness of each detection area Ri and extracts its average brightness value B at time t i (t), and calculate the brightness error:

[0070] Brightness error calculation formula:

[0071] ΔB i (t) = B target -B i (t)

[0072] in:

[0073] ΔB i (t): brightness error of the i-th detection area at time t;

[0074] B target : The target brightness value set by the system;

[0075] B i (t): The average brightness of the current frame image in the i-th region.

[0076] Subsequently, the automatic light intensity compensation control unit calculates the laser power adjustment amount ΔP through the following control function i (t):

[0077] Power regulation function:

[0078]

[0079] The parameters are described as follows:

[0080] σ: The time scale parameter of the Gaussian kernel function, which controls the weight decay rate of the historical brightness error;

[0081] Gaussian decay kernel, which emphasizes that errors near the current time point have a greater impact;

[0082] Represents the historical error ΔB i (τ) performs weighted integration and accumulates the brightness change trend;

[0083] λ: normalized suppression coefficient, used to control the global equilibrium pressure;

[0084] Represents the normalized ratio of the relative global error of the i-th region;

[0085] ε: a small positive number introduced to avoid division by zero (numerical stability processing);

[0086] (.) 2 : The square operation makes the adjustment in the large deviation area more sensitive and suppresses the misadjustment caused by small disturbances.

[0087] Finally, the automatic light intensity compensation control unit updates the output power of each laser emitting unit according to the adjustment amount:

[0088] Power update formula:

[0089] P i (t+1)=P i (t)+γ·ΔP i (t)

[0090] in:

[0091] P i (t): current output power of the i-th laser emitting unit;

[0092] γ: power regulation step coefficient, control system response speed;

[0093] P i (t+1): Updated power value, used for the next laser output.

[0094] This control mechanism enables each laser emitting unit to achieve local adaptive adjustment of the brightness of its detection area while maintaining global brightness balance. The control model possesses temporal memory, local enhancement, and global stability, effectively improving the system's robustness and imaging consistency on complex material surfaces or under dynamic lighting conditions.

[0095] A beam distribution assembly, connected to the laser light source module, for directing the modulated laser beam to the surface of the material in multiple directions;

[0096] In a preferred embodiment, the beam distribution component includes a multi-channel beam splitting structure and a micromirror array structure, which is used to distribute the modulated laser beam emitted by the laser light source module to multiple detection areas on the surface of the material.

[0097] The multi-channel beam splitting structure includes several beam splitting optical elements, such as polarization beam splitters, half-reflective mirrors, or prisms, which are placed in the output path of the laser light source module to separate the modulated laser beam of a single path into multiple sub-beam channels. Each sub-beam channel has an independent output path corresponding to a different spatial illumination area.

[0098] The micromirror array structure is arranged after the multi-channel beam splitting structure and includes multiple micro-electromechanical system (MEMS) mirror units, each of which corresponds to a sub-beam channel. The mirror units have a programmable drive interface that can achieve two-dimensional (pitch / yaw) angle fine adjustment.

[0099] The beam splitting structure and micromirror array are integrated to enable spatial adjustment of each laser sub-beam. The control system can preset the emission angle parameters for each channel or dynamically adjust them based on a material surface topography model, allowing the modulated laser beam to illuminate multiple target surfaces in different directions.

[0100] This structure enables a single laser light source to serve multiple inspection paths, and the micromirror array ensures that each path can independently control its illumination angle, thereby achieving a flexible multi-angle inspection lighting layout suitable for complex material inspection tasks with height changes, inclined structures or non-flat surfaces.

[0101] A dark field illumination structure is provided on the emission path of the modulated laser beam, the dark field illumination structure comprising an annular light source and an annular aperture, for illuminating the material surface at an oblique angle and blocking axial reflected light;

[0102] The dark field illumination structure includes an annular light source component, an annular aperture component and a polarization control component.

[0103] The annular light source assembly is configured as a coaxial annular structure symmetrically distributed around the axis of the laser beam. The light source is a plurality of directional light source units arranged in an array, which may include an LED light source or a laser array. The light source units are all set to a fixed inclined incident angle θ, which is used to make the irradiation beam incident on the material surface in a non-axial manner to achieve dark field illumination conditions;

[0104] The annular aperture assembly is coaxially arranged in front of the annular light source's exit path, and has a central light-shielding area and an outer ring light-transmitting area. This structure blocks the axial strong light reflected vertically from the material surface, allowing only light scattered or refracted at an oblique angle to enter the subsequent detection path, thereby enhancing the response to surface defect micro-scattering.

[0105] A polarization control assembly is placed between the annular light source assembly and the material surface. This assembly, comprising a rotatable polarizer or liquid crystal tunable polarizer, adjusts the polarization state of the incident light beam. This assembly can achieve linear polarization, circular polarization, or any polarization angle to accommodate the reflective properties and scattering directionality of different material surfaces.

[0106] The polarization control component can also be controlled in conjunction with the image processing module to select the optimal polarization parameters based on image contrast feedback to improve the visualization of specific defect types (such as small cracks and directional scratches) in imaging.

[0107] The three components are connected by structural sockets or coaxially assembled by brackets to form a multi-level optical path control structure as a whole. The modulated laser beam passes through the dark-field illumination structure and irradiates the material surface in an inclined and polarization-controlled manner. The main axial reflected light is shielded by the annular aperture, while the non-axial scattered light generated by micro-defects is allowed to pass, providing optimized lighting conditions for the subsequent image acquisition module.

[0108] A near-field illumination structure, disposed between the dark-field illumination structure and the material surface, for limiting the size of the illumination beam to a sub-wavelength scale;

[0109] In a preferred embodiment, the near-field illumination structure includes:

[0110] Near-field lens array assembly: comprising multiple nanostructured microlenses or collimated nanofiber arrays, the lens array is arranged on the laser emission path and is used to compress and focus the modulated laser beam to a sub-wavelength spot diameter range, thereby enhancing the light density of the irradiated area;

[0111] Spot size control module: integrated with the lens array assembly, including a focus-adjustable liquid crystal lens or a piezoelectric-driven microstructure assembly, for dynamically adjusting the beam exit angle or focus depth, thereby achieving continuous adjustment of the spot size within a set range (e.g., 200nm-1μm);

[0112] Light Field Calibration Module: This module includes a set of auxiliary detectors for irradiation light shape monitoring and a light field reconstruction algorithm. It is used to dynamically correct the shape, symmetry, and deviation of the light spot output by the near-field focusing structure after system initialization or changes in the temperature drift environment. The light field calibration module is connected to the control system and controls the focusing module to adjust parameters in real time based on feedback data.

[0113] Temperature drift compensation mechanism: It integrates a thermistor or micro temperature sensor, and a compensation adjustment unit to detect the ambient temperature near the near-field lens array in real time. By adjusting the piezoelectric drive structure or liquid crystal focusing parameters, it compensates for focusing deviations caused by factors such as thermal expansion and optical refractive index drift to ensure illumination stability.

[0114] In the above structure, the nanolens array is responsible for achieving subwavelength focusing, the control module gives the system the ability to change the light spot, the calibration module ensures that the light field morphology of the irradiated area meets the accuracy requirements, and the temperature compensation mechanism ensures that the light spot parameters do not drift with environmental changes during long-term operation.

[0115] Through the multifunctional integrated design of the near-field illumination structure, the system can flexibly adjust illumination parameters in different detection tasks, ensuring high-resolution illumination accuracy and improving detection sensitivity.

[0116] Industrial visual inspection module, which is set above or to the side of the material surface to collect material surface images and identify image abnormal areas caused by surface defects;

[0117] In a preferred embodiment, the industrial visual inspection module includes an industrial camera, an image acquisition device, and an image processing terminal. The industrial camera is arranged above or to the side of the inspection area and is used to acquire high-resolution images of the material surface under lighting conditions.

[0118] The image acquisition device, located between the industrial camera and the image processing terminal, includes an image buffer unit and a data transmission interface. The image buffer unit receives and temporarily stores the image frame data continuously output by the industrial camera to mitigate data rate differences between the camera and the back-end processing unit. The data transmission interface, using PCIe, Gigabit Ethernet, or other high-speed interface protocols, transmits the buffered image data to the image processing terminal in a predefined frame sequence and structured format.

[0119] The image processing terminal is configured to receive image data transmitted by the image acquisition device and perform decoding and image analysis processing. The image processing terminal includes an image decoding module, which is used to identify and analyze the frame sequence number, time stamp identifier, and compression format contained in the image data to complete the decoding and restoration operations of the image stream;

[0120] An image processing module, electrically connected to the industrial visual inspection module, for performing enhancement processing, defect feature extraction and spatial modeling on the collected images;

[0121] In a preferred embodiment, the image processing module includes an image calculation module, which is used to perform image preprocessing operations (such as denoising, enhancement, histogram equalization, etc.) on the decoded image data, as well as analysis operations such as defect edge extraction based on the image area and feature parameter calculation.

[0122] The image decoding module and the image calculation module work together to ensure the time synchronization of received images, the integrity of image structure, and the continuity of analysis and processing. This structure supports the stable transmission of image input streams from high-frame-rate industrial cameras and the hierarchical execution of high-performance image processing tasks.

[0123] The image calculation module includes the following multiple functional sub-units, including an image pre-processing sub-module, a defect feature extraction sub-module and a defect classification and recognition sub-module;

[0124] The image preprocessing submodule is used to perform image noise suppression and detail enhancement processing. In a specific embodiment, a bilateral filtering algorithm is used to remove noise from the original image while retaining edge information.

[0125] The calculation formula of the bilateral filtering algorithm is:

[0126]

[0127] in:

[0128] I(x): grayscale value of the original image;

[0129] I′(x): pixel value after filtering;

[0130] f r : Grayscale similarity function, suppressing noise;

[0131] f s : spatial weight function, preserving edge structure;

[0132] Ω: pixel neighborhood window;

[0133] W p : Normalization factor.

[0134] The defect feature extraction submodule performs edge detection and morphological analysis on the pre-processed image. In this embodiment, the Canny edge detection algorithm is used to identify defect edges.

[0135] The calculation steps are as follows:

[0136] 1. Perform Gaussian filtering and smoothing on the image;

[0137] 2. Calculate the image gradient magnitude and direction:

[0138]

[0139] Parameter explanation:

[0140] x, y: pixel coordinates in the image;

[0141] G x : Image gradient in the x direction (usually approximated using the Sobel operator or Roberts operator);

[0142] G y : image gradient in the y direction;

[0143] G(x, y): The gradient magnitude at the pixel point (x, y), indicating the edge strength;

[0144] θ(x, y): The gradient direction at the pixel point (x, y), which represents the normal angle of the edge (used for subsequent non-maximum suppression);

[0145] This formula is used to calculate the edge response and direction of each point in the image and is the core part of Canny edge detection.

[0146] 3. Non-maximum suppression and double threshold edge connection to obtain closed boundary contour;

[0147] 4. Calculate defect characteristic parameters based on the closed area, such as area A, perimeter C, and aspect ratio R:

[0148]

[0149] Parameter explanation:

[0150] Region: The defect region formed by edge closure (i.e., the target connected domain in the image);

[0151] L max : The length of the long side of the rectangle surrounding the region;

[0152] L min : The length of the short side of the rectangle surrounding the region;

[0153] R: Aspect ratio, a measure of the degree of stretch or extension of the defect (used to distinguish between "scratch" and "point" defects);

[0154] A: Area, which represents the number of pixels of the defect (i.e., the number of points with a pixel value of 1 in the area);

[0155] The unit sum of all pixels in the defect area is the area of ​​the area;

[0156] These parameters are used as input to the subsequent defect classification model or to estimate the defect volume in the 3D modeling module.

[0157] The defect classification and recognition submodule uses a lightweight convolutional neural network (such as MobileNet) to classify the extracted defect areas. The network output is the category label y∈{scratch, pit, particle...} and the confidence p∈[0, 1].

[0158] The classification loss function uses cross entropy loss:

[0159]

[0160] in:

[0161] C: number of categories;

[0162] y i : one-hot vector of the true label;

[0163] The network predicts probabilities.

[0164] The network structure can be deployed in the GPU inference engine of the image processing terminal, supporting edge AI computing or server-side batch inference.

[0165] Through the collaborative processing of the above three functional sub-modules, the system can complete the complete process from image denoising, edge detection to defect area identification and classification, forming a highly automated and scalable industrial visual inspection capability.

[0166] A data processing module is electrically connected to the image processing module. The data processing module includes a programmable logic device and a graphics processing unit. The programmable logic device is used for synchronous timing and control of image data. The graphics processing unit is used to analyze the position, size and morphology of material surface defects based on the spatial distribution, grayscale characteristics and time characteristics of abnormal image areas.

[0167] In a preferred embodiment, a joint modeling interface is provided between the image processing module and the data processing module to support the coordinated processing of defect image information and spatial structure information.

[0168] The image processing module includes a 3D defect reconstruction unit, which receives defect area images output by the image preprocessing module and the defect feature extraction module and generates a corresponding depth map or point cloud model based on the material surface lighting conditions or viewing angle information. This unit can construct a preliminary 3D topography model based on multi-frame visual structure matching, form-light inversion calculation, or surface reflection phase difference estimation methods.

[0169] The data processing module is equipped with a 3D analysis unit. The 3D analysis unit obtains the 3D defect model data output by the image processing module through the joint modeling interface and performs geometric parameter calculations on it. The 3D analysis unit can use model boundary information, spatial grayscale gradient, or depth mapping function to output the following defect structure parameters:

[0170] Maximum depth or height of the defect;

[0171] Defect profile inclination angle;

[0172] defect envelope volume;

[0173] Defect edge fluctuation parameters (such as curvature radius).

[0174] The above-mentioned defect spatial characteristics can be further formatted and stored, and output to the display interface, data archiving system or process feedback module for subsequent quality analysis, grading determination and defect tracing.

[0175] The joint modeling interface can adopt a shared memory interface or an intermediate data caching mechanism to support multi-threaded data exchange and ensure the consistency and real-time performance of image and spatial data.

[0176] This structure improves the system's modeling accuracy and geometric quantification capabilities when dealing with three-dimensional defects with morphological characteristics. It is suitable for application scenarios where structured modeling and classification output of defects such as pits, protrusions, indentations, and wrinkles on the material surface are required.

[0177] In a further preferred embodiment, the defect three-dimensional reconstruction unit includes a multi-angle image reconstruction module, which is used to construct a defect three-dimensional model based on image data from multiple shooting angles.

[0178] The multi-angle image reconstruction module is connected to multiple industrial cameras, which are arranged at preset angles relative to the surface of the material being tested to form a multi-baseline stereo imaging structure for simultaneously or sequentially acquiring image data of the defect area at different shooting angles.

[0179] The multi-angle image reconstruction module receives images captured by each industrial camera and performs stereo image matching operations to extract pixel-level corresponding point displacements between images and calculate disparity information between images;

[0180] Based on the relationship between the disparity value and the camera's internal and external parameters, the three-dimensional point cloud data or depth map corresponding to the defect area is generated, and then the spatial morphology model of the defect is constructed.

[0181] The reconstruction module can also integrate an image alignment module to perform geometric correction and registration on multi-camera images, improve pixel matching accuracy, and avoid reconstruction errors caused by perspective offset.

[0182] This structure can restore the true three-dimensional spatial morphology when there are complex surface structures, angle changes or depth details in the defect area, enhancing the system's detection adaptability and morphological quantification capabilities for non-planar defects.

[0183] In a preferred embodiment, the data processing module includes a defect level assessment unit and an early warning control unit. The defect level assessment unit calculates a score based on the defect space parameters (such as area A, maximum depth D, boundary length L) and identification type (such as crack, pit, foreign matter) output by the image processing module to form a defect level identifier G∈{1, 2, 3, ..., n};

[0184] The early warning control unit receives the level identifier and compares it with the classification threshold set by the system. When the level reaches or exceeds the alarm level, an alarm signal is sent to the control system or operation interface.

[0185] Early warning information may include alarm sounds, image annotations, MES system interface signals, etc., which are suitable for automatic rejection, shutdown re-inspection or quality traceability processes.

[0186] In summary, the embodiments of the present invention, through the collaborative design of image processing and data processing modules, have constructed an intelligent industrial visual inspection system for high-resolution image input. Within the image processing module, a 3D defect reconstruction unit is introduced, along with a multi-angle image reconstruction module. This module, combined with multi-camera parallax information, enables spatial modeling of defect areas, enhancing the ability to identify complex structural defects in three dimensions.

[0187] In the data processing module, a heterogeneous computing platform (programmable logic devices and graphics processing units) is deployed to achieve functional decoupling and parallel acceleration of the image processing chain, meeting the system's requirements for high frame rate, low latency, and real-time computing. At the same time, a defect grade assessment unit and an early warning control unit are installed, enabling the system to automatically complete grading judgments based on parameters such as defect size, type, and depth, and issue alarm signals based on set thresholds, establishing a closed-loop quality control capability from "defect detection" to "intelligent feedback."

[0188] The solution of the present invention has a clear structural hierarchy and a complete processing path. It is suitable for various material inspection scenarios with strict requirements on surface quality, especially under conditions of high-speed production lines, multiple defect types and complex lighting backgrounds, and has good adaptability and scalability.

[0189] The above-described embodiments do not constitute a limitation on the scope of protection of this technical solution. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the above-described embodiments shall be included in the scope of protection of this technical solution.

Claims

1. A material surface defect detection system based on industrial visual inspection, characterized in that: include: A laser light source module, comprising a plurality of wavelength-tunable lasers for generating a modulated laser beam; A beam distribution assembly, connected to the laser light source module, for directing the modulated laser beam to the surface of the material in multiple directions; A dark field illumination structure is provided on the emission path of the modulated laser beam, the dark field illumination structure comprising an annular light source and an annular aperture, for illuminating the material surface at an oblique angle and blocking axial reflected light; A near-field illumination structure, disposed between the dark-field illumination structure and the material surface, for limiting the size of the illumination beam to a sub-wavelength scale; Industrial visual inspection module, which is set above or to the side of the material surface to collect material surface images and identify image abnormal areas caused by surface defects; An image processing module, electrically connected to the industrial visual inspection module, for performing enhancement processing, defect feature extraction and spatial modeling on the collected images; A data processing module is electrically connected to the image processing module. The data processing module includes a programmable logic device and a graphics processing unit. The programmable logic device is used for synchronous timing and control of image data. The graphics processing unit is used to analyze the position, size and morphology of material surface defects based on the spatial distribution, grayscale characteristics and time characteristics of abnormal image areas.

2. The material surface defect detection system based on industrial visual inspection according to claim 1 is characterized in that: The laser light source module includes a laser array structure composed of a plurality of laser emitting units, wherein the laser emitting units are linearly arranged along the width direction of the material surface and configured to generate parallel modulated laser beams to irradiate multiple partitions of the material surface; The laser light source module further includes an automatic light intensity compensation control unit, which is electrically connected to each laser emitting unit and the image processing module, and is used to adjust the output power of each laser emitting unit according to the regional image brightness information provided by the image processing module; The automatic light intensity compensation control unit independently controls the power output of different laser emitting units based on a preset brightness threshold and partition parameters.

3. The material surface defect detection system based on industrial visual inspection according to claim 1 is characterized in that: The beam distribution assembly includes a multi-channel beam splitting structure and a micromirror array structure. The multi-channel beam splitting structure includes a plurality of beam splitting optical elements for splitting the modulated laser beam generated by the laser light source module into a plurality of sub-beam channels. The micromirror array structure is arranged on the emission path of each sub-beam channel, and includes a plurality of independently drivable micro-electromechanical mirror units for adjusting the emission direction and illumination angle of each sub-beam respectively. The multi-channel beam splitting structure is connected to the micromirror array structure. Each reflector unit of the micromirror array structure corresponds one-to-one to the corresponding sub-beam channel, forming a controllable multi-path beam output system for guiding multiple modulated laser beams to multiple detection areas on the material surface respectively.

4. The material surface defect detection system based on industrial visual inspection according to claim 1 is characterized in that: The industrial visual inspection module includes an industrial camera, an image acquisition device and an image processing terminal. The industrial camera is used to acquire images of the material surface. The image acquisition device is used to receive and cache the data output by the camera, and perform format conversion and time stamping. The image processing terminal is used to decode, preprocess and extract defect features of the received images.

5. The material surface defect detection system based on industrial visual inspection according to claim 4 is characterized in that: The image acquisition device includes an image cache unit and a data transmission interface, wherein the image cache unit is used to temporarily store image frame data transmitted by the industrial camera, and the data transmission interface is used to transmit the cached image data to the image processing terminal in a preset frame sequence structure; The image processing terminal includes an image decoding module, which is used to parse image frame sequence information and time stamp identifiers, and decode image data into a processable image format.

6. The material surface defect detection system based on industrial visual inspection according to claim 1 is characterized in that: The image processing module includes an image calculation module, which is used to perform image preprocessing operations based on the decoded image data and extract and encode regional defect features contained in the material surface image; The image calculation module is configured with multiple functional sub-units, including an image pre-processing sub-module, a defect feature extraction sub-module, and a defect classification and identification sub-module; The image preprocessing submodule is used to perform image noise suppression and detail enhancement processing, the defect feature extraction submodule performs edge detection and morphological analysis on the preprocessed image; the defect classification and recognition submodule uses a lightweight convolutional neural network to classify the extracted defect areas.

7. The material surface defect detection system based on industrial visual inspection according to claim 6, characterized in that: A joint modeling interface is provided between the image processing module and the data processing module. The image processing module includes a defect 3D reconstruction unit, which is used to generate a corresponding defect depth map or point cloud model based on the defect area image output by the image processing module and the spatial position information provided by the data processing module; The data processing module includes a three-dimensional analysis unit, which is used to calculate geometric morphology parameters of the defect depth map or point cloud model and output the spatial size, surface inclination angle and volume estimation results of the defect.

8. The material surface defect detection system based on industrial visual inspection according to claim 7 is characterized in that: The defect 3D reconstruction unit further includes a multi-angle image reconstruction module, The multi-angle image reconstruction module is connected to multiple industrial cameras set at different shooting angles, and is used to receive defect area images collected at multiple angles, and construct corresponding three-dimensional point cloud models or depth map data based on the parallax information between the images. The inter-image disparity information includes the displacement relationship between pixel-level corresponding points in images of different viewing angles.

9. The material surface defect detection system based on industrial visual inspection according to claim 1, characterized in that: The data processing module further includes a defect level assessment unit and an early warning control unit. The defect level assessment unit is used to grade defects according to parameters such as the area, depth, and type of the defect area; the early warning control unit determines whether to trigger an alarm instruction based on a preset level threshold and transmits the judgment result to the upper control system or display terminal.

Images