Multi-source fusion visual inspection method and system for weld surface defects

Abstract

This invention discloses a multi-source fusion visual inspection method and system for weld surface defects, belonging to the field of welding quality inspection technology. The method includes: sequentially applying bright-field illumination, dark-field illumination, and structured light projection illumination to the weld area to be inspected; acquiring corresponding weld images under each illumination condition to obtain multimodal image data; performing phase calculation based on the structured light image data to obtain three-dimensional height distribution data and extracting the contour curve of the weld cross-section; calculating the curvature distribution based on the contour curve to construct a curvature spectrum feature characterizing the geometric changes in the weld cross-section morphology; constructing an ideal phase distribution based on the image data acquired under structured light projection illumination; calculating the difference between the actual phase distribution and the ideal phase distribution to obtain the phase residual; and fusing the curvature spectrum feature and the phase residual across domains to construct a joint discrimination rule for identifying weld surface defects. This invention can improve the accuracy and reliability of weld quality inspection.

Description

Technical Field

[0001] This invention relates to the field of welding quality inspection technology, and in particular to a multi-source fusion visual inspection method and system for weld surface defects. Background Technology

[0002] Surface defect detection in welds is a crucial step in ensuring the safety and reliability of welded structures. Common weld surface defects include cracks, porosity, and undercut, with cracks being one of the most dangerous. Geometric undulations such as weld scales and variations in weld reinforcement are considered normal weld morphology and should not be classified as defects. However, in actual inspections, cracks and geometric undulations like weld scales can exhibit similar abrupt geometric changes in two-dimensional or three-dimensional images, making it difficult for existing detection methods to effectively distinguish between the two, resulting in a high false alarm rate.

[0003] Currently, weld surface defect detection methods are mainly divided into two categories. One category is geometric measurement methods based on structured light 3D reconstruction. For example, some studies use line structured light to acquire weld contour data and use methods such as second derivative extremum calculation, curve fitting, and variance analysis to solve the weld geometry in real time during scanning, which can distinguish defects such as porosity, cracks, and undercut. Other studies further utilize line structured light 3D point cloud data, enhance the point cloud through power law transformation and map it into a depth image, and combine it with an improved YOLOv8 network for defect identification. The other category is 2D image detection methods based on bright or dark field illumination, which use grayscale features to identify surface defects. For example, some studies have proposed an improved YOLOv5 model (introducing an attention mechanism, bidirectional feature pyramid, and optimized loss function) to improve detection accuracy while solving the problems of uneven illumination and noise interference. Other studies have adopted a 2D / 3D heterogeneous dual-path detection strategy, fusing RGB-D images and point cloud slice contour lines to accurately locate defects such as undercut. However, the above methods still have shortcomings: pure geometric measurement methods are sensitive to geometric morphology and are prone to misjudging normal undulations such as fish scale patterns as cracks; two-dimensional image methods are easily affected by lighting conditions, surface reflection, rust, etc., and are not sensitive to the small openings of cracks, resulting in a high false negative rate; deep learning methods rely on a large number of labeled samples, consume a lot of computational resources, and the black box model lacks interpretability and has limited generalization ability for different welding processes and surface conditions.

[0004] In recent years, some studies have attempted to fuse two-dimensional images with three-dimensional topographic data, but most have only focused on simple data overlay or feature stitching, failing to uncover the fundamental differences between geometric topography and optical response from a physical mechanism perspective. For example, cracks not only cause local geometric abrupt changes but also produce strong optical scattering due to material discontinuities; while the geometric undulations of welds also alter the surface profile, the material remains continuous, resulting in a relatively smooth optical response. Existing technologies lack effective modeling and utilization of these physical differences, leading to insufficient ability of detection methods to distinguish between different types of anomalies.

[0005] Therefore, there is an urgent need for a weld surface defect detection method that can integrate geometric morphology and optical response characteristics, and distinguish between crack-like defects and weld geometric undulations from a physical mechanism perspective, in order to reduce false alarm rate and false negative rate, and improve the accuracy and reliability of weld quality inspection. Summary of the Invention

[0006] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a multi-source fusion visual inspection method for weld surface defects, comprising the following steps:

[0008] Bright field illumination, dark field illumination, and structured light projection illumination are applied sequentially to the weld area to be inspected, and corresponding weld images are acquired under each illumination condition to obtain multimodal image data for industrial vision analysis.

[0009] Phase calculation is performed on the image data acquired under the structured light projection illumination to obtain the three-dimensional height distribution data of the weld surface, and the contour curve of the weld cross section is extracted based on the three-dimensional height distribution data.

[0010] Based on the contour curve, the curvature distribution is calculated, and a curvature spectrum feature characterizing the geometric changes in the cross-sectional morphology of the weld is constructed.

[0011] Based on the image data acquired under the structured light projection illumination, a corresponding ideal phase distribution is constructed, and the difference between the actual phase distribution and the ideal phase distribution is calculated to obtain the phase residual characterizing the local structural changes on the weld surface.

[0012] The curvature spectrum features and the phase residuals are fused across domains to construct a joint discrimination rule for identifying surface defects in the weld. The joint discrimination rule can distinguish between crack-like defects, weld geometric undulations, and normal areas.

[0013] As a preferred embodiment of the multi-source fusion visual inspection method for weld surface defects described in this invention, the construction of the curvature spectrum features includes:

[0014] Obtain the discrete sampling point set of the contour curve;

[0015] The first and second derivatives at each sampling point are calculated using a local window curve fitting method to obtain the curvature value of each sampling point;

[0016] The curvature value is smoothed using a sliding window.

[0017] The smoothed curvature values ​​are arranged according to the cross-sectional direction of the weld to form a curvature distribution sequence;

[0018] The set of curvature extrema, the rate of change of curvature, and the periodic parameters of the curvature distribution are extracted from the curvature distribution sequence and vectorized to form the curvature spectrum features.

[0019] As a preferred embodiment of the multi-source fusion visual inspection method for weld surface defects described in this invention, the calculation of the phase residual includes:

[0020] An ideal phase distribution model is established based on the preset parameters of the structured light projection fringes, and the ideal phase distribution is obtained based on the ideal phase distribution model.

[0021] The actual phase distribution is calculated from the image data acquired under the structured light projection illumination using a multi-step phase-shifting method.

[0022] The phase residual distribution is obtained by performing point-by-point difference calculation between the actual phase distribution and the ideal phase distribution;

[0023] The phase residual distribution is spatially filtered, and continuous abnormal regions are extracted to obtain the phase residual.

[0024] As a preferred embodiment of the multi-source fusion visual inspection method for weld surface defects described in this invention, the joint discrimination rule includes:

[0025] Curvature anomaly indices are constructed based on the curvature spectrum features respectively. A phase anomaly index is constructed based on the phase residual. ;

[0026] Regarding the curvature anomaly index Phase anomaly index Normalization was performed to obtain the normalized curvature anomaly index. With normalized phase anomaly index ;

[0027] Preset first threshold Second threshold The first threshold Used to determine the normalized phase anomaly index The second threshold Used to determine the normalized curvature anomaly index And the first threshold and the second threshold Determined through the calibration process;

[0028] Based on the normalized curvature anomaly index Normalized phase anomaly index First threshold Second threshold Perform stratification:

[0029] when When this occurs, it is determined to be a normal area;

[0030] when and At that time, it was determined to be a crack-type defect;

[0031] when and When this occurs, it is determined to be a geometric undulation of the weld.

[0032] As a preferred embodiment of the multi-source fusion visual inspection method for weld surface defects described in this invention, the method involves: after acquiring multimodal image data, spatially registering the images acquired under bright field illumination, dark field illumination, and structured light projection illumination to ensure that the three correspond in pixel coordinates; and using the registered bright field image or dark field image to perform texture verification on the three-dimensional height distribution data.

[0033] As a preferred embodiment of the multi-source fusion visual inspection method for weld surface defects described in this invention, the method involves: after obtaining the three-dimensional height distribution data, sequentially performing filtering and outlier removal processing on the three-dimensional height distribution data to obtain a processed height distribution surface; and extracting a contour curve based on the height distribution surface.

[0034] As a preferred embodiment of the multi-source fusion visual inspection method for weld surface defects described in this invention, the preset range of the local window is adaptively adjusted according to the sampling density in the cross-sectional direction of the weld, and the window size of the sliding window is dynamically determined based on the local rate of change of the curvature distribution.

[0035] As a preferred embodiment of the multi-source fusion visual inspection method for weld surface defects described in this invention, the multi-step phase shifting method uses at least four stripe images with different phase offsets for phase calculation; the spatial filtering process uses adaptive median filtering or wavelet filtering.

[0036] As a preferred embodiment of the multi-source fusion visual inspection method for weld surface defects described in this invention, wherein: the first threshold Second threshold The determination is made by calibrating samples, and the calibration includes:

[0037] Collect multiple sets of weld sample data, including crack-like defects, weld geometry undulations, and normal areas;

[0038] Curvature spectrum features and phase residual distributions of each sample are extracted to construct a curvature anomaly index. Phase anomaly index ;

[0039] Based on the statistical results of the characteristics of various samples, the value range of each indicator is determined;

[0040] The first threshold is optimized by combining interval boundary analysis or the principle of minimizing distribution overlap with statistical learning methods. Second threshold .

[0041] This invention also provides a multi-source fusion visual inspection system for weld surface defects, applied to the aforementioned multi-source fusion visual inspection method for weld surface defects, comprising:

[0042] Multi-source lighting module, used to provide bright field lighting, dark field lighting and structured light projection lighting;

[0043] The image acquisition module is used to acquire weld images corresponding to the weld area under various lighting conditions to obtain multimodal image data;

[0044] The three-dimensional reconstruction module is used to perform phase calculation based on the image data acquired under the structured light projection illumination to obtain the three-dimensional height distribution data of the weld surface;

[0045] The filtering and removal module is used to sequentially filter and remove outliers from the three-dimensional height distribution data to obtain the processed height distribution surface.

[0046] The curvature analysis module is used to extract the contour curve of the weld cross section based on the height distribution surface and construct curvature spectrum features;

[0047] The phase residual calculation module is used to establish an ideal phase distribution based on the preset parameters of the structured light projection fringes, solve the actual phase distribution using the multi-step phase shift method, calculate the difference between the actual phase distribution and the ideal phase distribution, and obtain the phase residual.

[0048] The registration and verification module is used to spatially register images acquired under bright field illumination, dark field illumination, and structured light projection illumination, so that the three correspond in pixel coordinates, and to use the registered bright field image or dark field image to verify the texture of the three-dimensional height distribution data.

[0049] The defect discrimination module is used to perform cross-domain fusion of the curvature spectrum features and the phase residual, and to identify weld surface defects based on the joint discrimination rules, distinguishing between crack-type defects, weld geometric undulations and normal areas.

[0050] The beneficial effects of this invention are:

[0051] 1. This invention sequentially applies bright field illumination, dark field illumination, and structured light projection illumination to the weld area to be inspected, thereby acquiring the macroscopic morphology, surface microstructure defects, and phase encoding information of the weld. Then, based on the structured light image, a three-dimensional height distribution is reconstructed and curvature spectrum features (characterizing geometric morphological abrupt changes and periodicity) are extracted. At the same time, an ideal phase distribution is constructed and the phase residual (characterizing optical scattering anomalies) is calculated. Finally, the curvature spectrum features and phase residual are fused across domains to construct a joint discrimination rule, which realizes the fine distinction between crack-like defects, weld geometric undulations, and normal areas. This solves the technical problem that existing single-modal detection methods cannot effectively distinguish between cracks and normal undulations such as fish scale patterns, and significantly reduces the false alarm rate and false negative rate.

[0052] 2. In the joint discrimination rule of this invention, curvature anomaly indicators and phase anomaly indicators are constructed based on curvature spectrum features and phase residuals, respectively. These indicators are then normalized, and a dual-threshold hierarchical judgment strategy is employed for defect classification. Specifically, normal regions are first screened based on the normalized curvature anomaly indicators, and then crack-like defects and weld geometric undulations are identified based on the normalized phase anomaly indicators. This rule fully utilizes the sensitivity of curvature spectrum features to changes in weld geometric morphology and the sensitivity of phase residuals to local structural anomalies. It maintains a stable recognition accuracy for weak-contrast cracks and weld surfaces with complex periodic undulations. Furthermore, the first and second thresholds are predetermined through offline calibration, eliminating the need for online training for each weld. This allows for direct application to industrial sites with different welding processes, weld types, and lighting conditions, significantly improving the versatility and practical deployment efficiency of the detection system.

[0053] 3. After acquiring multimodal images, this invention performs spatial registration of bright-field, dark-field, and structured light images, and uses the registered bright-field or dark-field images to verify the texture of the three-dimensional height distribution data: By mapping the marked abnormal candidate regions in the three-dimensional data to the two-dimensional image for grayscale cross-validation, false defects caused by interference such as three-dimensional reconstruction noise or surface reflection are effectively eliminated. At the same time, the height distribution surface after filtering and abnormal point removal is used for contour extraction, further ensuring the accuracy and reliability of curvature spectrum features. This forms a complete closed-loop detection chain from image acquisition, three-dimensional reconstruction, feature extraction to cross-domain fusion, significantly improving the accuracy of weld surface defect detection and industrial site adaptability. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0055] Figure 1 This is a flowchart illustrating the overall process of the multi-source fusion visual inspection method for weld surface defects according to the present invention.

[0056] Figure 2 This is a flowchart illustrating the curvature spectrum feature construction process of the multi-source fusion visual inspection method for weld surface defects according to the present invention.

[0057] Figure 3 This is a flowchart illustrating the phase residual calculation process of the multi-source fusion visual inspection method for weld surface defects according to the present invention.

[0058] Figure 4 This is a flowchart illustrating the spatial registration and texture verification process of the multi-source fusion visual inspection method for weld surface defects according to the present invention.

[0059] Figure 5 This is a flowchart illustrating the filtering and anomaly removal process of the multi-source fusion visual inspection method for weld surface defects according to the present invention. Detailed Implementation

[0060] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0061] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0062] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0063] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include three-dimensional spatial dimensions of length, width, and depth.

[0064] Example 1

[0065] Reference Figure 1-5 The first embodiment of the present invention provides a multi-source fusion visual inspection method for weld surface defects, comprising the following steps:

[0066] S1. Apply bright field illumination, dark field illumination and structured light projection illumination to the weld area to be inspected in sequence, and collect the corresponding weld images under each illumination condition to obtain multimodal image data. The multimodal image data is used for industrial vision analysis.

[0067] In one specific implementation, the bright-field lighting uses a ring-shaped white LED light source, with an angle of [angle] with the normal to the weld surface. Irradiating the weld area in a specific direction produces uniform reflection on the flat areas of the weld surface, highlighting the overall outline and macroscopic shape of the weld.

[0068] In one specific implementation, the dark field illumination uses a low-angle line light source, with an angle of [angle not specified] with the normal to the weld surface. The light grazes the weld surface in a direction that causes the tiny bumps, scratches, and crack edges on the weld surface to produce strong scattered light, while the flat areas remain dark, thus enhancing the contrast of defects.

[0069] In one specific implementation, structured light projection illumination uses a digital projector to project a phase-shifted fringe pattern onto the weld surface, with a fringe period... According to the weld width Determine using the following formula: ,in, The number of pixels corresponding to each stripe period ranges from 5 to 15. For example, when the weld width... When = 8mm, take =8, then =1mm; when the weld width When = 50mm, take =5, =10mm. The angle between the projection direction and the normal to the weld surface is 30~60° to avoid specular reflection interference.

[0070] In one specific implementation, image acquisition uses an industrial area scan camera with a resolution of no less than 5 megapixels and a frame rate of ≥30fps. A resolution of 5 megapixels (approximately 2500×2000) provides a lateral resolution of 0.008mm / pixel with a typical field of view of 20mm, sufficient to distinguish microcracks wider than 0.05mm; a frame rate of 30fps meets the requirements for subsequent acquisition intervals.

[0071] In one specific implementation, the acquisition sequence of bright field illumination, dark field illumination, and structured light projection illumination follows the order of "bright field → dark field → structured light," using a single industrial camera, with each acquisition interval... The time should not exceed 0.5 seconds to reduce spatial position deviation of the weld caused by thermal deformation or workpiece movement. According to the welding travel speed and camera field of view Sure: For example, when =10mm / s When =20mm, ≤1 second, this embodiment uses 0.5 seconds to ensure registration redundancy. The acquired bright-field image, dark-field image, and structured light fringe image are respectively denoted as... , and ,in These are the pixel coordinates of the image.

[0072] In another implementation, to improve acquisition efficiency, three cameras can be used to simultaneously acquire bright-field, dark-field, and structured-light images. The three light sources are sequentially and rapidly illuminated via hardware trigger signals (each illumination time < 10ms), and the three cameras expose synchronously at their respective illumination times, still falling under the category of sequential illumination. In this case, the single acquisition time can be shortened to the longest single-channel exposure time (typically < 50ms). This implementation is suitable for online production scenarios with high requirements for inspection cycle speed.

[0073] Furthermore, after acquiring multimodal image data, spatial registration is performed on the images acquired under bright field illumination, dark field illumination, and structured light projection illumination, so that the three correspond in pixel coordinates.

[0074] In one specific implementation, spatial registration employs a homography transformation method based on a calibration board: a planar calibration board with a dot array is pre-imaged, and images are acquired under bright field, dark field, and structured light illumination, respectively. The centers of the dots are extracted as feature points, and the homography matrix from the bright field image to the structured light image is calculated for each. → And the homography matrix from dark field image to structured light image. → For each set of images actually detected, the bright field image will be... and dark field images The registered image is obtained by mapping the homography matrix onto the coordinate system of the structured light image. and The registration accuracy is controlled within 0.5 pixels. The registered image is used for subsequent texture verification.

[0075] It should be noted that by sequentially applying bright-field illumination, dark-field illumination, and structured light projection illumination and acquiring images, bright-field images for characterizing the macroscopic morphology of the weld, dark-field images for highlighting surface microstructural defects, and structured light images carrying fringe phase distribution information (images acquired under structured light projection illumination) were obtained. These images together constitute multimodal image data. Next, based on the acquired structured light images, phase resolution technology will be used to reconstruct the three-dimensional height distribution of the weld surface, so as to extract geometric morphology features and perform defect classification and identification.

[0076] S2. Based on the image data acquired under the structured light projection illumination, perform phase calculation to obtain the three-dimensional height distribution data of the weld surface, and extract the contour curve of the weld cross section according to the three-dimensional height distribution data.

[0077] In one specific implementation, phase calculation is achieved using a four-step phase shift method combined with a multi-frequency heterodyne method. The specific steps are as follows:

[0078] First, four sinusoidal fringe images with phase shifts of 0, π / 2, π, and 3π / 2 are projected, and four fringe patterns are acquired. , , , Wrap phase Calculate using the following formula: Because the package phase is truncated in For intervals, phase unwrapping is required. For example, projecting three different frequency fringes (frequency ratio 1:8:64) yields the corresponding wrapper phases, which are then synthesized step-by-step using the heterodyne principle to obtain the absolute phase. Specifically, the wrapper phase with frequency 8 is heterodilated with the wrapper phase with frequency 1 to obtain the equivalent phase with frequency 7, which is then heterodilated with the wrapper phase with frequency 64 to finally obtain the absolute phase of the entire field.

[0079] Next, the absolute phase is mapped to a three-dimensional height. The reference phase is obtained by pre-measuring a reference plane (a standard plane without weld seams) under the same camera field of view and projection configuration. Then the weld surface height The relationship with absolute phase is: ,in This is the calibration factor (unit: mm / rad), obtained by measuring a standard gauge block of known height. For example, when the fringe period is 2 mm, it is obtained through a calibration experiment. Approximately 0.15 mm / rad. The final three-dimensional height distribution data of the weld surface was obtained. .

[0080] Furthermore, the registered bright-field or dark-field images are used to perform texture verification on the three-dimensional height distribution data.

[0081] In one specific implementation, texture verification is performed after obtaining three-dimensional height distribution data. Then proceed. The specific steps are as follows:

[0082] First of all, The abnormal candidate regions are marked. The marking criteria include:

[0083] I. The local height gradient magnitude exceeds a preset threshold (e.g., 0.1 mm / mm). Specifically, the local height gradient magnitude is calculated using the following formula: ,in and The heights are respectively at direction and The partial derivatives in the direction (calculated using the Sobel operator) are set to a threshold of 0.1 mm / mm.

[0084] II. Regions with large height fitting residuals before filtering. Specifically, "height fitting residuals" here refers to the regions with large residuals based on the 3D height distribution data before filtering. The calculated local height fitting deviation is used. For example, for each pixel, a 3×3 window is taken centered on it, and the local height distribution is fitted using a quadratic surface. The root mean square (RMS) of the fitting residual is calculated. If the RMS exceeds a preset threshold (e.g., 0.02 mm), the region is marked as having a large residual. This simplified estimation does not depend on the phase residual calculation results in subsequent steps and can be completed independently in this step.

[0085] III. Dark-field image after registration In the image, bright linear regions are detected through adaptive thresholding. For example, connected regions in dark-field images with gray values ​​higher than twice the global mean standard deviation are selected as candidate regions.

[0086] Then, the pixel coordinates of these abnormal candidate regions are mapped to the registered bright-field image. or dark field image The grayscale features of the corresponding regions are extracted for cross-validation. For example, if an anomaly candidate region exhibits a continuous bright linear structure in the dark field image and a dark linear structure in the bright field image, the anomaly is determined to be a real crack; if the anomaly candidate region shows no obvious grayscale anomaly in either the bright field or dark field image, the anomaly is determined to be noise introduced by 3D reconstruction or a normal boundary of geometric undulations. Texture verification can effectively eliminate false defects and improve detection accuracy.

[0087] Furthermore, after obtaining the three-dimensional height distribution data, the three-dimensional height distribution data is sequentially filtered and outlier removal processed to obtain the processed height distribution surface; the contour curve is extracted based on the height distribution surface.

[0088] In one specific implementation, the filtering process employs bilateral filtering, which smooths noise while preserving the abrupt changes in defect edges. The filter parameters are set as follows: spatial domain standard deviation. Pixel, grayscale standard deviation After filtering, a smooth three-dimensional height distribution is obtained. .

[0089] In one specific implementation, outlier removal employs a statistical outlier method: for each pixel... The absolute value of the height difference between the pixel and other pixels in its 5×5 neighborhood is calculated, and the average value is taken as the local variability. Then, the mean and standard deviation of the local variability of all pixels are calculated. Pixels with a local variability greater than "mean + 3 times standard deviation" are marked as outliers and replaced with the median of their neighborhood. After the above filtering and outlier removal, the processed height distribution surface is obtained. .

[0090] In one specific implementation, the contour curve is extracted by: every [number] intervals along the longitudinal direction of the weld (welding travel direction). = 0.5 mm to extract a transverse contour line. The corresponding pixel spacing is determined by the horizontal resolution calibrated by the camera. For example, if the camera calibration yields an actual physical size of 0.02 mm / pixel for each pixel, then... =0.5 mm corresponds to a pixel interval of 25 pixels, meaning a contour line is extracted every 25 pixels. Each contour line is composed of the height values ​​of all pixels at that cross-section position, denoted as . , ,in The spatial coordinates are in the transverse direction. The height value of the corresponding pixel (derived from the processed height distribution surface). The extracted contour curves will be used as input data for subsequent step S3, for curvature spectrum feature construction.

[0091] It should be noted that step S2, through phase calculation, texture verification, filtering and culling, and contour extraction, obtains a clean and reliable three-dimensional height distribution surface and its cross-sectional contour curve, providing a high-quality data foundation for subsequent geometric shape analysis based on curvature spectrum features.

[0092] S3. Calculate the curvature distribution based on the contour curve to construct a curvature spectrum feature that characterizes the geometric changes in the cross-sectional morphology of the weld.

[0093] Specifically, constructing the curvature spectrum features includes:

[0094] Obtain the discrete sampling point set of the contour curve;

[0095] The first and second derivatives at each sampling point are calculated using a local window curve fitting method to obtain the curvature value of each sampling point;

[0096] The curvature value is smoothed using a sliding window.

[0097] The smoothed curvature values ​​are arranged according to the cross-sectional direction of the weld to form a curvature distribution sequence;

[0098] The set of curvature extrema, the rate of change of curvature, and the periodic parameters of the curvature distribution are extracted from the curvature distribution sequence and vectorized to form the curvature spectrum features.

[0099] In one specific implementation, the contour curve extracted in step S2 is represented as a discrete set of sampling points. ( The actual physical distance between adjacent pixels in the transverse direction The horizontal resolution is determined by the camera's calibration (e.g., 0.02 mm / pixel). It should be noted that directly using this "discrete sampling point set" in this step does not require additional resampling of the contour curve.

[0100] In one specific implementation, the curvature value of the sampling point is calculated as follows:

[0101] With the current sampling point Centered on the left and right, select each A local window is formed by several adjacent points, and the window width is... ( (Take values ​​from 3 to 8). Perform least-squares fitting on the discrete points within the window using a quadratic polynomial: After fitting, the first derivative Second derivative Then the sampling point curvature at Calculate using the following formula: .

[0102] For example, when the sampling density is high ( When ≤ 0.01 mm, take =3, window width is 7 points; when the sampling density is low ( When ≥ 0.05 mm, take =8, with a window width of 17 points to ensure fitting stability.

[0103] In one specific implementation, a sliding window is used to smooth the curvature value, including: setting the width of the sliding window to be... (Take an odd number, such as 5, 7, or 9), then the smoothed curvature value The arithmetic mean of the curvature values ​​within the window:

[0104]

[0105] in, This is the index of the current sampling point; For the first The original curvature values ​​of each sampling point; For the smoothed first The curvature value of each sampling point.

[0106] For example, when When = 5, the range of summation is: arrive ,Right now:

[0107] ;

[0108] It should be noted that for general welds, the following should be taken: =5 is sufficient to effectively suppress isolated noise points; for welds with large surface roughness, the window value can be appropriately increased to 5. =9.

[0109] In one specific implementation, the smoothed curvature values ​​are arranged according to the cross-sectional direction of the weld to form a curvature distribution sequence, including: arranging the smoothed curvature values... By x-axis Arranged from smallest to largest, forming a curvature distribution sequence. This sequence reflects the degree of curvature at various points on the weld cross-section.

[0110] In one specific implementation, the method for extracting the set of curvature extrema points is as follows: traverse the curvature distribution sequence; if the curvature value of a certain point is greater than that of its immediate and next-to-after points (i.e., ... and If ), then it is marked as a local maximum point; if and Then it is marked as a local minimum point. For the endpoints of the sequence ( or Only compare with adjacent points on the existing side; if the extreme value condition is still satisfied, mark it as well. Record the positions of all extreme points. and curvature value .

[0111] In one specific implementation, the rate of change of curvature is defined as the ratio of the absolute value of the curvature difference between adjacent extreme points to the cross-sectional distance. Let two adjacent extreme points... and The curvature values ​​are respectively and The transverse distance is Then the rate of change of curvature For example, the largest rate of curvature change among all adjacent extreme points is taken as the characteristic value to characterize the degree of abrupt change in weld morphology.

[0112] In one specific implementation, the periodicity parameters of the curvature distribution are extracted through autocorrelation analysis. For the curvature distribution sequence... Perform autocorrelation calculation:

[0113] ;

[0114] in The mean of the sequence. The number of lag steps (with values ​​of...) , (where the sequence length is 1). Autocorrelation function The first main peak (i.e., except for) outside, The number of lag steps corresponding to the point where the maximum value is taken. Multiply by the sampling interval That is, the main cycle The peak value of the main peak denoted as amplitude Its value range is To facilitate subsequent feature construction, the absolute value is taken. As a periodic amplitude, and its value range is This is used to characterize the strength of the periodicity of the curvature distribution. For example, the fish-scale pattern region of the weld, due to its regular periodic undulations, has an autocorrelation function in... A clear peak is observed at this location. Typically greater than 0.6; however, crack regions, due to their irregular morphology and lack of obvious periodicity, Typically lower (e.g., below 0.3).

[0115] In one specific implementation, the extracted features are vectorized to form a curvature spectrum feature vector. :

[0116] ;

[0117] in and These represent the maximum and minimum values ​​of the curvature distribution, respectively. It is the maximum rate of change of curvature (i.e., the maximum value of the ratio of the midpoint curvature difference to the transverse distance among all adjacent extreme points). The number of extreme points.

[0118] It should be noted that the elements in the feature vector can be added or removed according to the actual detection needs. For example, statistics such as the standard deviation and skewness of the curvature distribution can be added. Only the core features are listed here.

[0119] Furthermore, the preset range of the local window is adaptively adjusted according to the sampling density in the cross-sectional direction of the weld, and the window size of the sliding window is dynamically determined based on the local rate of change of the curvature distribution.

[0120] In a preferred embodiment, the local window width The adaptive adjustment rules are as follows:

[0121] Calculate the average interval of sampling points ,like If < 0.02 mm, then take = 5 ( =2); if 0.02 mm ≤ If < 0.05 mm, then take = 9 ( =4); if If the value is ≥ 0.05 mm, then take... =15 ( =7). This rule ensures the physical length of the window coverage ( × Keep it within the range of 0.1~0.5 mm to make curvature calculation sensitive to local deformation.

[0122] In a preferred embodiment, the sliding window size The local rate of change is dynamically determined based on the curvature distribution. Specifically, for each sampling point... Calculate the coefficient of variation of curvature values ​​in its neighborhood. ,in The standard deviation of the curvature values ​​within the neighborhood. The mean of the curvature values ​​within the neighborhood. Take The neighborhood centered at the current sampling point has an initial window width of 5. It should be noted that CV is calculated in real-time based on the curvature data within the neighborhood of the current sampling point, rather than being a preset fixed value. If A value > 0.3 indicates a drastic change in local curvature (such as at the edge of a defect). Take a smaller value (such as 3 or 5) to preserve details; if ≤0.3 indicates that the local curvature is gentle. Use a larger value (such as 9 or 11) to enhance the smoothing effect. For example, in the crack tip region, It can reach above 0.5, at this time Take 3; in the normal weld area, Approximately 0.1, Take 7.

[0123] It should be noted that step S3, through the above discrete point set processing, local window curvature fitting, sliding window smoothing, extreme value analysis, rate of change calculation, and autocorrelation periodicity analysis, successfully constructed a curvature spectrum feature vector that can quantify the geometric morphology changes of the weld cross-section. This feature has a good ability to distinguish between crack-like defects (manifested as high curvature extremes and abrupt rate of change) and normal weld geometric fluctuations (manifested as periodic curvature distribution). Next, the phase residual will be calculated based on the image data under structured light projection illumination to obtain optical features that reflect local structural anomalies on the weld surface, providing complementary detection dimensions for subsequent cross-domain fusion discrimination.

[0124] S4. Construct the corresponding ideal phase distribution based on the image data acquired under the structured light projection illumination, and calculate the difference between the actual phase distribution and the ideal phase distribution to obtain the phase residual characterizing the local structural changes on the weld surface.

[0125] Specifically, calculating the phase residual includes:

[0126] An ideal phase distribution model is established based on the preset parameters of the structured light projection fringes, and the ideal phase distribution is obtained based on the ideal phase distribution model.

[0127] The actual phase distribution is calculated from the image data acquired under the structured light projection illumination using a multi-step phase-shifting method.

[0128] The phase residual distribution is obtained by performing point-by-point difference calculation between the actual phase distribution and the ideal phase distribution;

[0129] The phase residual distribution is spatially filtered, and continuous abnormal regions are extracted to obtain the phase residual.

[0130] In one specific implementation, the ideal phase distribution model is established based on the geometric parameters of the projected fringes. Assuming the projected fringes vary linearly along the longitudinal (or transverse) direction of the weld, the ideal phase distribution... It can be represented as: ,in The stripe period (unit: mm). The fringe direction angle (unit: radians). The initial phase (unit: radians) is given. The above parameters are predetermined based on the digital projector settings and system calibration results. It should be noted that the purpose of establishing an ideal phase distribution model is to provide a phase reference under defect-free conditions, so that when the actual phase is compared with it, the phase deviation caused by local structural anomalies on the weld surface can be highlighted, thereby achieving sensitive detection of defects. For example, when the stripes are perpendicular to the weld longitudinal direction (i.e., the stripe direction is parallel to the transverse direction), ,but When the stripe period When =2mm, the ideal phase follows The direction changes by 2π radians for every 2mm.

[0131] In one specific implementation, the actual phase distribution The solution method is consistent with the phase solution method in step S2, that is, the absolute phase is obtained by combining the four-step phase shift method with the multi-frequency heterodyne method. Specifically, the four phase shift fringe patterns acquired in step S1 are used. , , , Calculate the package phase using the following formula: The absolute phase is then obtained by multi-frequency heterodyne expansion. That is, the actual phase distribution It should be noted that the absolute phase obtained in step S2 has already been used for 3D reconstruction. This step can directly reuse the calculation result without repeating the calculation, thereby significantly reducing the computational cost and improving the overall efficiency of the detection method.

[0132] In one specific implementation, the point-by-point difference calculation is performed according to the following formula: The difference is then phase-normalized to ensure it is within the specified range. Interval: ,in This is the rounding function. It should be noted that the normalization process eliminates the distortion caused by phase periodicity. The jump avoids introducing spurious abrupt changes in the phase residuals, ensuring the continuity and authenticity of the residual distribution. For example, if at a certain point... , ,but After normalization (because , Therefore, it remains unchanged; if ,but , , The normalized phase residual distribution is denoted as... Its physical meaning is the deviation of the actual stripe deformation from the ideal stripe, reflecting the optical phase change caused by local structural anomalies (such as cracks and depressions) on the weld surface.

[0133] In one specific implementation, spatial filtering employs adaptive median filtering. The specific steps are as follows:

[0134] The initial filter window size is set to 3×3, and the maximum window size is set to 11×11.

[0135] For each pixel Calculate the median of the phase residuals within the current window. Maximum value and minimum value (The input data is the normalized phase residual distribution) ).

[0136] like In and Between and center pixel value In and Between, then output Otherwise, increase the window size and repeat the above judgment until the window reaches the maximum size or the condition is met.

[0137] If the condition is still not met even after reaching the maximum window size, then output... .

[0138] Let the filtered output be denoted as .

[0139] It should be noted that this filtering method can effectively remove isolated noise points while retaining high-frequency local anomaly information in the defect area.

[0140] In another implementation, spatial filtering employs wavelet filtering. Specifically, it involves processing the normalized phase residual distribution... A three-level wavelet decomposition was performed (using the db4 wavelet basis), preserving coefficients with concentrated energy in the high-frequency detail subbands. The coefficient preservation threshold was set using a general threshold formula: ,in For noise standard deviation estimation, the median estimate of the absolute values ​​of the highest frequency subband coefficients by wavelet decomposition is: , This represents the total number of pixels. The coefficient with an absolute value greater than [a certain value] is considered. The coefficients of are retained, and the remaining coefficients are set to zero. Then, the phase residual distribution is reconstructed, and the reconstruction result is denoted as . Wavelet filtering can better preserve the detailed features of defect edges.

[0141] In one specific implementation, the method for extracting continuous abnormal regions is as follows:

[0142] Take the absolute value of the filtered phase residual distribution, denoted as .

[0143] Set threshold Exemplary (That is, a 30° phase difference corresponds to a height change of approximately 0.05 mm).

[0144] Will The pixels are marked as candidate anomalies.

[0145] An 8-connectivity analysis algorithm is used to aggregate adjacent candidate outliers into connected regions, and the deletion area is smaller than 1. Isolated noise areas (e.g., 10 pixels).

[0146] The preserved connected regions are the continuous outlier regions. The area, average residual, maximum residual, and location information of each region are recorded as phase residual features. Output. Beneficially, this feature directly quantifies the geometric dimensions and phase distortion degree of the anomalous region, facilitating joint judgment with curvature spectrum features during subsequent cross-domain fusion and improving the accuracy of defect classification.

[0147] It should be noted that step S4 successfully extracts the phase residual distribution reflecting local structural anomalies on the weld surface by establishing an ideal phase distribution, using the actual phase calculation results from step S2, point-by-point differencing, and adaptive filtering or wavelet filtering. This feature is sensitive to strong scattering defects such as cracks and complements the curvature spectrum feature (reflecting the macroscopic periodicity of the geometric morphology) from step S3 in terms of physical mechanism, providing two complementary feature dimensions for subsequent cross-domain fusion discrimination. Next, the curvature spectrum feature and the phase residual are fused across domains to construct a joint discrimination rule, achieving accurate differentiation between crack-like defects, weld geometric undulations, and normal areas.

[0148] S5. The curvature spectrum features and the phase residual are fused across domains to construct a joint discrimination rule for identifying surface defects of the weld; wherein, the joint discrimination rule can distinguish between crack-like defects, weld geometric undulations and normal areas.

[0149] Specifically, the joint discrimination rule includes:

[0150] Curvature anomaly indices are constructed based on the curvature spectrum features respectively. A phase anomaly index is constructed based on the phase residual. ;

[0151] Regarding the curvature anomaly index Phase anomaly index Normalization was performed to obtain the normalized curvature anomaly index. With normalized phase anomaly index ;

[0152] Preset first threshold Second threshold The first threshold Used to determine the normalized phase anomaly index The second threshold Used to determine the normalized curvature anomaly index And the first threshold and the second threshold Determined through the calibration process;

[0153] Based on the normalized curvature anomaly index Normalized phase anomaly index First threshold Second threshold Perform stratification:

[0154] when When this occurs, it is determined to be a normal area;

[0155] when and At that time, it was determined to be a crack-type defect;

[0156] when and When this occurs, it is determined to be a geometric undulation of the weld.

[0157] In one specific implementation, the curvature anomaly index Based on the extraction in step S3 Build. For example, A weighted summation method can be used:

[0158] ;

[0159] in The maximum value of the curvature distribution. The maximum allowable curvature threshold for a normal weld. The maximum rate of change of curvature, The threshold for the maximum rate of change of curvature; Periodic amplitude (range of values) ), Used to characterize the degree of aperiodicity; , , Let be the weighting coefficient, satisfying It should be noted that, and This is an empirical threshold, which can be preset based on the statistical results of normal weld samples. Example weights are taken separately. and ; , , The value can be determined through calibration based on actual testing needs, or an empirical value can be used. An example is... , , The larger this index is, the greater the degree to which the weld morphology deviates from the normal geometric fluctuations.

[0160] In one specific implementation, the phase anomaly index Based on the phase residual features extracted in step S4 Build. For example, The following formula can be used:

[0161] ;

[0162] in The absolute value of the average phase residual in the continuous anomaly region. The phase residual threshold is set to π / 6. This represents the area (in pixels) of the abnormal region. This is for reference only. It should be noted that... The pixel area covered by a single stripe period can be taken as an example, when the stripe period corresponds to 10 pixels. =10×10=100 pixels, this selection makes It is dimensionless and has a moderate range. This index comprehensively reflects the phase distortion intensity and spatial range of the anomalous region; the larger the value, the higher the probability of strong scattering defects such as cracks.

[0163] In one specific implementation, the normalization process uses the Min-Max normalization method:

[0164] ;

[0165] ;

[0166] in, , These are the minimum and maximum values ​​of the curvature anomaly index in the calibration sample, respectively. , Similarly. For example, if the calibration sample contains... The range of values ​​is ,but hour, After normalization, and All were mapped to The interval is used to facilitate subsequent comparison and determination with the corresponding threshold.

[0167] In a preferred embodiment, the typical range of values ​​for the calibrated parameters is as follows: , For example, after calibration... , At this time, if a certain area , If it is, then it is determined to be a crack-type defect; if , If so, it is determined to be a geometric undulation of the weld; if It was directly identified as a normal area.

[0168] Furthermore, the first threshold Second threshold Determined through calibration samples; calibration includes:

[0169] Collect multiple sets of weld sample data, including crack-like defects, weld geometry undulations, and normal areas;

[0170] Curvature spectrum features and phase residual distributions of each sample are extracted to construct a curvature anomaly index. Phase anomaly index ;

[0171] Based on the characteristic statistical results of various samples, the normalized curvature anomaly index was determined. and normalized phase anomaly index The range of values ​​for ;

[0172] The first threshold is optimized by combining interval boundary analysis or the principle of minimizing distribution overlap with statistical learning methods. Second threshold .

[0173] In one specific implementation, multiple sets of weld sample data are collected, specifically: at least 50 typical weld samples are collected, including at least 20 samples of crack defects, at least 20 samples of weld geometric undulations, and at least 20 samples of normal areas, ensuring that each category covers different severity levels (e.g., crack width from 0.05mm to 0.5mm, and fish-scale pattern period from 0.5mm to 2.0mm). It should be noted that the selection of the sample size must ensure statistical significance; 50 samples can meet basic parameter optimization requirements.

[0174] In one specific implementation, the curvature spectrum features and phase residual distribution corresponding to each sample are extracted to construct a curvature anomaly index. Phase anomaly index Specifically, steps S1 to S4 are performed for each sample to extract the curvature spectrum features and phase residual distribution corresponding to each sample, and a curvature anomaly index is constructed according to the method defined in step S5. Phase anomaly index Specifically, Based on curvature spectrum feature vector In , , Equal element weighted calculation; Based on phase residual characteristics average residuals and the area of ​​the abnormal region Calculation. The final result is the feature pair set of the sample set. ,in , The total number of samples, with each sample also labeled with the true class label. {Normal area, weld geometric undulations, crack-like defects}.

[0175] In one specific implementation, the method for determining the value range is as follows:

[0176] Statistical analysis was performed on crack-type samples, weld geometry fluctuation samples, and normal samples respectively. and Distribution range:

[0177] Normal sample: Typically low (e.g., 0~0.5). It is also relatively low (e.g., 0~0.3);

[0178] Sample of weld geometry undulations: Moderately high (e.g., 0.3~1.2), but Lower (e.g., 0.1~0.4);

[0179] Crack-type samples: Moderately high (e.g., 0.4~1.5), and Significantly high (e.g., 0.6~1.2).

[0180] It should be noted that this value range analysis provides a priori range for subsequent threshold optimization, for example... It should be set in normal samples Upper limit and weld geometry fluctuation sample Between the lower limit (e.g., 0.2~0.4). It should be set in the geometric undulation sample Upper limit and crack sample Between the lower limit (e.g., 0.5~0.7).

[0181] In one specific implementation, the statistical learning method employs at least one of the following approaches:

[0182] Method 1 (Interval Boundary Analysis): Based on the distribution intervals of various sample indicators determined above, the interval boundaries are directly selected as the initial thresholds. For example, if normal samples... The maximum value is 0.28, weld geometric undulation sample. The minimum value is 0.32, then The midpoint between the two can be taken as 0.30; if the weld geometry undulation sample The maximum value is 0.68, for crack samples. The minimum value is 0.72, then The middle value of 0.70 is acceptable.

[0183] Method 2 (Principle of Minimizing Distribution Overlap): To maximize classification accuracy, a grid search combined with cross-validation is used for optimization. , For example, setting Search scope Step size 0.05; The search range is [0.1, 0.5], with a step size of 0.05. For each group... , The classification results are output according to the hierarchical judgment rules, and compared with the true labels. Compare the thresholds and select the one with the highest accuracy.

[0184] In a preferred embodiment, the method further includes validating the calibration parameters by using an independent test set (no fewer than 20 samples) to verify the generalization ability of the calibration parameters. For example, the optimized... , It is applied to the test set to calculate indicators such as classification accuracy, crack detection rate, and geometric fluctuation misclassification rate. If the performance does not meet the requirements (such as accuracy below 95%), it returns to the sample data collection to supplement the sample for recalibration.

[0185] It should be noted that the above calibration process can be completed in advance offline, and the calibrated parameters... , It is embedded in the detection system and used for real-time defect classification during actual online detection, eliminating the need for repeated calibration for each detection.

[0186] It should be noted that step S5, by cross-domain fusion of the curvature spectrum features of the geometric domain and the phase residual of the optical domain, constructs a dual-threshold hierarchical judgment rule based on curvature anomaly indicators and phase anomaly indicators, thus achieving accurate classification of weld surface defects. Compared with single feature detection methods, the joint discrimination rule of this invention has the following beneficial effects:

[0187] Low false alarm rate: when It directly identifies areas as normal and uses curvature indices to preferentially filter out a large number of defect-free flat areas, avoiding false alarms caused by surface reflections, rust, and other interferences when relying solely on phase residuals or grayscale images.

[0188] High discrimination: for Geometric anomaly regions, further utilizing phase residual indices Secondary discrimination is performed: Crack-type defects will produce strong phase distortion due to the disruption of material continuity. While weld geometric undulations (such as fish-scale patterns and variations in reinforcement height) cause curvature changes, the surface remains continuous and smooth, resulting in a low phase residual. This effectively distinguishes between the two and significantly reduces the rate of misjudging normal weld morphology as cracks;

[0189] Parameter offline calibration and cross-weld applicability: threshold , Once pre-determined through offline calibration and embedded in the detection system, it can be used for online real-time detection without the need for retraining for each weld seam, and it has good adaptability to different welding processes and weld seam types.

[0190] Thus, this embodiment completes the entire process from multi-source image acquisition, 3D reconstruction, curvature spectrum and phase residual feature extraction, to cross-domain fusion and defect classification. It achieves accurate identification of weld surface crack-like defects, geometric undulations and normal areas, and has good adaptability to different lighting conditions, surface reflection and noise interference, providing an efficient and reliable solution for industrial vision online weld quality inspection.

[0191] In summary, this invention sequentially applies bright-field illumination, dark-field illumination, and structured light projection illumination to the weld area to be inspected, acquiring the macroscopic morphology, surface microstructure defects, and phase encoding information of the weld, respectively. Then, based on the structured light image, it reconstructs the three-dimensional height distribution and extracts curvature spectrum features (characterizing geometric morphological abrupt changes and periodicity). Simultaneously, it constructs an ideal phase distribution and calculates the phase residual (characterizing optical scattering anomalies). Finally, it fuses the curvature spectrum features and phase residual across domains to construct a joint discrimination rule, achieving refined differentiation between crack-like defects, weld geometric undulations, and normal areas. This solves the technical problem of existing single-modal detection methods being unable to effectively distinguish between cracks and normal undulations such as fish-scale patterns, significantly reducing false alarm and false negative rates. In the joint discrimination rule, this invention constructs curvature anomaly indicators and phase anomaly indicators based on curvature spectrum features and phase residuals, respectively, and normalizes these indicators. A dual-threshold hierarchical judgment strategy is used for defect classification; specifically, normal areas are first screened based on the normalized curvature anomaly indicator, and then crack-like defects and weld geometric undulations are distinguished based on the normalized phase anomaly indicator. This rule fully utilizes the sensitivity of curvature spectrum features to changes in weld geometry and the sensitivity of phase residuals to local structural anomalies. It maintains a stable recognition accuracy for weak-contrast cracks and weld surfaces with complex periodic undulations. Furthermore, the first and second thresholds are predetermined through offline calibration, eliminating the need for online training for each weld. This allows it to be directly applied to industrial sites with different welding processes, weld types, and lighting conditions, significantly improving the versatility and practical deployment efficiency of the detection system. This invention performs spatial registration of bright-field, dark-field, and structured-light images after multimodal image acquisition, and uses the registered bright-field or dark-field images to verify the texture of three-dimensional height distribution data. By mapping the marked abnormal candidate regions in the three-dimensional data to the two-dimensional image for grayscale cross-validation, false defects caused by interference such as three-dimensional reconstruction noise or surface reflection are effectively eliminated. At the same time, the height distribution surface after filtering and abnormal point removal is used for contour extraction, which further ensures the accuracy and reliability of curvature spectrum features. This forms a complete closed-loop detection chain from image acquisition, three-dimensional reconstruction, feature extraction to cross-domain fusion, which significantly improves the accuracy of weld surface defect detection and industrial field adaptability.

[0192] Example 2, a second embodiment of the present invention, provides a multi-source fusion visual inspection system for weld surface defects corresponding to the method of the previous embodiment. This system can be integrated into industrial robots or automated welding production lines to achieve online, real-time weld quality inspection and defect classification. The system includes the following modules:

[0193] Multi-source lighting module, used to provide bright field lighting, dark field lighting and structured light projection lighting;

[0194] The image acquisition module is used to acquire weld images corresponding to the weld area under various lighting conditions to obtain multimodal image data;

[0195] The three-dimensional reconstruction module is used to perform phase calculation based on the image data acquired under the structured light projection illumination to obtain the three-dimensional height distribution data of the weld surface;

[0196] The filtering and removal module is used to sequentially filter and remove outliers from the three-dimensional height distribution data to obtain the processed height distribution surface.

[0197] The curvature analysis module is used to extract the contour curve of the weld cross section based on the height distribution surface and construct curvature spectrum features;

[0198] The phase residual calculation module is used to establish an ideal phase distribution based on the preset parameters of the structured light projection fringes, solve the actual phase distribution using the multi-step phase shift method, calculate the difference between the actual phase distribution and the ideal phase distribution, and obtain the phase residual.

[0199] The registration and verification module is used to spatially register images acquired under bright field illumination, dark field illumination, and structured light projection illumination, so that the three correspond in pixel coordinates, and to use the registered bright field image or dark field image to verify the texture of the three-dimensional height distribution data.

[0200] The defect discrimination module is used to perform cross-domain fusion of the curvature spectrum features and the phase residual, and to identify weld surface defects based on the joint discrimination rules, distinguishing between crack-type defects, weld geometric undulations and normal areas.

[0201] It should be noted that the data flow between the above modules is as follows: the multi-source illumination module and the image acquisition module work together to output multimodal images to the registration and verification module; the registered structured light image is sent to the 3D reconstruction module to generate a 3D height distribution; the 3D height distribution is processed by the filtering and culling module and then sent to the curvature analysis module and the registration and verification module (for texture verification); at the same time, the absolute phase output by the 3D reconstruction module is also sent to the phase residual calculation module; the curvature analysis module outputs curvature spectrum features, and the phase residual calculation module outputs phase residual features. Both are input into the defect discrimination module, and finally, the classification results of weld surface defects are output.

[0202] This embodiment, through the coordinated operation of the eight modules described above, realizes a multi-source fusion visual inspection system for weld surface defects that is completely corresponding to the method in Embodiment 1. This system can be deployed in industrial sites to achieve fully automated, high-precision online inspection of weld quality.

[0203] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A multi-source fusion visual inspection method for weld surface defects, characterized in that, Includes the following steps: Bright field illumination, dark field illumination, and structured light projection illumination are applied sequentially to the weld area to be inspected, and corresponding weld images are acquired under each illumination condition to obtain multimodal image data for industrial vision analysis. Phase calculation is performed on the image data acquired under the structured light projection illumination to obtain the three-dimensional height distribution data of the weld surface, and the contour curve of the weld cross section is extracted based on the three-dimensional height distribution data. Based on the contour curve, the curvature distribution is calculated, and a curvature spectrum feature characterizing the geometric changes in the cross-sectional morphology of the weld is constructed. Based on the image data acquired under the structured light projection illumination, a corresponding ideal phase distribution is constructed, and the difference between the actual phase distribution and the ideal phase distribution is calculated to obtain the phase residual characterizing the local structural changes on the weld surface. The curvature spectrum features and the phase residuals are fused across domains to construct a joint discrimination rule for identifying surface defects in the weld. The joint discrimination rule can distinguish between crack-like defects, weld geometric undulations, and normal areas.

2. The multi-source fusion visual inspection method for weld surface defects as described in claim 1, characterized in that: Constructing the curvature spectrum features includes: Obtain the discrete sampling point set of the contour curve; The first and second derivatives at each sampling point are calculated using a local window curve fitting method to obtain the curvature value of each sampling point; The curvature value is smoothed using a sliding window. The smoothed curvature values ​​are arranged according to the cross-sectional direction of the weld to form a curvature distribution sequence; The set of curvature extrema, the rate of change of curvature, and the periodic parameters of the curvature distribution are extracted from the curvature distribution sequence and vectorized to form the curvature spectrum features.

3. The multi-source fusion visual inspection method for weld surface defects as described in claim 1, characterized in that: Calculating the phase residual includes: An ideal phase distribution model is established based on the preset parameters of the structured light projection fringes, and the ideal phase distribution is obtained based on the ideal phase distribution model. The actual phase distribution is calculated from the image data acquired under the structured light projection illumination using a multi-step phase-shifting method. The phase residual distribution is obtained by performing point-by-point difference calculation between the actual phase distribution and the ideal phase distribution; The phase residual distribution is spatially filtered, and continuous abnormal regions are extracted to obtain the phase residual.

4. The multi-source fusion visual inspection method for weld surface defects as described in claim 1, characterized in that: The joint discrimination rules include: Curvature anomaly indices are constructed based on the curvature spectrum features respectively. A phase anomaly index is constructed based on the phase residual. ; Regarding the curvature anomaly index Phase anomaly index Normalization was performed to obtain the normalized curvature anomaly index. With normalized phase anomaly index ; Preset first threshold Second threshold The first threshold Used to determine the normalized phase anomaly index The second threshold Used to determine the normalized curvature anomaly index And the first threshold and the second threshold Determined through the calibration process; Based on the normalized curvature anomaly index Normalized phase anomaly index First threshold Second threshold Perform stratification: when When this occurs, it is determined to be a normal area; when and At that time, it was determined to be a crack-type defect; when and When this occurs, it is determined to be a geometric undulation of the weld.

5. The multi-source fusion visual inspection method for weld surface defects as described in claim 1, characterized in that: After acquiring multimodal image data, spatial registration is performed on the images acquired under bright field illumination, dark field illumination, and structured light projection illumination to ensure that the three correspond in pixel coordinates; the registered bright field image or dark field image is then used to verify the texture of the three-dimensional height distribution data.

6. The multi-source fusion visual inspection method for weld surface defects as described in claim 1, characterized in that: After obtaining the three-dimensional height distribution data, the three-dimensional height distribution data is sequentially filtered and outlier removal processed to obtain the processed height distribution surface; the contour curve is extracted based on the height distribution surface.

7. The multi-source fusion visual inspection method for weld surface defects as described in claim 2, characterized in that: The preset range of the local window is adaptively adjusted according to the sampling density in the cross-sectional direction of the weld, and the window size of the sliding window is dynamically determined based on the local rate of change of the curvature distribution.

8. The multi-source fusion visual inspection method for weld surface defects as described in claim 3, characterized in that: The multi-step phase shifting method uses at least four stripe images with different phase offsets for phase calculation; the spatial filtering process uses adaptive median filtering or wavelet filtering.

9. The multi-source fusion visual inspection method for weld surface defects as described in claim 4, characterized in that: The first threshold Second threshold The determination is made by calibrating samples, and the calibration includes: Collect multiple sets of weld sample data, including crack-like defects, weld geometry undulations, and normal areas; Curvature spectrum features and phase residual distributions of each sample are extracted to construct a curvature anomaly index. Phase anomaly index ; Based on the statistical results of the characteristics of various samples, the value range of each indicator is determined; The first threshold is optimized by combining interval boundary analysis or the principle of minimizing distribution overlap with statistical learning methods. Second threshold .

10. A multi-source fusion visual inspection system for weld surface defects, applied to the multi-source fusion visual inspection method for weld surface defects as described in any one of claims 1-9, characterized in that, include: Multi-source lighting module, used to provide bright field lighting, dark field lighting and structured light projection lighting; The image acquisition module is used to acquire weld images corresponding to the weld area under various lighting conditions to obtain multimodal image data; The three-dimensional reconstruction module is used to perform phase calculation based on the image data acquired under the structured light projection illumination to obtain the three-dimensional height distribution data of the weld surface; The filtering and removal module is used to sequentially filter and remove outliers from the three-dimensional height distribution data to obtain the processed height distribution surface. The curvature analysis module is used to extract the contour curve of the weld cross section based on the height distribution surface and construct curvature spectrum features; The phase residual calculation module is used to establish an ideal phase distribution based on the preset parameters of the structured light projection fringes, solve the actual phase distribution using the multi-step phase shift method, calculate the difference between the actual phase distribution and the ideal phase distribution, and obtain the phase residual. The registration and verification module is used to spatially register images acquired under bright field illumination, dark field illumination, and structured light projection illumination, so that the three correspond in pixel coordinates, and to use the registered bright field image or dark field image to verify the texture of the three-dimensional height distribution data. The defect discrimination module is used to perform cross-domain fusion of the curvature spectrum features and the phase residual, and to identify surface defects of the weld seam based on the joint discrimination rules, distinguishing between crack-type defects, weld geometric undulations and normal areas.

Images