Laser welding line on-line quality detection system and method based on multi-mode visual fusion

Abstract

The invention discloses a laser welding seam online quality detection system and method based on multi-mode visual fusion, and particularly relates to the technical field of laser welding seam quality detection. Comprising a data acquisition and synchronous control unit, a 2D image processing and defect detection unit, a 3D point cloud processing and size measurement unit, a multi-modal information fusion and collaborative analysis unit and a comprehensive quality judgment and output unit. According to the laser welding seam online quality detection system and method based on multi-modal visual fusion, the 2D and 3D multi-modal visual fusion technology is adopted, an improved defect detection algorithm and an accurate size measurement method are combined, welding seam surface defects are accurately recognized through 2D images, key size data are obtained by means of 3D point cloud, and the detection accuracy is improved. And the limitation of'heavy defects and light sizes' or'heavy sizes and light defects' in single-modal detection is avoided, and comprehensive and accurate evaluation of the welding seam quality is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of laser weld quality detection, in particular to a laser weld online quality detection system and method based on multi-modal vision fusion. BACKGROUND

[0002] Laser welding technology has been widely used in high-end manufacturing fields such as automobile manufacturing, aerospace, precision instruments, etc. due to its high energy density, fast welding speed, and small heat-affected zone. The weld quality directly affects the structural strength, sealing performance, and service life of the product, so it is crucial to achieve online efficient detection of weld quality.

[0003] Currently, laser weld quality detection technology mainly includes offline detection and online detection. Offline detection (such as destructive detection, ultrasonic detection, and X-ray detection) has high accuracy, but it needs to interrupt the production process, and the detection efficiency is low, which cannot meet the real-time quality control requirements of modern production lines. Online detection can provide real-time feedback of quality information and timely adjustment of welding parameters, so it has become the mainstream direction of the industry. SUMMARY

[0004] The main purpose of the present application is to provide a laser weld online quality detection system and method based on multi-modal vision fusion, which can effectively solve the problems mentioned above.

[0005] To achieve the above purpose, the technical solution adopted by the present application is as follows:

[0006] The laser weld online quality detection system based on multi-modal vision fusion comprises a data acquisition and synchronous control unit, a 2D image processing and defect detection unit, a 3D point cloud processing and size measurement unit, a multi-modal information fusion and collaborative analysis unit, and a comprehensive quality judgment and output unit.

[0007] The data acquisition and synchronous control unit is used to control the 2D vision acquisition module and the 3D vision acquisition module to collect 2D images and 3D point cloud data (X, Y, Z coordinates) of the weld at the same time point or under strict timing through a synchronous trigger device, and transmit the 2D images to the 2D image processing and defect detection unit and the 3D point cloud data to the 3D point cloud processing and size measurement unit.

[0008] The 2D image processing and defect detection unit is used to preprocess the received 2D images by denoising, enhancing, and extracting the ROI of the weld area, detect the weld area defects through a trained YOLOv11 model, and output the defect type, 2D image coordinate position, confidence, and bounding box.

[0009] The 3D point cloud processing and dimension measurement unit is configured to preprocess the received 3D point cloud data, including denoising, registration and workpiece coordinate system conversion, extract the point cloud of the weld area through the defect position / ROI information provided by the 2D detection module or the point cloud geometric feature, calculate the key dimensions such as weld width, weld height / extra height, weld leg and theoretical weld thickness and output the actual measurement values.

[0010] The multi-modal information fusion and collaborative analysis unit is configured to establish a mapping relationship between the 2D pixel coordinates and the 3D world coordinates based on the intrinsic and extrinsic calibration parameters of the 2D camera and the 3D camera, map the 2D defect position to the 3D space, verify the 2D suspected defect through the 3D geometric feature, trigger the 2D review on the 3D dimension exceeding area, optimize the 2D ROI based on the 3D weld boundary, and form a fusion feature vector in combination with the 2D texture feature and the 3D geometric feature.

[0011] The comprehensive quality judgment and output unit is configured to receive the 2D defect information, the 3D dimension information and the fusion feature vector, judge the quality result based on a preset rule, output a quality judgment report and transmit the quality judgment report to the welding equipment PLC, the upper computer or the MES system.

[0012] Preferably, in the 2D image processing and defect detection unit, the YOLOv11 model comprises an improved Defect-YOLOv11, the Defect-YOLOv11 comprises a C3K2 module improved by adaptive sampling attention and an EMA-SlideLoss function with dynamic IoU threshold adjustment, the improved C3K2 module enhances the ability of capturing subtle defect features through the adaptive sampling attention mechanism, and the attention weight calculation formula is as follows:

[0013]

[0014] wherein, W i,j is the attention weight of the image pixel (i, j), S i,j is the feature saliency value of the pixel.

[0015] The EMA-SlideLoss function is used to optimize the sample imbalance problem, and the formula is as follows:

[0016]

[0017] wherein, is the loss value of the dynamic IoU threshold adjustment, and EMA is an exponential moving average operator.

[0018] Preferably, in the preprocessing of the 3D point cloud processing and dimension measurement unit, the point cloud denoising adopts a statistical filtering algorithm, and the formula is as follows:

[0019] P out ={p∈P|d(p,μ)>kσ}

[0020] wherein, P is the original point cloud, p is a point in the point cloud, μ is the average distance of the neighborhood points of point p, σ is the standard deviation of the distance, k is a preset threshold (value 1.5-2.0), P out is the set of removed noise points.

[0021] Preferably, when the 3D point cloud processing and the size measurement unit calculate the weld width, a cross section is taken along the direction perpendicular to the weld, the maximum transverse distance of the weld contour in the cross section is taken, and based on the defect position / ROI information provided by the 2D detection module, the formula is:

[0022] W = max (X r -X l )

[0023] wherein, W is the weld width, X r , X l are the X coordinates of the right and left edges of the weld in the cross section, respectively.

[0024] Preferably, when the 3D point cloud processing and the size measurement unit calculate the weld height / remaining height, the vertical distance between the highest point of the weld and the reference plane of the base material surface is taken, and the formula is:

[0025] H = Z max -Z base

[0026] wherein, H is the weld height / remaining height, Z max is the maximum Z coordinate value of the point cloud in the weld area, and Z base is the Z coordinate value of the reference plane of the base material surface.

[0027] Preferably, when the 3D point cloud processing and the size measurement unit calculate the theoretical weld thickness, the effective bearing area estimation based on the weld cross section contour is taken, and the formula is:

[0028]

[0029] wherein, T is the theoretical weld thickness, A is the effective bearing area of the weld in the cross section, and L is the reference length of the weld cross section.

[0030] Preferably, when the multi-modal information fusion and collaborative analysis unit forms the fusion feature vector, the 2D texture features and 3D geometric features are standardized and weighted fused, and the formula is:

[0031] F = α × F 2D +(1-α) × F 3D

[0032] wherein, F is the fusion feature vector, F 2D is the standardized 2D texture feature vector, and F 3DFor the standardized 3D geometric feature vector, alpha is the weight coefficient.

[0033] The laser weld seam online quality detection method based on multi-modal visual fusion uses the system, including the following steps:

[0034] S1, the 2D vision acquisition module and the 3D vision acquisition module are controlled by a synchronous trigger device to synchronously acquire 2D images and 3D point cloud data of the weld based on an encoder signal or an external trigger signal;

[0035] S2, the 2D image is preprocessed by denoising, enhancing and ROI extraction, the improved YOLOv11 model is used to detect defects and output information; the 3D point cloud is preprocessed by denoising, registration and coordinate system conversion, the weld seam area point cloud is extracted, the key size is calculated by the formula and the measurement value is output;

[0036] S3, the 2D and 3D coordinate mapping is established based on the camera calibration parameters, the 2D defect position is mapped to the 3D space, the fusion feature vector is formed by the formula, and the information complementary verification and collaborative analysis are completed;

[0037] S4, the quality result is comprehensively judged based on the preset rule, and the report and the visual content are output.

[0038] Compared with the prior art, the present application has the following beneficial effects:

[0039] 1. The present application adopts 2D and 3D multi-modal visual fusion technology, combines improved defect detection algorithm and accurate size measurement method, accurately identifies weld surface defects through 2D image, acquires key size data through 3D point cloud, avoids the limitations of "heavy defect and light size" or "heavy size and light defect" in single modal detection, and realizes comprehensive and accurate evaluation of weld quality.

[0040] 2. The present application ensures data consistency through synchronous acquisition, realizes single-frame data processing time < 50ms relying on fast processing algorithm, and the detection result can be transmitted to the welding equipment and management system to form a closed loop feedback, thereby improving the real-time quality control ability and efficiency of the production line and meeting the demand of high-precision manufacturing scene. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 The present application is a whole module flow chart;

[0042] Figure 2 The present application is a whole method step flow chart. DETAILED DESCRIPTION

[0043] In order to make the technical means, creative features, purposes and effects realized by the present application easy to understand, the present application is further described in conjunction with specific embodiments.

[0044] As shown in Figure 1 and Figure 2 The laser weld seam online quality detection system and method based on multi-modal visual fusion, comprising: a data acquisition and synchronous control unit, a 2D image processing and defect detection unit, a 3D point cloud processing and size measurement unit, a multi-modal information fusion and collaborative analysis unit, and a comprehensive quality judgment and output unit.

[0045] The data acquisition and synchronous control unit is used to control the 2D vision acquisition module and the 3D vision acquisition module to collect 2D images and 3D point cloud data (X, Y, Z coordinates) of the weld seam at the same time point or under strict timing through a synchronous trigger device, and transmit the 2D images to the 2D image processing and defect detection unit and transmit the 3D point cloud data to the 3D point cloud processing and size measurement unit.

[0046] In the above, the 2D vision acquisition module adopts a CCD industrial camera with a resolution of 2048x1536, equipped with a ring LED light source, for capturing weld surface texture and defects such as pores and spatter, the 3D vision acquisition module adopts a laser profile sensor with a scanning frequency of 10kHz and a point cloud accuracy of ±0.01mm, for obtaining weld three-dimensional coordinate data (X, Y, Z), and the synchronous trigger device is triggered based on an encoder signal to ensure that the collection time difference of 2D images and 3D point clouds is ≤1ms, and the timing error is controlled within ±0.5ms.

[0047] The implementation process is as follows:

[0048] First, the synchronous trigger device receives the position feedback signal of the welding equipment (such as the welding torch moving to the detection area), generates a synchronous pulse, and then the D camera and the 3D sensor collect data at the same time under the pulse trigger, the data is transmitted to the industrial computer through the GigE interface, and the industrial computer buffers the original data and marks the time stamp to ensure the timing consistency of subsequent processing.

[0049] Further, the 2D image processing and defect detection unit is used to pre-process the received 2D images for denoising, enhancement and weld region ROI extraction, detect the weld region defects through the trained YOLOv11 model, and output the defect type, 2D image coordinate position, confidence and bounding box.

[0050] In the 2D image processing and defect detection unit, the YOLOv11 model contains an improved Defect-YOLOv11, the Defect-YOLOv11 includes an adaptive sampling attention improved C3K2 module and an EMA-SlideLoss function with dynamic IoU threshold adjustment, the improved C3K2 module enhances the fine defect feature capture ability through the adaptive sampling attention mechanism, and the attention weight calculation formula is:

[0051]

[0052] wherein W i,j is the attention weight of the image pixel (i,j), S i,j is the feature saliency value of the pixel;

[0053] The EMA-SlideLoss function is used to optimize the sample imbalance problem, and the formula is:

[0054]

[0055] wherein, is the loss value of dynamic IoU threshold adjustment, and EMA is an exponential moving average operator.

[0056] In the above, in the preprocessing of denoising, enhancing and ROI extraction of the weld area of the 2D image, the denoising adopts adaptive median filtering, the window size is dynamically adjusted according to the local noise intensity, and the salt and pepper noise generated by the welding spatter is removed; the enhancement is to optimize the contrast of the image through the Retinex algorithm, and highlight the gray difference between the weld and the base material; the ROI extraction is based on the prior shape of the weld (such as straight line, circular arc), and the Hough transform is used to locate the weld area, and the ROI (region of interest) is cropped to reduce irrelevant information interference.

[0057] In the above, the Retinex algorithm is used for the enhancement preprocessing of the 2D image, and its core function is to optimize the contrast of the weld area and the base material area, highlight the texture details and defect features (such as pores, undercut, spatter, etc.) of the weld surface, and provide a clearer image basis for subsequent ROI extraction and defect detection (such as YOLOv11 model detection);

[0058] In specific implementation, according to the characteristics of the weld image, the processing logic of the Retinex algorithm is as follows:

[0059] The collected original 2D weld image is subjected to multi-scale Gaussian filtering, the low-frequency component (smooth light distribution) representing the light change and the high-frequency component (reflectivity information) representing the weld texture and defects are separated out, the influence of the light component is removed through logarithmic transformation and difference operation, the detail features (such as gray level mutation of the weld edge, texture difference of the defect area) in the reflection component are retained and enhanced, and then the processed image is subjected to dynamic range compression and gray scale normalization, so that the gray distribution of the weld area is more in line with the requirements of human eye perception and subsequent algorithm recognition, and the YOLOv11 model can more accurately capture the features of the subtle defects.

[0060] The formula of the Retinex algorithm is:

[0061]

[0062] Where I(x, y) is the observed image gray value (such as the pixel value of the weld 2D image); R(x, y) is the inherent reflectivity of the object (the target to be extracted, reflecting the essential characteristics of the weld defects, edges, etc.); L(x, y) is the light intensity (interference to be suppressed, such as strong light during welding, surface reflection).

[0063] In the above, the model training data set contains 5000 labeled images (covering 8 types of defects such as pores, undercut, and incomplete fusion), and the YOLOv11 weight is initialized using transfer learning, and trained for 100 epochs on an NVIDIA RTX4090 graphics card.

[0064] Further, the 3D point cloud processing and dimension measurement unit is used for preprocessing of the received 3D point cloud data, including denoising, registration, and workpiece coordinate system conversion. The defect position / ROI information provided by the 2D detection module or the point cloud geometric feature is used to extract the weld area point cloud, calculate the key dimensions such as weld width, weld height / extra height, weld leg, and theoretical weld thickness, and output the actual measurement values.

[0065] In the above, the denoising uses a statistical filtering algorithm to calculate the average distance μ and standard deviation σ of 50 neighborhood points for each point, and removes noise points that satisfy (d(p, μ) > kσ) (k = 1.8) and retains effective weld point cloud; registration is achieved by ICP algorithm to splice multi-view point cloud with an error controlled within 0.02mm; coordinate system conversion is to convert the point cloud from the sensor coordinate system to the workpiece coordinate system (with the weld starting point as the origin and the welding direction as the X axis);

[0066] Weld area point cloud extraction:

[0067] Based on the ROI information of 2D detection, the corresponding area (Z-axis range is ±5mm of the base material surface) is intercepted in 3D space, the weld and base material are segmented combined with geometric features (such as curvature mutation), and the weld point cloud subset is extracted.

[0068] When the 3D point cloud processing and dimension measurement unit calculates the weld width, a cross section perpendicular to the weld direction is taken, the maximum transverse distance of the weld contour in the cross section is taken, and based on the defect position / ROI information provided by the 2D detection module, the formula is:

[0069] W = max(X r -X l )

[0070] Where W is the weld width, X r and X l are the X coordinates of the right and left edges of the weld in the cross section, respectively.

[0071] The 3D point cloud processing and size measurement unit calculates the weld height / extra height by taking the vertical distance between the highest point of the weld and the reference plane of the base material surface, and the formula is:

[0072] H = Z max -Z base

[0073] Wherein, H is the weld height / extra height, Z max is the maximum Z coordinate value of the weld area point cloud, Z base is the Z coordinate value of the base material surface reference plane;

[0074] The 3D point cloud processing and size measurement unit calculates the theoretical weld thickness based on the effective load-bearing area estimation of the weld cross-sectional profile, and the formula is:

[0075]

[0076] Wherein, T is the theoretical weld thickness, A is the effective load-bearing area of the weld cross section, and L is the reference length of the weld cross section.

[0077] Further, the multi-modal information fusion and collaborative analysis unit is used to establish the mapping relationship between 2D pixel coordinates and 3D world coordinates based on the intrinsic and extrinsic calibration parameters of 2D cameras and 3D cameras, map 2D defect positions to 3D space, verify 2D suspected defects through 3D geometric features, trigger 2D review of 3D size exceeding areas, and optimize 2D ROI with 3D weld boundary; and form a fusion feature vector combining 2D texture features and 3D geometric features;

[0078] When the multi-modal information fusion and collaborative analysis unit forms a fusion feature vector, the 2D texture features and 3D geometric features are standardized and weighted fused, and the formula is:

[0079] F = α × F 2D +(1-α)×F 3D

[0080] Wherein, F is the fusion feature vector, F 2D is the standardized 2D texture feature vector, F 3D is the standardized 3D geometric feature vector, and α is the weight coefficient.

[0081] In the above, camera calibration: Zhang Zhengyou calibration method is used to obtain 2D camera intrinsic parameters (focal length, principal point) and extrinsic parameters (rotation matrix R and translation vector t with 3D sensor);

[0082] Mapping relationship: convert 2D pixel coordinates (u, v) to 3D world coordinates (X, Y, Z), and the formula is as follows:

[0083]

[0084] wherein K is a 2D camera intrinsic matrix.

[0085] In the above, information collaborative analysis:

[0086] Defect verification: for 2D detected suspected pores (confidence 0.7-0.8), check whether there is a depression (depth > 0.1 mm) in the corresponding 3D region, if there is, it is confirmed as a defect;

[0087] Size anomaly trigger: when the 3D measured weld width < 90% of the design value, trigger the 2D module to perform secondary detection on the region (enlarge ROI to 1.5 times the original size);

[0088] ROI optimization: adjust 2D ROI based on 3D weld boundary (Y direction ± 2 mm) to reduce base material area interference.

[0089] Further, the comprehensive quality determination and output unit is used to receive 2D defect information, 3D size information and fusion feature vector, determine the quality result based on a preset rule, output a quality determination report and transmit it to the welding equipment PLC, upper computer or MES system.

[0090] In the above, the determination rule:

[0091] Key defect determination: if cracks, incomplete fusion (2D confidence ≥ 0.8) are detected, it is directly determined as "unqualified";

[0092] Minor defect determination: the number of single small pores (diameter < 0.3 mm) is ≤ 3, and the 3D size of the region is qualified (weld width, weld height is within ± 5% of the design value), it is determined as "qualified"; otherwise, it is "unqualified";

[0093] Size determination: the theoretical weld thickness ≥ 95% of the design value, and the weld width and weld height are within the qualified range, it is determined as "qualified", and any size exceeds the standard is "unqualified".

[0094] Output content:

[0095] Quality report, including defect list (type, 3D position, size), size measurement value (compared with design value), determination result;

[0096] Visualization: 2D image with defect annotation, 3D point cloud model (defect area marked red);

[0097] Communication: transmit the results to the welding equipment PLC through the EtherCAT interface (response time < 100 ms), trigger an alarm signal (audible and visual alarm) when the standard is exceeded; at the same time, upload to the MES system (OPCUA protocol) for quality traceability.

[0098] A laser weld seam online quality detection method based on multi-modal visual fusion, comprising the following steps:

[0099] S1, a 2D vision acquisition module and a 3D vision acquisition module are controlled by a synchronous triggering device to synchronously acquire 2D images and 3D point cloud data of a weld seam based on an encoder signal or an external triggering signal;

[0100] S2, the 2D images are preprocessed by denoising, enhancing and ROI extraction, defects are detected by using the improved YOLOv11 model and information is output, the 3D point cloud is preprocessed by denoising, registration and coordinate system conversion, weld seam area point cloud is extracted, key dimensions are calculated by the formula and measurement values are output;

[0101] S3, 2D and 3D coordinate mapping is established based on camera calibration parameters, 2D defect positions are mapped to 3D space, fusion feature vectors are formed by the formula, information complementary verification and collaborative analysis are completed;

[0102] S4, quality results are comprehensively determined based on preset rules, reports and visualized contents are output.

[0103] The present application realizes comprehensive detection of weld seam quality through multi-modal information fusion, the detection precision reaches 0.01mm, the detection speed meets the online demand (single frame processing time <50ms), and the present application can be effectively applied to high-precision welding scenes such as automobiles and aerospace.

[0104] The basic principles and main features of the present application and the advantages of the present application are shown and described. It should be understood by those skilled in the art that the present application is not limited by the above examples, the above examples and descriptions in the specification are only to illustrate the principles of the present application, various changes and improvements can be made to the present application without departing from the spirit and scope of the present application, and these changes and improvements all fall within the scope of the present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A laser weld seam online quality inspection system based on multimodal vision fusion, characterized in that, include: The system includes a data acquisition and synchronization control unit, a 2D image processing and defect detection unit, a 3D point cloud processing and size measurement unit, a multimodal information fusion and collaborative analysis unit, and a comprehensive quality judgment and output unit. The data acquisition and synchronization control unit is used to control the 2D vision acquisition module and the 3D vision acquisition module to acquire 2D images and 3D point cloud data (X,Y,Z coordinates) of the weld at the same time point or under strict time sequence correspondence through the synchronization triggering device, and transmit the 2D images to the 2D image processing and defect detection unit, and transmit the 3D point cloud data to the 3D point cloud processing and dimension measurement unit. The 2D image processing and defect detection unit is used to preprocess the received 2D image by denoising, enhancing and extracting the ROI of the weld area, detect defects in the weld area by the trained YOLOv11 model, and output the defect type, 2D image coordinate position, confidence score and bounding box. The 3D point cloud processing and dimension measurement unit is used to preprocess the received 3D point cloud data by denoising, registration and workpiece coordinate system transformation. It extracts the weld area point cloud through the defect location / ROI information or point cloud geometric features provided by the 2D detection module, calculates key dimensions such as weld width, weld height / residual height, weld leg and theoretical weld thickness and outputs the actual measurement value. The multimodal information fusion and collaborative analysis unit is used to establish a mapping relationship between 2D pixel coordinates and 3D world coordinates based on the intrinsic and extrinsic calibration parameters of 2D and 3D cameras, mapping the 2D defect location to 3D space; verifying suspected 2D defects through 3D geometric features, triggering 2D re-inspection of 3D size-exceeding areas, optimizing 2D ROI with 3D weld boundaries; and forming a fused feature vector by combining 2D texture features and 3D geometric features. The integrated quality judgment and output unit is used to receive 2D defect information, 3D size information and fused feature vector, judge the quality result based on preset rules, output a quality judgment report and transmit it to the welding equipment PLC, host computer or MES system.

2. The online quality inspection system for laser welds based on multimodal vision fusion according to claim 1, characterized in that, In the 2D image processing and defect detection unit, the YOLOv11 model includes an improved Defect-YOLOv11. Defect-YOLOv11 incorporates an improved C3K2 module with adaptive sampling attention and an EMA-Sl ideLoss function with dynamic IoU threshold adjustment. The improved C3K2 module enhances the ability to capture subtle defect features through an adaptive sampling attention mechanism. The attention weight calculation formula is as follows: Among them, W i,j S represents the attention weights for image pixel (i,j). i,j This is the saliency value of the feature of that pixel; The EMA-SlideLoss function is used to optimize imbalanced sample problems. The formula is: in, The loss value is adjusted for the dynamic IoU threshold, and EMA is the exponential moving average operator.

3. The online quality inspection system for laser welds based on multimodal vision fusion according to claim 1, characterized in that, In the preprocessing of the 3D point cloud processing and size measurement unit, point cloud denoising employs a statistical filtering algorithm, the formula of which is: P out ={p∈P|d(p,μ)>kσ} Where P is the original point cloud, p is a point in the point cloud, μ is the average distance of the neighborhood points of point p, σ is the standard deviation of the distance, and k is a preset threshold (valued between 1.5 and 2.0). out This is the set of noise points to be removed.

4. The online quality inspection system for laser welds based on multimodal vision fusion according to claim 1, characterized in that, When calculating the weld width, the 3D point cloud processing and dimensional measurement unit takes a cross-section perpendicular to the weld direction, and calculates the maximum lateral distance of the weld contour within the cross-section. Based on the defect location / ROI information provided by the 2D detection module, the formula is: W=max(X r -X l ) Where W is the weld width, X r X l These are the X coordinates of the right and left edges of the weld within the cross-section, respectively.

5. The online quality inspection system for laser welds based on multimodal vision fusion according to claim 1, characterized in that, When the 3D point cloud processing and dimensional measurement unit calculates the weld height / remaining height, it takes the vertical distance between the highest point of the weld and the reference plane of the base material surface, and the formula is: H=Z max -Z base Where H is the weld height / remaining height, and Z is the weld height / remaining height. max Z represents the maximum Z-coordinate value of the point cloud in the weld area. base The Z-coordinate value is the reference plane of the base material surface.

6. The online quality inspection system for laser welds based on multimodal vision fusion according to claim 1, characterized in that: When the 3D point cloud processing and dimensional measurement unit calculates the theoretical weld thickness, the effective load-bearing area is estimated based on the weld cross-sectional profile using the following formula: Where T is the theoretical weld thickness, A is the effective bearing area of ​​the weld in the cross-section, and L is the reference length of the weld cross-section.

7. The online quality inspection system for laser welds based on multimodal vision fusion according to claim 1, characterized in that: When the multimodal information fusion and collaborative analysis unit forms the fused feature vector, it performs standardized weighted fusion of 2D texture features and 3D geometric features, using the following formula: F=α×F 2D +(1-a)×F 3D Where F is the fused feature vector, F 2D F is the standardized 2D texture feature vector. 3D α is the standardized 3D geometric feature vector, and α is the weighting coefficient.

8. A method for online quality inspection of laser welds based on multimodal vision fusion, using the system described in any one of claims 1-7, characterized in that, Includes the following steps: S1. The 2D vision acquisition module and the 3D vision acquisition module are controlled by a synchronous triggering device to synchronously acquire 2D images and 3D point cloud data of the weld seam based on encoder signals or external triggering signals. S2. Denoising, enhancement, and ROI extraction preprocessing are performed on the 2D image. The improved YOLOv11 model described in claim 2 is used to detect defects and output information. Denoising, registration, and coordinate system transformation preprocessing are performed on the 3D point cloud. The point cloud of the weld area is extracted. The key dimensions are calculated using the formulas described in claims 4-6 and the measured values ​​are output. S3. Based on the camera calibration parameters, establish a 2D and 3D coordinate mapping, map the 2D defect location to the 3D space, and form a fusion feature vector through the formula described in claim 7 to complete information complementarity verification and collaborative analysis. S4. Based on preset rules, comprehensively determine the quality results and output reports and visualizations.

Images