Laser welding full-process intelligent detection system based on machine vision and deep learning

Abstract

The application discloses a laser welding full-process intelligent detection system based on machine vision and deep learning, relates to the technical field of workpiece laser welding processing, and comprises a hardware execution module and a software processing module, and the two modules are cooperatively used to realize full-process detection of a weld; the hardware execution module comprises a visual acquisition submodule, a motion synchronization submodule and a data transmission submodule, the visual acquisition submodule comprises an adjustable-focus industrial camera, a ring laser transmitter and a detachable positioning support; the software processing module comprises a system calibration submodule, an image preprocessing submodule, a deep learning detection submodule, a defect quantification and judgment submodule and a result output submodule; the double-layer anti-interference structure formed by the narrowband filter and the diffuse reflection inhibition coating of the visual acquisition submodule effectively filters out the arc light and metal splashing interference in a welding site, and the pressure-sensitive coaxial positioning head is used to realize accurate coaxial positioning of the camera and the laser.

Description

Technical Field

[0001] This invention relates to the field of laser welding processing technology, and more specifically, to an intelligent inspection system for the entire laser welding process based on machine vision and deep learning. Background Technology

[0002] Laser welding is a crucial joining process in fields such as oil extraction, chemical equipment, and automobile manufacturing. The quality of the weld directly determines the structural strength and operational safety of the equipment. Especially in the welding of pressure-bearing components such as oil pipes and wellhead equipment used in oilfield extraction, defects such as weld cracks, incomplete penetration, and porosity can easily lead to crude oil leaks, equipment failures, and even safety accidents. Currently, the industry's inspection methods for laser welding are mainly divided into two categories: manual inspection and traditional machine vision inspection. Manual inspection relies on operators to observe the weld section by section with their naked eyes or handheld inspection tools. This not only suffers from poor consistency in defect identification due to limitations in human experience, but also makes it difficult to cover the entire length of long welds. It has a low recognition rate for small defects (such as porosity with a diameter <0.5mm and undercut with a depth <0.1mm). At the same time, the inspection efficiency is difficult to match with the welding progress, which can easily cause production bottlenecks. Traditional machine vision inspection, while using cameras and lasers to image weld seams, generally suffers from three core problems: First, it fails to optimize anti-interference structures for interference such as arc light and metal spatter at the welding site, resulting in distorted laser contour imaging and affecting feature extraction accuracy. Second, image preprocessing only uses simple grayscale segmentation or fixed threshold binarization, without designing unfolding algorithms based on the geometric characteristics of the laser ring light band, making it impossible to accurately restore the weld seam cross-sectional shape. Third, defect recognition relies heavily on traditional machine learning algorithms (such as SVM and decision trees), which lack generalization ability for multiple types and sizes of defects and fail to establish linkage with welding process parameters, making it difficult to achieve defect tracing and process optimization.

[0003] In addition, most existing detection systems are single-point detection modes, lacking full-process data storage and synchronous control mechanisms. This can easily lead to missed welds or a disconnect between the detection results and the welding process, failing to meet the needs of industrial-scale production for real-time control, accurate traceability, and efficient optimization of welding quality. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides an intelligent inspection system for the entire laser welding process based on machine vision and deep learning.

[0005] This invention provides an intelligent inspection system for the entire laser welding process based on machine vision and deep learning, including a hardware execution module and a software processing module, which work together to achieve full-process inspection of weld seams; The hardware execution module includes a vision acquisition submodule, a motion synchronization submodule, and a data transmission submodule. The vision acquisition submodule includes an adjustable-focus industrial camera, a ring-shaped laser emitter, and a detachable positioning bracket. The motion synchronization submodule is linked with the welding equipment to drive the vision acquisition submodule to move at a constant speed along the weld seam. The data transmission submodule connects the vision acquisition submodule and the software processing module. The software processing module includes a system calibration submodule, an image preprocessing submodule, a deep learning detection submodule, a defect quantification and judgment submodule, and a result output submodule. The system calibration submodule establishes a multi-coordinate system mapping relationship, the image preprocessing submodule processes the laser contour image, the deep learning detection submodule identifies the defect category, the defect quantification and judgment submodule calculates the defect size and determines the level, and the result output submodule outputs a report and alarm.

[0006] Preferably, the visual acquisition submodule is equipped with an anti-interference and adaptive positioning structure: the outer layer of the ring-shaped laser emitter is equipped with a narrow-band filter, the center wavelength of the narrow-band filter is matched with the laser wavelength, and the inner layer is equipped with a diffuse reflection suppression coating. The two work together to filter out welding arc light and metal spatter interference, ensuring the quality of laser contour imaging. The telescopic arm of the detachable positioning bracket is equipped with a pressure-sensitive coaxial positioning head at its end. When the arm contacts the workpiece surface, the pressure sensor feedback signal triggers angle locking, automatically correcting the coaxiality of the camera and the laser emitter to ensure accurate laser projection. The telescopic range of the arm is adapted to the detection needs of workpieces of different thicknesses. The clarity of the edge of the annular light band projected onto the weld surface by the laser meets the requirements for subsequent feature extraction.

[0007] Preferably, the system calibration submodule establishes a mapping relationship between the world coordinate system, camera coordinate system, and pixel coordinate system using a standard welded sample containing defects of known dimensions. The world coordinate system has the weld center as its origin and is marked as... - , , The camera coordinate system has its origin at the camera's optical center and is labeled oC-XcYcZc, while the pixel coordinate system is labeled kx-20-uv; specifically, it is established using the following formula: , Where s is a non-zero scaling factor and K is the camera intrinsic parameter matrix; , , These are the scale factors for the image's x-axis and y-axis, respectively. and The principal point coordinates of the pixel coordinate system are defined by Xc, Yc, and Zc, which are the coordinate components of point Pc in the camera coordinate system, and u and v, which are the coordinate components of point Pi in the pixel coordinate system. The transformation relationship between point Pw in the world coordinate system and point Pc in the camera coordinate system: , Where R is a 3×3 rotation matrix and t is a 3×1 translation matrix. , , , For translation components, , , Let Pw be the coordinate components of point Pw in the world coordinate system; Combining the above formulas, we obtain the mapping relationship between the world coordinate system and the pixel coordinate system: , The parameters K, R, and t are solved and stored using known characteristic points of a standard specimen for the quantitative calculation of defect size.

[0008] Preferably, the annular laser strip unfolding step of the image preprocessing submodule adopts the pixel mapping method, and the specific process is as follows: The center point and fitting radius of the laser ring region in the acquired image are extracted by the least squares fitting algorithm. The coordinates of the center point are marked as A and B, and the fitting radius is marked as R. The fitting accuracy is ensured to guarantee the subsequent unfolding effect. Let the inner diameter of the annular image to be unfolded be R1, the outer diameter be R2, and the width of the rectangular image to be unfolded be... Height is R1 and R2 are determined based on the fitted radius R, covering the entire laser ring region. and Based on the characteristics of R1, R2 and the annular circumference, it is adapted to the requirements of laser contour unfolding. The mapping relationship between pixels in a circular image and pixels in a rectangular image is as follows: , , The polar angle of a pixel in a circular image relative to its center point is expressed in radians and ranges from 0 to 2π. , are the pixel coordinate components of the circular image, and u and v are the pixel coordinate components of the rectangular image; For rectangular image pixels with non-integer coordinates, the gray value is calculated using the four-neighbor weighted summation method. The weight of the weighted summation is inversely proportional to the pixel distance, resulting in a complete rectangular unfolded image.

[0009] Preferably, the noise filtering in the image preprocessing submodule employs a median filtering algorithm, which achieves noise filtering by adapting the window size to the characteristics of the laser contour image; dynamic threshold binarization uses the local maximum gray-level averaging method, specifically: Calculate the maximum grayscale value of each column of the rectangular unfolded image. The maximum grayscale value is denoted as Gmax(x), where x is the column coordinate. Set a threshold function. Where s is the grayscale coefficient and t is the offset, both of which are adaptively adjusted based on the characteristics of the welding material. Threshold for each column; Binarize each column of the image: , in The original grayscale values ​​of the rectangular unfolded image. This is the binarized pixel matrix, where 255 represents the laser light strip, 0 represents the background, and y is the row coordinate.

[0010] Preferably, the image preprocessing submodule further includes a laser centerline extraction step, wherein the laser centerline extraction adopts the gray-scale centroid method, and the specific formula is as follows: , Where yk is the gray-scale centroid coordinate of the x-th column pixel, which is the y-coordinate of the laser center line in the x-th column, m is the number of pixels in the x-th column with a gray value ≥ T(x), and G(x,y) is the gray value of the pixel after binarization; by extracting yk of each column through equation (8), the complete laser contour center line is obtained.

[0011] Preferably, the defect identification model of the deep learning detection submodule is an improved YOLOv8 model; the training dataset of the improved YOLOv8 model contains a sufficient number of weld images, covering multiple types of defects and different welding materials, and expands the sample diversity after data augmentation processing such as rotation, flipping, and noise addition to ensure the model's generalization ability. The model input is a grayscale image extracted from the laser centerline, and the output is the defect category, bounding box coordinates, and confidence score. The defect categories include cracks, porosity, incomplete penetration, undercut, and weld beads. The bounding box coordinates are based on the pixel coordinate system, and the confidence score criterion is determined based on the reliability requirements of defect recognition, effectively avoiding false detections.

[0012] Preferably, the defect quantization and glitch signal filtering process of the defect quantization and judgment submodule is as follows: Defect size quantization: The pixel coordinates of the defect bounding box are reverse-mapped to world coordinates. The pixel coordinates of the defect bounding box are labeled as xi and yi, and the world coordinates are labeled as Xw, Yw, and Zw. The actual size is calculated using the following formula: , , di is the distance from the defect boundary point to the weld center, the weld center coordinates are x0, y0, and k is the pixel-to-actual size conversion coefficient, which is obtained by system calibration to ensure quantization accuracy. SC is the defect deformation, D is the standard weld diameter, and SC is determined to be a valid defect based on weld quality requirements; Glitch signal filtering: If multiple frames of images are continuously acquired, and the SC at the same position all meet the valid defect judgment and the defect area meets the characteristics of a real defect, then the defect is confirmed to exist; if the SC at the same position does not meet the valid defect judgment or appears discontinuously in consecutive frames, it is judged as a glitch signal and filtered out. Glitch signals include interference signals generated by splashing, obstruction, etc.

[0013] Preferably, the hardware execution module includes a process-detection data linkage storage and closed-loop synchronization unit: The hardware execution module also includes a local storage submodule, which has the capacity to meet the data storage needs of the entire process. It stores image data, defect detection results and welding process parameters in association with precise timestamps. Welding process parameters include welding current, voltage, speed and shielding gas flow rate, ensuring efficient response to data queries. The motion synchronization submodule adopts dual closed-loop control of encoder + visual feedback. The encoder acquires the speed signal of the welding equipment in real time, and the visual feedback unit corrects the movement speed through laser contour offset, so that the motion synchronization accuracy between the visual acquisition submodule and the welding equipment is adapted to the welding speed range, ensuring that there are no missed welds.

[0014] Preferably, the result output submodule includes a defect-process correlation analysis and hierarchical early warning unit: The output module supports multi-terminal synchronization, including local terminals, remote terminals, and mobile terminals. The local terminal is the welding equipment operation screen, which displays the detection screen and defect warnings in real time. When fatal defects such as cracks or incomplete penetration are detected, the equipment is triggered to stop the production of unqualified welds in a timely manner. The remote terminal is the monitoring center server, which supports historical data retrieval on the web. The retrieval dimensions include time, workpiece number, and defect type. It automatically generates a heat map of the correlation between defects and process parameters, which intuitively shows the trend of the influence of process parameters on the defect incidence rate. The mobile terminal is an app for management personnel, which receives tiered alarm push notifications. Tiered alarms include audible and visual alarms and SMS alarms triggered by critical defects, and pop-up alarms triggered by general defects. The inspection report automatically marks the defect location and the corresponding welding process parameters.

[0015] Beneficial effects: This invention uses a double-layer anti-interference structure consisting of a narrow-band filter and a diffuse reflection suppression coating in the visual acquisition submodule to effectively filter out interference from arc light and metal spatter at the welding site. Combined with a pressure-sensitive coaxial positioning head, it achieves precise coaxial positioning of the camera and laser, ensuring that the edge of the annular light band formed by the laser projection onto the weld surface is clear and stable.

[0016] Meanwhile, the image preprocessing submodule uses the least squares fitting algorithm to extract the center point and radius of the laser ring region, expands the ring light band into a rectangular image through pixel mapping, and then combines median filtering and adaptive dynamic threshold binarization to avoid laser contour imaging distortion, providing a high-fidelity feature basis for subsequent defect detection. Compared with traditional machine vision detection, the accuracy of laser contour extraction and the recognition rate of small defects are greatly improved. Full-process automation and real-time optimization: The motion synchronization submodule of the hardware execution module adopts a dual closed-loop control method combining encoders and visual feedback to correct the movement speed of the visual acquisition submodule in real time, achieving dynamic coordination with the welding equipment and effectively avoiding weld seam omissions. The deep learning detection submodule of the software processing module uses an improved YOLOv8 lightweight model. By optimizing the model backbone structure, it ensures high defect detection accuracy while the inference speed is fully adapted to the real-time detection needs of industrial sites. The entire system can autonomously complete the entire detection process from image acquisition, preprocessing, defect identification, size quantization to early warning without manual intervention. The detection efficiency significantly surpasses manual inspection, completely solving the production bottleneck problem caused by the disconnect between traditional inspection methods and welding progress. Breakthrough in Defect Tracing and Process Optimization Capabilities: The hardware execution module of this invention is equipped with a local storage submodule, which can associate and store image data, defect detection results, and welding process parameters according to precise timestamps, enabling full-process traceability of welding quality and facilitating subsequent quality problem tracing and analysis. The results output submodule can automatically generate a heatmap of the correlation between defects and process parameters, intuitively displaying the influence trend of different process parameters on the defect incidence rate. Simultaneously, a tiered early warning mechanism is constructed, which can trigger equipment shutdown signals to terminate the production of unqualified welds for critical defects and provide timely alerts for general defects. This design not only enables real-time control of welding quality but also provides targeted data support for welding process adjustments, significantly reducing material loss and cost waste caused by rework due to defects. Enhanced Scene Adaptability and Generalization: The telescopic arm of the vision acquisition submodule has adaptive adjustment capabilities, allowing for flexible adjustment of the positioning position based on workpiece thickness to meet the inspection needs of workpieces of different specifications. The system calibration submodule establishes multi-coordinate system mapping relationships through standard welding samples, ensuring consistent inspection of different welding materials and weld types. The improved model of the deep learning inspection submodule, trained with diverse data augmentation, covers multiple types of defect samples. Its generalization recognition ability for various defect types such as cracks, porosity, incomplete penetration, undercut, and weld beads is significantly superior to traditional machine learning algorithms. It can be widely applied to laser welding inspection scenarios in various fields such as oil pipelines, chemical containers, and automotive parts, with an applicability far exceeding that of existing single-point inspection systems. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0018] like Figure 1 As shown: A laser welding full-process intelligent inspection system based on machine vision and deep learning, including a hardware execution module and a software processing module, which work together to achieve full-process inspection of weld seams; The hardware execution module includes a vision acquisition submodule, a motion synchronization submodule, and a data transmission submodule. The vision acquisition submodule includes an adjustable-focus industrial camera, a ring-shaped laser emitter, and a detachable positioning bracket. The motion synchronization submodule is linked with the welding equipment to drive the vision acquisition submodule to move at a constant speed along the weld seam. The data transmission submodule connects the vision acquisition submodule and the software processing module. The software processing module includes a system calibration submodule, an image preprocessing submodule, a deep learning detection submodule, a defect quantification and judgment submodule, and a result output submodule. The system calibration submodule establishes a multi-coordinate system mapping relationship, the image preprocessing submodule processes the laser contour image, the deep learning detection submodule identifies the defect category, the defect quantification and judgment submodule calculates the defect size and determines the level, and the result output submodule outputs a report and alarm.

[0019] It should be noted that the inspection process is as follows: multi-coordinate system calibration → weld laser contour acquisition → image feature optimization → defect identification and size quantification → defect level determination → inspection result output and early warning.

[0020] This system achieves automated inspection of the entire welding process through the collaborative design of hardware and software modules. It can complete the entire process from image acquisition to defect warning without human intervention, significantly improving the efficiency and accuracy of welding inspection and making it suitable for industrial-scale welding production scenarios. As an optional embodiment, the visual acquisition submodule is equipped with an anti-interference and adaptive positioning structure: the outer layer of the ring-shaped line laser emitter is equipped with a narrow-band filter, the center wavelength of the narrow-band filter is matched with the laser wavelength, and the inner layer is equipped with a diffuse reflection suppression coating. The two work together to filter out welding arc light and metal spatter interference, ensuring the quality of laser contour imaging. The telescopic arm of the detachable positioning bracket is equipped with a pressure-sensitive coaxial positioning head at its end. When the arm contacts the workpiece surface, the pressure sensor feedback signal triggers angle locking, automatically correcting the coaxiality of the camera and the laser emitter to ensure accurate laser projection. The telescopic range of the arm is adapted to the detection needs of workpieces of different thicknesses. The clarity of the edge of the annular light band projected onto the weld surface by the laser meets the requirements for subsequent feature extraction.

[0021] This anti-interference and adaptive positioning structure can effectively resist interference from the complex environment of the welding site, avoid contour imaging distortion caused by arc light and spatter, and adapt to workpieces of different thicknesses through adaptive positioning to ensure stable and complete laser contour features, laying a reliable foundation for subsequent image preprocessing and defect detection.

[0022] As an optional embodiment, the system calibration submodule establishes a mapping relationship between the world coordinate system, camera coordinate system, and pixel coordinate system using a standard welded sample containing defects of known dimensions. The world coordinate system has the weld center as its origin and is marked as... - , , The camera coordinate system has its origin at the camera's optical center and is labeled oC-XcYcZc, while the pixel coordinate system is labeled kx-20-uv; specifically, it is established using the following formula: , Where s is a non-zero scaling factor and K is the camera intrinsic parameter matrix; , , These are the scale factors for the image's x-axis and y-axis, respectively. and The principal point coordinates of the pixel coordinate system are defined by Xc, Yc, and Zc, which are the coordinate components of point Pc in the camera coordinate system, and u and v, which are the coordinate components of point Pi in the pixel coordinate system. The transformation relationship between point Pw in the world coordinate system and point Pc in the camera coordinate system: , Where R is a 3×3 rotation matrix and t is a 3×1 translation matrix. , , , For translation components, , , Let Pw be the coordinate components of point Pw in the world coordinate system; Combining the above formulas, we obtain the mapping relationship between the world coordinate system and the pixel coordinate system: , The parameters K, R, and t are solved and stored using known characteristic points of a standard specimen for the quantitative calculation of defect size.

[0023] It should be noted that this multi-coordinate system mapping relationship and parameter solving method provides an accurate coordinate transformation benchmark for defect size quantification, establishes a relationship between image pixel coordinates and actual physical space coordinates, ensures the accuracy of subsequent actual defect size calculation, and lays a data foundation for accurate defect level determination. As an optional embodiment, the annular laser strip unfolding step of the image preprocessing submodule adopts the pixel mapping method, and the specific process is as follows: The center point and fitting radius of the laser ring region in the acquired image are extracted by the least squares fitting algorithm. The coordinates of the center point are marked as A and B, and the fitting radius is marked as R. The fitting accuracy is ensured to guarantee the subsequent unfolding effect. Let the inner diameter of the annular image to be unfolded be R1, the outer diameter be R2, and the width of the rectangular image to be unfolded be... Height is R1 and R2 are determined based on the fitted radius R, covering the entire laser ring region. and Based on the characteristics of R1, R2 and the annular circumference, it is adapted to the requirements of laser contour unfolding. The mapping relationship between pixels in a circular image and pixels in a rectangular image is as follows: , , The polar angle of a pixel in a circular image relative to its center point is expressed in radians and ranges from 0 to 2π. , are the pixel coordinate components of the circular image, and u and v are the pixel coordinate components of the rectangular image; For rectangular image pixels with non-integer coordinates, the gray value is calculated using the four-neighbor weighted summation method. The weight of the weighted summation is inversely proportional to the pixel distance, resulting in a complete rectangular unfolded image.

[0024] By using this pixel mapping method and grayscale value calculation method, the annular laser strip can be transformed into a rectangular image that is easy to extract features. At the same time, it avoids the image from being discontinuous or distorted after unfolding, ensuring the continuity and integrity of the laser contour, and providing a clear and standardized image basis for subsequent laser centerline extraction and defect feature identification. As an optional embodiment, the noise filtering of the image preprocessing submodule adopts a median filtering algorithm, which achieves noise filtering by adapting the window size to the characteristics of the laser contour image; the dynamic threshold binarization adopts the local maximum gray-level averaging method, specifically: Calculate the maximum grayscale value of each column of the rectangular unfolded image. The maximum grayscale value is denoted as Gmax(x), where x is the column coordinate. Set a threshold function. Where s is the grayscale coefficient and t is the offset, both of which are adaptively adjusted based on the characteristics of the welding material. Threshold for each column; Binarize each column of the image: , in The original grayscale values ​​of the rectangular unfolded image. This is the binarized pixel matrix, where 255 represents the laser light strip, 0 represents the background, and y is the row coordinate.

[0025] It should be noted that by using a median filtering algorithm to achieve targeted noise removal and retain the effective features of the laser band, combined with dynamic threshold binarization processing based on material adaptive adjustment, the laser band can be effectively distinguished from the background area. Even when faced with image grayscale differences of different welding materials, the light band segmentation effect can be guaranteed, significantly highlighting the laser contour features of the weld seam and reducing the impact of interference factors on subsequent detection.

[0026] As an optional embodiment, the image preprocessing submodule further includes a laser centerline extraction step, wherein the laser centerline extraction adopts the gray-scale centroid method, and the specific formula is: , Where yk is the gray-scale centroid coordinate of the x-th column pixel, which is the y-coordinate of the laser center line in the x-th column, m is the number of pixels in the x-th column with a gray value ≥ T(x), and G(x,y) is the gray value of the pixel after binarization; by extracting yk of each column through equation (8), the complete laser contour center line is obtained.

[0027] It should be noted that by extracting the laser centerline using this gray-scale centroid method, the gray-scale distribution characteristics of the laser band can be fully utilized, effectively eliminating the centerline offset problem caused by the unevenness of the laser band, and obtaining a continuous and smooth laser contour centerline. This centerline can accurately reflect the actual shape of the weld, providing accurate feature input for the deep learning detection submodule and improving the accuracy and reliability of defect identification.

[0028] As an optional embodiment, the defect identification model of the deep learning detection submodule is an improved YOLOv8 model; the training dataset of the improved YOLOv8 model contains a sufficient number of weld images, covering multiple types of defects and different welding materials, and expands the sample diversity after data augmentation processing such as rotation, flipping, and noise addition to ensure the model's generalization ability. The model backbone adopts the MobileNetV3 architecture to achieve a lightweight design to meet the needs of real-time industrial inspection; the model output layer uses the CIoU loss function to optimize the detection capability of small-sized defects and ensure high recognition accuracy of multiple types of weld defects. The model input is a grayscale image extracted from the laser centerline, and the output is the defect category, bounding box coordinates, and confidence score. The defect categories include cracks, porosity, incomplete penetration, undercut, and weld beads. The bounding box coordinates are based on the pixel coordinate system, and the confidence score criterion is determined based on the reliability requirements of defect recognition, effectively avoiding false detections.

[0029] This improved YOLOv8 model, through its lightweight structural design, ensures real-time inference speed while combining diverse training samples and an optimized loss function to achieve accurate identification of multiple types and sizes of weld defects. The confidence level determination mechanism further reduces the risk of false detection, adapts to the dual requirements of detection efficiency and accuracy in industrial sites, and provides intelligent recognition support for welding quality control.

[0030] As an optional embodiment, the defect quantization and glitch signal filtering process of the defect quantization and judgment submodule is as follows: Defect size quantization: The pixel coordinates of the defect bounding box are reverse-mapped to world coordinates. The pixel coordinates of the defect bounding box are labeled as xi and yi, and the world coordinates are labeled as Xw, Yw, and Zw. The actual size is calculated using the following formula: , , di is the distance from the defect boundary point to the weld center, the weld center coordinates are x0, y0, and k is the pixel-to-actual size conversion coefficient, which is obtained by system calibration to ensure quantization accuracy. SC is the defect deformation, D is the standard weld diameter, and SC is determined to be a valid defect based on weld quality requirements; Glitch signal filtering: If multiple frames of images are continuously acquired, and the SC at the same position all meet the valid defect judgment and the defect area meets the characteristics of a real defect, then the defect is confirmed to exist; if the SC at the same position does not meet the valid defect judgment or appears discontinuously in consecutive frames, it is judged as a glitch signal and filtered out. Glitch signals include interference signals generated by splashing, obstruction, etc.

[0031] This defect size quantification method can accurately convert image coordinates into actual physical dimensions, providing a quantitative basis for defect level determination. Combined with multi-frame verified burr signal filtering logic, it can effectively distinguish between real defects and on-site interference signals, ensuring the reliability of defect determination, avoiding misjudgments caused by interference, and providing accurate data support for welding quality assessment.

[0032] As an optional embodiment, the hardware execution module design process – detection data linkage storage and closed-loop synchronization unit: The hardware execution module also includes a local storage submodule, which has the capacity to meet the data storage needs of the entire process. It stores image data, defect detection results and welding process parameters in association with precise timestamps. Welding process parameters include welding current, voltage, speed and shielding gas flow rate, ensuring efficient response to data queries. The motion synchronization submodule adopts dual closed-loop control of encoder + visual feedback. The encoder acquires the speed signal of the welding equipment in real time, and the visual feedback unit corrects the movement speed through laser contour offset, so that the motion synchronization accuracy between the visual acquisition submodule and the welding equipment is adapted to the welding speed range, ensuring that there are no missed welds.

[0033] The process-inspection data linkage storage unit enables precise correlation between welding process data and inspection results, solving the problem of welding quality traceability and improving the efficiency of quality problem tracing with efficient data query response; the motion synchronization unit with dual closed-loop control ensures dynamic coordination between visual acquisition and welding process, avoiding missed weld inspections due to synchronization deviation, ensuring the integrity of the entire process inspection and data traceability, and supporting welding process optimization and quality control.

[0034] As an optional embodiment, the result output submodule includes a defect-process correlation analysis and hierarchical early warning unit: The output module supports multi-terminal synchronization, including local terminals, remote terminals, and mobile terminals. The local terminal is the welding equipment operation screen, which displays the detection screen and defect warnings in real time. When fatal defects such as cracks or incomplete penetration are detected, the equipment is triggered to stop the production of unqualified welds in a timely manner. The remote terminal is the monitoring center server, which supports historical data retrieval on the web. The retrieval dimensions include time, workpiece number, and defect type. It automatically generates a heat map of the correlation between defects and process parameters, which intuitively shows the trend of the influence of process parameters on the defect incidence rate. The mobile terminal is an app for management personnel, which receives tiered alarm push notifications. Tiered alarms include audible and visual alarms and SMS alarms triggered by critical defects, and pop-up alarms triggered by general defects. The inspection report automatically marks the defect location and the corresponding welding process parameters.

[0035] The defect-process correlation analysis and graded early warning unit enables real-time risk management of the welding process, preventing the continuous production of substandard welds; it provides targeted data support for process optimization through correlation heat maps, helping to accurately locate process parameters affecting welding quality; the graded alarm mechanism ensures timely risk communication, and automatically labeled inspection reports further improve the efficiency of process adjustment, comprehensively reducing the quality risks and cost losses in welding production.

[0036] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of this template.

Claims

1. A fully intelligent inspection system for laser welding based on machine vision and deep learning, characterized in that, It includes a hardware execution module and a software processing module, which work together to achieve full-process inspection of weld seams; The hardware execution module includes a vision acquisition submodule, a motion synchronization submodule, and a data transmission submodule. The vision acquisition submodule includes an adjustable-focus industrial camera, a ring-shaped laser emitter, and a detachable positioning bracket. The motion synchronization submodule is linked with the welding equipment to drive the vision acquisition submodule to move at a constant speed along the weld seam. The data transmission submodule connects the vision acquisition submodule and the software processing module. The software processing module includes a system calibration submodule, an image preprocessing submodule, a deep learning detection submodule, a defect quantification and judgment submodule, and a result output submodule. The system calibration submodule establishes a multi-coordinate system mapping relationship, the image preprocessing submodule processes the laser contour image, the deep learning detection submodule identifies the defect category, the defect quantification and judgment submodule calculates the defect size and determines the level, and the result output submodule outputs a report and alarm.

2. The intelligent detection system for the entire laser welding process based on machine vision and deep learning according to claim 1, characterized in that, The visual acquisition submodule is equipped with an anti-interference and adaptive positioning structure: the outer layer of the ring-shaped linear laser emitter is equipped with a narrow-band filter, the center wavelength of the narrow-band filter is matched with the laser wavelength, and the inner layer is equipped with a diffuse reflection suppression coating. The two work together to filter out welding arc light and metal spatter interference, ensuring the quality of laser contour imaging. The telescopic arm of the detachable positioning bracket is equipped with a pressure-sensitive coaxial positioning head at its end. When the arm contacts the workpiece surface, the pressure sensor feedback signal triggers angle locking, automatically correcting the coaxiality of the camera and the laser emitter to ensure accurate laser projection. The telescopic range of the arm is adapted to the detection needs of workpieces of different thicknesses. The clarity of the edge of the annular light band projected onto the weld surface by the laser meets the requirements for subsequent feature extraction.

3. The intelligent inspection system for the entire laser welding process based on machine vision and deep learning according to claim 1, characterized in that, The system calibration submodule establishes a mapping relationship between the world coordinate system, camera coordinate system, and pixel coordinate system using a standard welded sample containing defects of known dimensions. The world coordinate system has the weld center as its origin and is marked as... - , , The camera coordinate system has the camera optical center as its origin and is marked as oC-XcYcZc, and the pixel coordinate system is marked as kx-20-uv; Specifically, it is established using the following formula: , Where s is a non-zero scaling factor and K is the camera intrinsic parameter matrix; ， , These are the x-axis and y-axis scale factors of the image, respectively. and The principal point coordinates of the pixel coordinate system are defined by Xc, Yc, and Zc, which are the coordinate components of point Pc in the camera coordinate system, and u and v, which are the coordinate components of point Pi in the pixel coordinate system. The transformation relationship between point Pw in the world coordinate system and point Pc in the camera coordinate system: , Where R is a 3×3 rotation matrix and t is a 3×1 translation matrix. , , , For translation components, , , Let Pw be the coordinate components of point Pw in the world coordinate system; Combining the above formulas, we obtain the mapping relationship between the world coordinate system and the pixel coordinate system: , The parameters K, R, and t are solved and stored using known characteristic points of a standard specimen for the quantitative calculation of defect size.

4. The intelligent inspection system for the entire laser welding process based on machine vision and deep learning according to claim 3, characterized in that, The annular laser strip unfolding step of the image preprocessing submodule adopts the pixel mapping method, and the specific process is as follows: The center point and fitting radius of the laser ring region in the acquired image are extracted by the least squares fitting algorithm. The coordinates of the center point are marked as A and B, and the fitting radius is marked as R. The fitting accuracy is ensured to guarantee the subsequent unfolding effect. Let the inner diameter of the annular image to be unfolded be R1, the outer diameter be R2, and the width of the rectangular image to be unfolded be... Height is R1 and R2 are determined based on the fitted radius R, covering the entire laser ring region. and Based on the characteristics of R1, R2 and the annular circumference, it is adapted to the requirements of laser contour unfolding. The mapping relationship between pixels in a circular image and pixels in a rectangular image is as follows: , ， The polar angle of a pixel in a circular image relative to its center point is expressed in radians and ranges from 0 to 2π. , are the pixel coordinate components of the circular image, and u and v are the pixel coordinate components of the rectangular image; For rectangular image pixels with non-integer coordinates, the gray value is calculated using the four-neighbor weighted summation method. The weight of the weighted summation is inversely proportional to the pixel distance, resulting in a complete rectangular unfolded image.

5. The intelligent inspection system for the entire laser welding process based on machine vision and deep learning according to claim 4, characterized in that, The image preprocessing submodule employs a median filtering algorithm for noise removal. This algorithm achieves noise removal by adapting the window size to the characteristics of the laser contour image. Dynamic threshold binarization uses the local maximum gray-level averaging method, specifically: Calculate the maximum grayscale value of each column of the rectangular unfolded image. The maximum grayscale value is denoted as Gmax(x), where x is the column coordinate. Set a threshold function. Where s is the grayscale coefficient and t is the offset, both of which are adaptively adjusted based on the characteristics of the welding material. Threshold for each column; Binarize each column of the image: ， in The original grayscale values ​​of the rectangular unfolded image. This is the binarized pixel matrix, where 255 represents the laser light strip, 0 represents the background, and y is the row coordinate.

6. The intelligent inspection system for the entire laser welding process based on machine vision and deep learning according to claim 5, characterized in that, The image preprocessing submodule also includes a laser centerline extraction step, which uses the gray-scale centroid method for laser centerline extraction. The specific formula is as follows: ， Where yk is the gray-scale centroid coordinate of the x-th column pixel, which is the y-coordinate of the laser center line in the x-th column, m is the number of pixels in the x-th column with a gray value ≥ T(x), and G(x,y) is the gray value of the pixel after binarization; by extracting yk of each column through equation (8), the complete laser contour center line is obtained.

7. The intelligent inspection system for the entire laser welding process based on machine vision and deep learning according to claim 6, characterized in that, The defect identification model of the deep learning detection submodule is an improved YOLOv8 model. The training dataset of the improved YOLOv8 model contains a sufficient number of weld images, covering multiple types of defects and different welding materials. After data augmentation processing such as rotation, flipping, and noise addition, the sample diversity is expanded to ensure the model's generalization ability. The model input is a grayscale image extracted from the laser centerline, and the output is the defect category, bounding box coordinates, and confidence score. The defect categories include cracks, porosity, incomplete penetration, undercut, and weld beads. The bounding box coordinates are based on the pixel coordinate system, and the confidence score criterion is determined based on the reliability requirements of defect recognition, effectively avoiding false detections.

8. The intelligent inspection system for the entire laser welding process based on machine vision and deep learning according to claim 7, characterized in that, The defect quantization and glitch signal filtering process of the defect quantization and judgment submodule is as follows: Defect size quantization: The pixel coordinates of the defect bounding box are reverse-mapped to world coordinates. The pixel coordinates of the defect bounding box are labeled as xi and yi, and the world coordinates are labeled as Xw, Yw, and Zw. The actual size is calculated using the following formula: ， ， di is the distance from the defect boundary point to the weld center, the weld center coordinates are x0, y0, and k is the pixel-to-actual size conversion coefficient, which is obtained by system calibration to ensure quantization accuracy. SC is the defect deformation, D is the standard weld diameter, and SC is determined to be a valid defect based on weld quality requirements; Filtering out burr signals: If multiple frames of images are continuously acquired, and the SCs at the same location all meet the criteria for valid defects and the defect area satisfies the characteristics of a real defect, then the existence of the defect is confirmed. If the SC at this position does not meet the valid defect judgment or appears discontinuously in consecutive frames, it is judged as a glitch signal and filtered out. Glitch signals include interference signals generated by splashing, obstruction, etc.

9. The intelligent detection system for the entire laser welding process based on machine vision and deep learning according to claim 1, characterized in that, The hardware execution module and the process-detection data linkage storage and closed-loop synchronization unit: The hardware execution module also includes a local storage submodule, which has the capacity to meet the data storage needs of the entire process. It stores image data, defect detection results and welding process parameters in association with precise timestamps. Welding process parameters include welding current, voltage, speed and shielding gas flow rate, ensuring efficient response to data queries. The motion synchronization submodule adopts dual closed-loop control of encoder + visual feedback. The encoder acquires the speed signal of the welding equipment in real time, and the visual feedback unit corrects the movement speed through laser contour offset, so that the motion synchronization accuracy between the visual acquisition submodule and the welding equipment is adapted to the welding speed range, ensuring that there are no missed welds.

10. The intelligent inspection system for the entire laser welding process based on machine vision and deep learning according to claim 1, characterized in that, The result output submodule includes a defect-process correlation analysis and hierarchical early warning unit: The output module supports multi-terminal synchronization, including local terminals, remote terminals, and mobile terminals. The local terminal is the welding equipment operation screen, which displays the detection screen and defect warnings in real time. When fatal defects such as cracks or incomplete penetration are detected, the equipment is triggered to stop the production of unqualified welds in a timely manner. The remote terminal is the monitoring center server, which supports historical data retrieval on the web. The retrieval dimensions include time, workpiece number, and defect type. It automatically generates a heat map of the correlation between defects and process parameters, which intuitively shows the trend of the influence of process parameters on the defect incidence rate. The mobile terminal is an app for management personnel, which receives tiered alarm push notifications. Tiered alarms include audible and visual alarms and SMS alarms triggered by critical defects, and pop-up alarms triggered by general defects. The inspection report automatically marks the defect location and the corresponding welding process parameters.

Images