Weld defect detection method, device and related equipment

Abstract

The present application provides a kind of weld defect detection method, device and related equipment, method includes: obtaining the two-dimensional image data and three-dimensional scanning data of target joint weld area;Determine the tube plate hole center position and the weld center position based on two-dimensional image data and three-dimensional scanning data, and calculate the center offset according to tube plate hole center position and weld center position;Based on three-dimensional scanning data and weld center position, extract weld profile data, and calculate the actual size data of target joint weld area according to weld profile data;Two-dimensional image data is input into pre-trained defect recognition model for identification, and the defect prediction result of target joint weld area is obtained;Based on center offset, actual size data and defect prediction result, combined with preset weld standard parameter, output the quality determination result of target joint weld area.The present application can improve the detection accuracy of weld defect.

Description

Technical Field

[0001] This application relates to the field of intelligent inspection, and in particular to a method, apparatus and related equipment for detecting weld defects. Background Technology

[0002] In the industrial manufacturing sector, especially in key equipment such as steam generators and heat exchange containers in nuclear power plants, tube-to-tube sheet welded structures are widely used. The quality of these welds directly determines the safety and service life of critical nuclear power equipment. Currently, tube-to-tube sheet welding robots mainly use lasers to scan the weld seams, collect weld appearance and dimensional data, and compare the collected data with preset specified values. If the data exceeds the specified values, the weld is judged as unqualified.

[0003] In related technologies, existing weld defect detection methods mainly rely on single sensor data, such as 2D images or 3D laser scanning that can only measure surface dimensions. These methods have limited feature dimensions and poor anti-interference capabilities. When faced with complex and variable working conditions, they are highly susceptible to interference from high image noise and low contrast, and lack cross-validation of multi-dimensional data. This results in insufficient ability to identify minute and complex defects such as depth deviations, local protrusions, and incomplete fusion on the inner side, leading to low detection accuracy. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application provides a method, apparatus, and related equipment for detecting weld defects, which can improve the accuracy of weld defect detection.

[0005] In a first aspect, embodiments of this application provide a method for detecting weld defects, the method comprising: Two-dimensional image data and three-dimensional scan data of the target joint weld area are acquired; wherein, the target joint weld area is the joint weld area formed by welding a tubular workpiece and a plate-shaped workpiece; Based on the two-dimensional image data and the three-dimensional scan data, the center position of the tube sheet hole and the center position of the weld are determined, and the center offset is calculated according to the center position of the tube sheet hole and the center position of the weld. Based on the three-dimensional scanning data and the weld center position, weld profile data is extracted, and the actual size data of the target joint weld area is calculated based on the weld profile data. The two-dimensional image data is input into a pre-trained defect recognition model for recognition, and the defect prediction result of the target joint weld area is obtained. Based on the center offset, the actual size data, and the defect prediction results, combined with the preset weld standard parameters, the quality judgment result of the target joint weld area is output.

[0006] Secondly, embodiments of this application provide a weld defect detection device, the device comprising: The acquisition module is used to acquire two-dimensional image data and three-dimensional scan data of the target joint weld area, wherein the target joint weld area is the joint weld area formed by welding a tubular workpiece and a plate workpiece. The determination module is used to determine the center position of the tube sheet hole and the center position of the weld based on the two-dimensional image data and the three-dimensional scan data, and to calculate the center offset based on the center position of the tube sheet hole and the center position of the weld. The extraction module is used to extract weld profile data based on the three-dimensional scanning data and the weld center position, and to calculate the actual size data of the weld area of ​​the target joint based on the weld profile data; The input module is used to input the two-dimensional image data into a pre-trained defect recognition model for recognition, and obtain the defect prediction result of the target joint weld area; The output module is used to output the quality judgment result of the weld area of ​​the target joint based on the center offset, the actual size data and the defect prediction result, combined with the preset weld standard parameters.

[0007] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the weld defect detection method as described in any one of the embodiments of the first aspect of this application.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that is executed by a processor to implement the weld defect detection method as described in any one of the embodiments of the first aspect of this application.

[0009] The weld defect detection method according to an embodiment of this application includes: acquiring two-dimensional image data and three-dimensional scanning data of a target joint weld area; wherein, the target joint weld area is a joint weld area formed by welding a tubular workpiece and a plate-shaped workpiece; determining the center position of the tube sheet hole and the center position of the weld based on the two-dimensional image data and the three-dimensional scanning data, and calculating the center offset based on the center position of the tube sheet hole and the center position of the weld; extracting weld profile data based on the three-dimensional scanning data and the weld center position, and calculating the actual size data of the target joint weld area based on the weld profile data; inputting the two-dimensional image data into a pre-trained defect recognition model for recognition, and obtaining a defect prediction result for the target joint weld area; and outputting a quality judgment result for the target joint weld area based on the center offset, the actual size data, and the defect prediction result, combined with preset weld standard parameters.

[0010] This application acquires two-dimensional image data and three-dimensional scan data of the weld area of ​​the target joint, and calculates the center offset by jointly determining the center of the tube sheet hole and the weld center based on the above two types of data. This allows the planar features of the two-dimensional image and the spatial geometric features of the three-dimensional scan to complement each other, reducing the positioning error of a single data source under environmental noise or low contrast interference, and providing an objective positional reference for subsequent size calculations. Then, by extracting the weld cross-section based on the three-dimensional scan data to calculate the actual size data, and inputting the two-dimensional image into the defect recognition model to output the defect prediction result, it is possible to obtain three-dimensional structural information including the weld depth, and at the same time extract local morphological defect features in the two-dimensional image, making up for the incomplete extraction of single-dimensional features, which makes it difficult to identify depth deviations and small defects. Finally, by fusing the center offset, actual size data, and model prediction results, and combining them with preset weld standard parameters, the quality judgment result is output. Compared with the method of relying only on single-dimensional size comparison in related technologies, this application avoids misjudgment caused by insufficient representation ability of a single data source through multi-dimensional judgment of physical spatial morphology evaluation and image classification prediction, thereby improving the detection accuracy of weld defects.

[0011] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0012] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 A flowchart of an optional weld defect detection method provided for an embodiment of this application; Figure 2 A flowchart of another optional weld defect detection method provided for embodiments of this application; Figure 3 A flowchart of another optional weld defect detection method provided for embodiments of this application; Figure 4 A flowchart of another optional weld defect detection method provided for embodiments of this application; Figure 5 A flowchart of another optional weld defect detection method provided for embodiments of this application; Figure 6 A flowchart of another optional weld defect detection method provided for embodiments of this application; Figure 7 A flowchart of another optional weld defect detection method provided for embodiments of this application; Figure 8A flowchart of another optional weld defect detection method provided for embodiments of this application; Figure 9 A flowchart of another optional weld defect detection method provided for embodiments of this application; Figure 10 This is a schematic diagram of the weld defect detection device provided in the embodiments of this application; Figure 11 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0013] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0014] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," "exceeding," etc. are understood to exclude the stated number, while "above," "below," "within," etc. are understood to include the stated number. Where "first" or "second" is mentioned, it is only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the sequential relationship of the indicated technical features.

[0015] In the description of this application, it should be understood that the orientation descriptions, such as up, down, left, right, front, and back, are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0016] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0017] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setting," "installation," and "connection" should be interpreted broadly. Those skilled in the art can reasonably determine the specific meaning of the above terms in this application based on the specific content of the technical solution. Furthermore, the identification of specific steps in the following text does not imply a limitation on the order of steps or execution logic. The execution order and logic between each step should be understood and inferred from the content described in the embodiments.

[0018] In the industrial manufacturing sector, especially in key equipment such as steam generators and heat exchange containers in nuclear power plants, tube-to-tube sheet welded structures are widely used. The quality of these welds directly determines the safety and service life of critical nuclear power equipment. Currently, tube-to-tube sheet welding robots mainly use lasers to scan the weld seams, collect weld appearance and dimensional data, and compare the collected data with preset specified values. If the data exceeds the specified values, the weld is judged as unqualified.

[0019] In related technologies, existing weld defect detection methods mainly rely on single sensor data, such as 2D images or 3D laser scanning that can only measure surface dimensions. These methods have limited feature dimensions and poor anti-interference capabilities. When faced with complex and variable working conditions, they are highly susceptible to interference from high image noise and low contrast, and lack cross-validation of multi-dimensional data. This results in insufficient ability to identify minute and complex defects such as depth deviations, local protrusions, and incomplete fusion on the inner side, leading to low detection accuracy.

[0020] Based on this, this application acquires two-dimensional image data and three-dimensional scan data of the target joint weld area, and calculates the center offset by jointly determining the center of the tube sheet hole and the weld center based on the above two types of data. This allows the planar features of the two-dimensional image and the spatial geometric features of the three-dimensional scan to complement each other, reducing the positioning error of a single data source under environmental noise or low contrast interference, and providing an objective positional reference for subsequent size calculations. Then, by extracting the weld cross-section based on the three-dimensional scan data to calculate the actual size data, and inputting the two-dimensional image into the defect recognition model to output the defect prediction result, it is possible to obtain three-dimensional structural information including the weld depth, and at the same time extract local morphological defect features in the two-dimensional image, making up for the incomplete extraction of single-dimensional features, which makes it difficult to identify depth deviations and small defects. Finally, by fusing the center offset, actual size data, and model prediction results, and combining them with preset weld standard parameters, the quality judgment result is output. Compared with the method of relying only on single-dimensional size comparison in related technologies, this application avoids misjudgment caused by insufficient representation ability of a single data source through multi-dimensional judgment of physical spatial morphology evaluation and image classification prediction, thereby improving the detection accuracy of weld defects.

[0021] Please see Figure 1The present invention provides a method for detecting weld defects, which may include, but is not limited to, the following steps 101 to 105: Step 101: Obtain two-dimensional image data and three-dimensional scan data of the weld area of ​​the target joint.

[0022] Step 102: Based on the two-dimensional image data and the three-dimensional scan data, determine the center position of the tube sheet hole and the center position of the weld, and calculate the center offset based on the center position of the tube sheet hole and the center position of the weld.

[0023] Step 103: Based on the 3D scanning data and the weld center position, extract the weld profile data, and calculate the actual size data of the target joint weld area based on the weld profile data.

[0024] Step 104: Input the two-dimensional image data into the pre-trained defect recognition model for recognition, and obtain the defect prediction result of the target joint weld area.

[0025] Step 105: Based on the center offset, actual size data, and defect prediction results, and combined with the preset weld standard parameters, output the quality judgment result of the weld area of ​​the target joint.

[0026] In step 101 of some embodiments, the target joint weld area refers to the annular or near-annular weld area formed by the tubular structure passing through or abutting the plate-like structure and welding at the joint; the two-dimensional image data refers to a two-dimensional pixel array containing the color, texture, and brightness features of the weld surface; the three-dimensional scan data refers to three-dimensional point cloud or depth data containing the spatial height and geometric features of the weld and surrounding area. Specifically, when performing a weld inspection task, the current weld joint area is acquired online by a vision acquisition module (such as an iCAM device) mounted on the welding robot to obtain the two-dimensional image data and three-dimensional scan data of the area, which serve as the data basis for subsequent multimodal localization and defect analysis.

[0027] In step 102 of some embodiments, the tube sheet hole center position refers to the geometric center coordinates of the hole through which the tubular workpiece passes on the plate-shaped workpiece; the weld center position refers to the physical center coordinates of the actual formed annular weld; and the center offset refers to the spatial deviation distance of the actual weld center relative to the ideal tube sheet hole center. Specifically, by combining the planar features in the two-dimensional image data and the height topography features in the three-dimensional scan data, feature extraction and positioning calculations are performed on the hole structure and weld metal region of the plate-shaped workpiece to obtain the objective tube sheet hole center position and weld center position. Subsequently, the relative distance between the tube sheet hole center position and the weld center position is calculated to obtain the center offset that can characterize the accuracy of the actual welding position.

[0028] Please see Figure 2In some embodiments, step 102 may include, but is not limited to, steps 201 to 204.

[0029] Step 201: Extract edges from the two-dimensional image data to obtain image edge data.

[0030] Step 202: Perform circle detection on the image edge data to obtain candidate circles.

[0031] Step 203: If no candidate circle is detected, the image edge data is used to extract the contour, find the largest circular contour, and take the center of the smallest circumscribed circle of the largest circular contour as the center position of the tube plate hole.

[0032] Step 204: If a candidate circle is detected, the center of the candidate circle is taken as the center position of the tube sheet hole.

[0033] In step 201 of some embodiments, edge extraction refers to obtaining regions of abrupt grayscale changes in the image through difference operations or gradient operations; image edge data refers to a set of binary geometric lines representing the boundary contours of various structures within the image. Specifically, to eliminate interference caused by uneven lighting in industrial settings, the acquired two-dimensional image data is first converted from a three-channel color space to a single-channel grayscale space, and a Gaussian kernel of a specific size (e.g., ...) is used. A Gaussian filter kernel is used to spatially smooth the image to suppress background noise. Subsequently, gradient calculations are performed on the image to be processed based on the Canny edge detection algorithm, and initial structural edges are extracted by setting preset high and low threshold parameters (e.g., low threshold 50, high threshold 150). Considering that scratches or reflections on actual metal surfaces may cause edge breakage, preset morphological structural elements (e.g., Gaussian filter kernel) are further employed. The elliptical structural element performs a morphological closing operation on the initial edge, connecting the broken boundary points to obtain continuous and complete image edge data.

[0034] In step 202 of some embodiments, circle detection refers to mapping the image space to a parameter space to identify circular geometric features; candidate circles refer to the set of circular coordinates that initially meet the expected aperture size, which are locked through spatial voting accumulation. Specifically, for the image edge data output from the previous step, the Hough circle transform algorithm is used to perform circle detection. When performing the Hough circle transform, the search radius range is dynamically constrained by combining the actual tube sheet hole diameter in the preset weld standard parameters; for example, the minimum radius threshold is constrained to 0.8 times the nominal tube sheet hole diameter, and the maximum radius threshold is constrained to 1.2 times the nominal tube sheet hole diameter. Using the above-set radius range, the parameter space is traversed in the image edge data. If a circular geometric structure that meets the curvature and gradient requirements is found, the corresponding center coordinates and radius parameter matrix are extracted and determined as candidate circles.

[0035] In step 203 of some embodiments, contour extraction refers to extracting the external closed boundary of the image through topological structure analysis; the maximum circular contour refers to the set of contours with the largest enclosing area among all closed boundaries; the minimum circumcircle center refers to the coordinates of the center of the circle with the smallest area that can completely enclose the above contours. Specifically, under complex nuclear power plant operating conditions, two-dimensional image data may have extremely high noise or extremely low contrast, resulting in an insignificant gradient accumulation peak of the Hough circle transform, thus causing a failure situation where no candidate circles are detected. Under this triggering condition, an external contour retrieval that extracts only the outermost boundary is performed on the image edge data to obtain multiple separate closed contours; subsequently, by calculating and traversing and comparing the pixel area of ​​each closed contour, the contour occupying the largest area is selected as the maximum circular contour. For this maximum circular contour, the minimum circumcircle algorithm is used to iteratively calculate the minimum circle that can completely enclose the boundary points of the contour, obtain the geometric center pixel coordinates of the circumcircle, and directly determine the final tube sheet hole center position.

[0036] In step 204 of some embodiments, the center of the candidate circle refers to the center of the circular structure with the highest matching degree obtained by the circle detection algorithm in the parameter space. Specifically, if the candidate circle array is successfully matched and output during the Hough circle transform in step 202, it indicates that the edge features of the current image are significant and the accurate position can be obtained directly by the circle detection algorithm. At this time, the parameter data in the candidate circle array is rounded to the nearest integer, and the two-dimensional geometric center coordinates of the first candidate circle with the highest confidence are extracted, and these coordinates are determined as the center position of the tube sheet hole.

[0037] Through steps 201 to 204 above, this embodiment of the application performs different localization calculation processes depending on whether candidate circles are output when performing circle detection on the edge features of two-dimensional image data. Specifically, under normal image conditions, the center of the detected candidate circle is directly used as the center position of the tube sheet hole; however, when the image has high noise or low contrast, making it impossible to directly detect candidate circles, the localization is completed by further extracting the maximum circular contour of the edge features and calculating the minimum circumscribed circle center. This processing combines circle detection in parameter space with contour circumscribed circle calculation in image space, making up for the deficiency of the single Hough circle transform being prone to failure under extreme image interference. This allows the center coordinates to be effectively obtained under different image quality conditions, avoiding localization interruption due to the failure of a single algorithm, thereby improving the success rate and accuracy of tube sheet hole center localization.

[0038] Please see Figure 3 In some embodiments, step 102 may include, but is not limited to, steps 301 to 304.

[0039] Step 301: Perform color space conversion and threshold segmentation on the two-dimensional image data to generate a two-dimensional feature mask.

[0040] Step 302: Extract height information from the 3D scan data and generate a 3D height mask based on the height information.

[0041] Step 303: Fuse the two-dimensional feature mask with the three-dimensional height mask to obtain the fused mask.

[0042] Step 304: Extract the contour of the fusion mask and calculate the center position of the weld based on the contour moments.

[0043] In step 301 of some embodiments, color space conversion refers to converting the image from the default color model of the acquisition device to a representation space that is easier to extract specific target color features, such as the HSV space; thresholding segmentation refers to a binarization process of the image based on the pixel value range; and a two-dimensional feature mask refers to a binary image representing a specific target weld area in a two-dimensional image. Specifically, in order to effectively separate luminance and chromaticity information to highlight weld areas that typically present a bright metallic color, the acquired two-dimensional image data is first converted from the original color space to the HSV color space. Subsequently, based on the color distribution characteristics of the actual metal weld in the HSV space, corresponding upper and lower limit parameters for color thresholds are set. For example, specific luminance channel and saturation channel ranges are set based on the reflective properties of metal. The HSV image is then thresholded using the set color thresholds, retaining pixels within the threshold range as target values ​​and setting pixels outside the range as background values, thereby extracting a two-dimensional feature mask containing the surface features of the metal weld area.

[0044] In step 302 of some embodiments, height information refers to the numerical values ​​in the three-dimensional scanning data that reflect the undulations of the target workpiece surface relative to the reference datum plane; the three-dimensional height mask refers to a binary image characterizing the target area in the three-dimensional scanning data where the height changes significantly. Specifically, since the weld formed by actual welding usually protrudes or retracts from the surrounding flat tube sheet reference surface, the height values ​​of each spatial point are extracted from the three-dimensional scanning data along the depth channel to generate a height map mapped to a two-dimensional plane. Subsequently, the average height and standard deviation of the pixels in the height map are calculated, and the combination of the average height and a specific multiple of the standard deviation is used as a judgment threshold to binarize the height map, retaining the protruding or retracted areas whose height changes meet the threshold conditions, thereby generating a three-dimensional height mask that highlights the physical geometric features.

[0045] In step 303 of some embodiments, the fusion mask refers to a comprehensive binary feature map that integrates two-dimensional surface color features and three-dimensional spatial height features to accurately define the actual weld boundary. Specifically, for the two-dimensional feature mask and the three-dimensional height mask generated in the above steps, a logical AND operation is performed on the corresponding pixel positions. Through the logical AND operation, only overlapping pixel regions that simultaneously possess metallic reflective features (i.e., are the target values ​​in the two-dimensional feature mask) and have physical height abrupt changes (i.e., are the target values ​​in the three-dimensional height mask) are retained, thereby eliminating interference items that only satisfy a single feature, such as bright reflective spots on a plane or pits in non-metallic areas. Further, a specific morphological kernel (such as...) is used. A set number of morphological closing and opening operations are performed on the mask image after the logical AND operation (all-one matrix) to eliminate scattered noise at the edges after fusion and smooth the boundaries, finally obtaining a fused mask that can accurately represent the actual physical range of the weld.

[0046] In step 304 of some embodiments, the contour moments are mathematical statistics used to describe the spatial distribution of the geometric shape of the target region by summing the coordinates of pixels within the image contour. Specifically, for the fused mask obtained after processing, a connected component search is performed on the outer boundary of its target region to extract the continuous contour lines constituting the outer boundary of the weld. Subsequently, the spatial moments of each order of the closed contour are extracted; specifically, the zeroth moment ( The total pixel quality or area of ​​the target region) and the first moment ( and These represent the sum of the distributions of pixel coordinates in the target region along the x-axis and y-axis in a Cartesian coordinate system, respectively. The ratio of the first moment to the zeroth moment is calculated (i.e.,...). , The geometric centroid pixel coordinates of the target contour are obtained, and these geometric centroid coordinates are directly determined as the final weld center position.

[0047] Through steps 301 to 304 above, this embodiment first uses color space conversion and threshold segmentation to obtain a two-dimensional feature mask representing planar color, and then generates a three-dimensional height mask based on the height abrupt change features of the three-dimensional scanning data. Finally, the two-dimensional masks are cross-fused using a logical AND operation. This operation requires that the final determined region simultaneously satisfy a specific color distribution on the surface and spatial height fluctuations, filtering out two-dimensional artifacts caused by illumination reflection and three-dimensional structural abrupt changes in non-weld areas. Finally, based on this cross-fused denoised mask, contour moments are used to calculate the center coordinates, overcoming the shortcomings of related technologies that rely solely on two-dimensional image color or solely on three-dimensional scanning height data, which are easily affected by environmental interference. This effectively reduces the positioning deviation caused by a single data source and improves the accuracy of weld center positioning.

[0048] Please see Figure 4 In some embodiments, step 102 may include, but is not limited to, steps 401 to 403.

[0049] Step 401: Calculate the pixel distance between the center position of the tube sheet hole and the center position of the weld.

[0050] Step 402: Obtain the camera intrinsic parameter data of the vision system corresponding to the acquired two-dimensional image data.

[0051] Step 403: Based on the camera intrinsic data, convert the pixel distance into the actual physical offset in physical space, and determine the actual physical offset as the center offset.

[0052] In step 401 of some embodiments, pixel distance refers to the linear relative distance between two pixel coordinate points in a two-dimensional image plane Cartesian coordinate system. Specifically, after obtaining the coordinates of the tube sheet hole center positions independently calculated in the previous steps... and the coordinates of the weld center position Then, in the two-dimensional image coordinate system, the square root of the sum of the squares of the differences between the horizontal and vertical coordinates is calculated using the Euclidean distance calculation rule. The pixel distance, which characterizes the degree of deviation between the center of the tube sheet hole and the center of the weld in the digital image space, is obtained.

[0053] In step 402 of some embodiments, the vision system refers to a hardware integrated module mounted on a welding robot for acquiring multimodal data of the area to be tested; camera intrinsic parameter data refers to a set of parameters characterizing the internal optical geometry and imaging projection relationship of the camera. Specifically, before performing the image-to-physical-size conversion, the system extracts the on-site calibration parameters directly associated with the currently captured two-dimensional image data by calling the application programming interface related to the vision system. These parameters specifically include the focal length of the camera lens in the horizontal and vertical directions of the two-dimensional image (…). ), Optical principal point coordinates ( ) and parameters such as the possible lens distortion coefficient.

[0054] In step 403 of some embodiments, the actual physical offset refers to the absolute displacement between two center points in the real three-dimensional physical world. Specifically, since the pixel distance calculated in step 401 can only represent the relative difference in the image plane, in order to obtain objective physical indicators, the aforementioned camera intrinsic parameter data is introduced. Based on the camera pinhole imaging model and perspective projection principle, the pixel deviation span in the two-dimensional space of the image is mapped and transformed into the actual offset in the physical three-dimensional space. Subsequently, this physical-level actual physical offset is determined as the final center offset of the target joint weld area and used as the numerical basis for subsequent position compliance checks.

[0055] Through steps 401 to 403 described above, this embodiment first obtains the pixel distance between the tube sheet hole and the weld center at the two-dimensional image level. Then, it introduces camera intrinsic parameter data specific to the vision system as a mapping benchmark, converting the relative deviation at the digital image level into an absolute displacement in real physical space. This calculation process eliminates the problem of being easily affected by changes in image resolution and shooting distance when relying solely on pixel evaluation. It constructs a quantification rule from two-dimensional pixel features to three-dimensional physical parameters, thereby providing an objective and unified actual size index for evaluating weld position accuracy.

[0056] In step 103 of some embodiments, weld profile data refers to the height contour information that reflects the undulations of the weld surface, formed by cutting three-dimensional scan data along a specific direction; actual size data refers to the three-dimensional physical size index of the weld obtained by geometrically quantizing the above-mentioned height contour information. Specifically, after determining the center position of the weld, the cross-sectional contour data spanning the weld region is extracted from the three-dimensional scan data based on the center position to obtain the weld profile data. Using the height change and span information of the weld region relative to the surrounding reference plane in the weld profile data, the actual size data of the target joint weld region in physical space, such as width and depth, is calculated, thereby obtaining objective weld morphology parameters.

[0057] Please see Figure 5 In some embodiments, step 103 may include, but is not limited to, steps 501 to 504.

[0058] Step 501: Calculate the directional angle from the center position of the tube sheet hole to the center position of the weld.

[0059] Step 502: Perform smoothing filtering on the 3D scanning data to obtain smoothed 3D scanning data.

[0060] Step 503: Based on the weld center position and direction angle, extract the first three-dimensional profile perpendicular to the direction angle and the second three-dimensional profile parallel to the direction angle from the smooth three-dimensional scanning data.

[0061] Step 504: Combine the first three-dimensional profile and the second three-dimensional profile to determine the weld profile data.

[0062] In step 501 of some embodiments, the direction angle refers to the angle formed between a ray pointing from the center of the tube sheet hole to the center of the weld and a reference coordinate axis in a two-dimensional plane coordinate system. Specifically, the coordinates of the tube sheet hole center calculated in the previous steps are obtained. ) and weld center coordinates ( The ratio of the difference in coordinates along the vertical axis to the difference in coordinates along the horizontal axis is calculated using trigonometric functions (such as the arctangent function arctan2). This yields the directional angle from the hole center to the actual weld center. This angle reflects the spatial tangential orientation of the weld at that local location, providing a directional reference for subsequent three-dimensional profile extraction.

[0063] In step 502 of some embodiments, smoothing filtering refers to signal processing that eliminates high-frequency random noise generated during data acquisition while preserving macroscopic geometric shapes; smoothing 3D scan data refers to a 3D depth matrix with a more continuous height distribution after filtering and denoising. Specifically, since the original 3D scan data usually contains spatial discrete noise introduced during laser sensor acquisition or depth jumps caused by reflections from metal surfaces, directly extracting the profile can easily lead to errors in subsequent dimension calculations. Therefore, a preset filtering algorithm is used to perform spatial domain smoothing filtering on the original 3D scan data. Through this processing operation, the interference from small abrupt changes caused by surface roughness is weakened, thereby outputting smooth 3D scan data with a smooth and continuous height distribution.

[0064] In step 503 of some embodiments, the first three-dimensional profile refers to the height profile information intercepted along a direction perpendicular to the weld direction; the second three-dimensional profile refers to the height profile information intercepted along the weld direction. Specifically, the determined weld center position is used as the spatial sampling origin, and the tangential direction of the current weld is determined based on the direction angle calculated in step 501. In the smoothed three-dimensional scan data, this direction angle is added with a specific deflection angle (such as an additional...). A cross-sectional sampling path perpendicular to the weld direction is generated (radian). The height value is extracted point by point along the path according to a preset length to form a first three-dimensional profile for quantifying the weld width. Similarly, a longitudinal sampling path parallel to the weld direction is generated with the angle of this direction as the reference path direction. The height value is extracted along the parallel path to form a second three-dimensional profile for quantifying the weld depth.

[0065] In step 504 of some embodiments, the weld profile data refers to a dataset containing orthogonal multi-angle cross-sectional height information of the target joint weld region. Specifically, the first three-dimensional profile reflects the span of the weld's bulges or depressions in the radial dimension, and the second three-dimensional profile reflects the height fluctuation of the weld relative to the base material in the circumferential dimension. The extracted first three-dimensional profile and the second three-dimensional profile are merged to form weld profile data for comprehensively quantifying the actual three-dimensional physical morphology, thereby providing complete data for subsequent steps to calculate multi-dimensional geometric and physical dimensions.

[0066] Through steps 501 to 504 above, this embodiment of the application introduces a direction angle calculated based on the tube sheet hole and weld center, and Gaussian smoothing filtering for the three-dimensional depth data before performing three-dimensional data analysis. Based on adaptive direction angle directional sampling, it ensures that the profile extraction path is accurately perpendicular or parallel to the actual annular weld direction, avoiding the contour section distortion caused by blindly fixing the direction sampling in related technologies. Simultaneously, the smoothing process eliminates high-frequency noise interference on the surface. Based on this, mutually orthogonal first and second three-dimensional profiles are extracted, constructing an objective data model for obtaining the true three-dimensional morphological features of the weld, thereby improving the accuracy of subsequent three-dimensional dimension quantification calculations.

[0067] Please see Figure 6 In some embodiments, the actual size data includes the actual width and actual depth, and step 103 may include, but is not limited to, steps 601 to 602.

[0068] Step 601: Extract the peak region span in the first three-dimensional profile and convert the peak region span into the actual width.

[0069] Step 602: Calculate the height difference between the average height of the weld area and the average height of the surrounding area in the second three-dimensional profile, and convert the height difference into the actual depth.

[0070] In step 601 of some embodiments, the peak region span refers to the distance between the starting and ending points of the convex portion (i.e., the actual weld metal area) that is significantly higher than the reference surface on the height profile curve in the horizontal direction; the actual width refers to the lateral physical dimension of the weld in real physical space. Specifically, for the extracted first three-dimensional profile perpendicular to the weld direction, the average height and standard deviation of the profile data are calculated using statistical methods, and the sum of the average height and the standard deviation of a preset multiple (e.g., 0.5 times) is set as the height threshold. In the first three-dimensional profile, continuous sampling points with height values ​​greater than the height threshold are determined to belong to the peak region of the weld. Subsequently, the pixel span between the maximum and minimum coordinate points within the peak region is extracted, and based on the system's preset pixel equivalent conversion ratio, for example, 10 pixels corresponding to 1 millimeter, the pixel-level peak region span is converted into an actual width in millimeters.

[0071] In step 602 of some embodiments, the average height refers to the average height values ​​of all sampling points within a specific area; the height difference refers to the relative vertical distance between the protruding or recessed surface of the weld and the surrounding flat base material surface; and the actual depth refers to the physical dimension of the weld's depth in three-dimensional physical space. Specifically, for the extracted second three-dimensional profile parallel to the weld direction, the same height threshold calculation method as in step 601 is used to divide all sampling points in the second three-dimensional profile into a weld area and surrounding non-weld areas. The average height of the sampling points within the weld area and the average height of the sampling points within the surrounding area are calculated respectively, and the absolute difference between the two is obtained to obtain the height difference. Finally, combined with the sensor's scaling factor in the depth direction, this height difference is converted into the actual depth.

[0072] Through steps 601 to 602 described above, this embodiment of the application introduces a dynamic height threshold segmentation mechanism based on the average value and standard deviation of the data itself when processing three-dimensional profile data. This mechanism adaptively separates the weld area from the surrounding base material area from the height profile. Based on the precisely segmented area, the span is calculated along the vertical direction to obtain the width, and the relative height difference is calculated along the parallel direction to obtain the depth. This objectively quantifies the relative height profile into the actual size in three-dimensional physical space, overcoming the absolute height measurement error caused by uneven base material surface or drift of the ranging reference in related technologies. It also solves the limitation of traditional laser scanning, which can only accurately measure the lateral dimensions of the surface but cannot accurately obtain the depth, thereby improving the comprehensiveness and accuracy of weld physical size measurement.

[0073] In step 104 of some embodiments, the defect identification model refers to a machine learning or deep network model that has been pre-trained and converged using a historical image sample set containing known weld defect types; the defect prediction result refers to the defect category prediction state output by the model for the currently input two-dimensional image data. Specifically, the collected two-dimensional image data is input into the deployed defect identification model, and the network layers within the model automatically extract features from the image and perform classification inference, outputting the defect prediction result for the weld area of ​​the target joint, providing a basis for judgment in subsequent comprehensive quality assessment.

[0074] In some embodiments, the defect prediction result includes the defect type and the corresponding defect confidence level. Step 104 may include, but is not limited to, the following steps: The two-dimensional image data is resized and normalized to obtain a standard input image; The standard input image is fed into the defect recognition model for feature extraction and classification, and the probability distribution of various candidate defects is output. The candidate defect with the highest probability is identified as the defect type, and the corresponding probability value is identified as the confidence level.

[0075] Here, resizing refers to the image processing operation of scaling the resolution of the acquired image to the fixed dimensions required by the model; normalization refers to the mathematical transformation operation of mapping image pixel values ​​to a specific numerical range proportionally; and the standard input image refers to the data matrix that meets the input tensor format requirements of the deep learning model after the above preprocessing. Specifically, after acquiring the original two-dimensional image data, the image resizing function is called to uniformly scale its spatial resolution to a preset image size, for example, 256×256 pixels. Subsequently, the resized image undergoes color space conversion, for example, from BGR format to RGB format, and the value of each pixel channel is divided by a preset normalization coefficient (e.g., 255.0), so that all pixel values ​​are distributed within a uniform range of 0 to 1, thereby obtaining a standard input image that eliminates differences in image scale and numerical magnitude.

[0076] Feature extraction refers to the process of obtaining deep semantic and texture features of an image through convolution operations; probability distribution refers to the set of quantified numerical values ​​representing the likelihood of the input data belonging to various preset categories. Specifically, the standard input image obtained in the above steps is constructed into an input tensor that meets the dimensionality requirements and input into a pre-deployed defect recognition model. Multiple convolutional and pooling layers within this model perform layer-by-layer local feature extraction and dimensionality reduction operations on the input image, followed by non-linear mapping through fully connected layers. Finally, softmax and other activation function layers are used for classification calculations, thereby outputting probability distribution vectors for various preset candidate defect categories, including but not limited to nine categories such as qualified, unwelded, too wide, local protrusion, and internal non-fusion.

[0077] Here, defect type refers to the category label representing the specific quality state of the weld in the final output of the model; confidence level refers to the objective quantitative indicator of the model's certainty regarding the above classification results. Specifically, after obtaining the probability distribution of various candidate defects output by the defect recognition model, all probability values ​​in the probability distribution vector are compared and optimized to find the maximum probability value. The candidate defect corresponding to the maximum probability value is directly extracted and determined as the final defect type of the target joint weld area; at the same time, the found maximum probability value is converted to floating-point format and determined as the confidence level representing the reliability of this defect classification inference.

[0078] Please see Figure 7 In some embodiments, steps 701 to 703 may be included before step 104.

[0079] Step 701: Obtain a set of historical two-dimensional image samples carrying defect type labels; Step 702: Perform data augmentation processing on the historical two-dimensional image sample set to obtain the training sample set; Step 703: Construct an initial convolutional neural network, using images from the training sample set as input and the corresponding defect type label as the expected output. Iterate the initial convolutional neural network until the convergence condition is met to obtain a pre-trained defect recognition model.

[0080] In step 701 of some embodiments, the defect type label refers to the classification result of manual or prior annotation of the quality state of the weld; the historical two-dimensional image sample set refers to a large collection of weld two-dimensional image data that has been collected and annotated in advance. Specifically, during the first use or initial training phase, the pre-annotated weld defect dataset is read from a preset specified directory. The images in this historical two-dimensional image sample set are divided according to specific category labels, and the defect type labels specifically include nine common weld state samples: qualified, unwelded, half-welded, too wide, full-circumference defect, local protrusion, local depression, rusted carbon steel, and inner lack of fusion.

[0081] In step 702 of some embodiments, data augmentation refers to data processing that expands the number and diversity of samples by performing spatial geometric transformations or pixel-level perturbations on the original image; the training sample set refers to the dataset that, after augmentation, is actually used to input the neural network for parameter optimization. Specifically, to avoid overfitting of the model due to the limited number or singular features of the original historical two-dimensional image sample set, spatial geometric operations such as rotation (e.g., random rotation within a 30-degree range), translation (e.g., width and height translation by a ratio of 0.2), scaling (e.g., random scaling by a ratio of 0.2), shearing, and horizontal or vertical flipping are performed on the acquired sample images. Through the above data augmentation processing, different shooting angles and workpiece poses that may occur in industrial settings are simulated, thereby generating a training sample set with richer features.

[0082] In step 703 of some embodiments, the initial convolutional neural network refers to a deep learning network architecture that has not yet undergone parameter optimization and whose internal weights are in a state of random initialization; the convergence condition refers to the set criteria for stopping training, such as reaching a set number of training epochs or the loss function value stabilizing. Specifically, an initial convolutional neural network containing multi-layer feature extraction and classification structures is constructed; this network architecture is specifically configured with 4 convolutional layers (containing 32, 64, 128, and 256 convolutional kernels respectively) and 2 fully connected layers (containing 512 and 256 neurons respectively), combined with a Dropout layer (e.g., a dropout rate set to 0.5) to prevent network overfitting. During training, resized images (e.g., 256×256×3 three-channel images) from the training sample set are used as input data, and their 9 defect type labels are used as the expected output. A specific optimizer (e.g., the Adam optimizer with a learning rate of 0.001) and loss function are used, the batch size and total number of training epochs are set, and the network parameters are iteratively updated through forward and backward propagation. When the preset number of training rounds is reached or the accuracy on the validation set no longer improves and the convergence condition is met, training is stopped and the network weight parameters are saved, thus obtaining the pre-trained defect recognition model.

[0083] Through steps 701 to 703 described above, this embodiment of the application introduces historical data containing nine types of defect labels before performing online detection, and expands the sample diversity using data augmentation operations such as spatial geometric transformations. Based on this diverse sample, a deep learning network containing a specific convolutional layer and fully connected layer architecture is constructed and trained, effectively preventing model overfitting and expanding the coverage of defect types. This pre-training process builds a foundational model with strong generalization capabilities, reducing the model's generalization cost for new working conditions and new defect types, thereby providing technical support for obtaining high-accuracy classification and recognition results during subsequent online detection.

[0084] In step 105 of some embodiments, the preset weld standard parameters refer to allowable reference thresholds pre-configured according to specific production process requirements, including nominal dimensions and allowable tolerance ranges; the quality judgment result refers to the qualitative conclusion of whether the target joint weld area meets the process requirements after comprehensively considering multiple evaluation indicators. Specifically, the calculated center offset and the measured actual size data are compared with the preset weld standard parameters to confirm whether the physical position and size are out of tolerance; at the same time, combined with the defect prediction results, it is determined whether there are defect types that affect quality. When the positional accuracy, dimensional tolerance, and defect status all meet the requirements of the preset standards, a qualified quality judgment result is output; otherwise, an unqualified quality judgment result is output.

[0085] Please see Figure 8In some embodiments, the preset weld standard parameters include center offset tolerance, nominal width tolerance and nominal depth tolerance. Step 105 may include, but is not limited to, steps 801 to 804.

[0086] Step 801: Determine whether the center offset is within the center offset tolerance to obtain the first judgment result.

[0087] Step 802: Determine whether the actual width is within the nominal width tolerance and whether the actual depth is within the nominal depth tolerance to obtain the second determination result.

[0088] Step 803: Based on the defect type and defect confidence level, determine whether there is a serious defect, and obtain the third judgment result.

[0089] Step 804: When the first judgment result, the second judgment result, and the third judgment result all meet the qualification conditions, output the quality judgment result that the weld area of ​​the target joint is qualified.

[0090] In step 801 of some embodiments, the center offset tolerance refers to a pre-configured maximum physical deviation (e.g., preset to 0.5 mm) between the center of the tube sheet hole and the actual weld center; the first judgment result refers to a judgment conclusion specifically given for the compliance of weld position accuracy. Specifically, the actual center offset calculated in the previous steps is obtained and compared with a preset center offset tolerance threshold. If the actual calculated center offset is less than or equal to the center offset tolerance, it is determined that the weld position has not been seriously offset, and a first judgment result indicating position compliance is generated; otherwise, if it exceeds the tolerance range, a first judgment result indicating position non-compliance is generated.

[0091] In step 802 of some embodiments, the nominal width tolerance and nominal depth tolerance refer to the legal value range calculated based on the nominal size value required by the production process and the allowable fluctuation ratio; the second judgment result refers to the judgment conclusion given on whether the three-dimensional physical morphology of the weld exceeds the standard. Specifically, standard value parameters (such as nominal width 5.2 mm, nominal depth 1.8 mm) are extracted according to the preset specific process requirements, and the allowable upper and lower limits of width and upper and lower limits of depth are calculated respectively in combination with the preset tolerance ratio (such as width tolerance ratio 0.2, depth tolerance ratio 0.2). Subsequently, the actual width and actual depth obtained from the previous measurement are substituted into the corresponding legal ranges calculated above for judgment. Only when the actual width is within the nominal width tolerance range and the actual depth is also within the nominal depth tolerance range, a second judgment result indicating overall dimensional compliance is generated; otherwise, a second judgment result indicating dimensional non-compliance is generated.

[0092] In step 803 of some embodiments, a serious defect refers to a welding quality defect that is sufficient to affect the structural strength or sealing of the target joint weld and cannot be approved by the process standard; the third judgment result refers to the judgment conclusion given on whether the appearance of the weld surface meets the quality and safety standards. Specifically, the defect type and the corresponding defect confidence level output by the model forward inference are extracted and subjected to joint rule matching: it is determined whether the defect type is in a "qualified" state, or whether the defect type belongs to a preset minor morphological defect (such as "local protrusion" or "local depression") and whether its corresponding defect confidence level is lower than a preset confidence threshold (such as 0.7). If any of the above conditions are met, that is, the model determines that it is qualified or the model determines that it is a local defect but the confidence level is low, then it is determined that there is no serious defect in the current weld and a third judgment result characterizing the compliance of the defect inspection is generated; if it belongs to other clear defect states or the confidence level of minor defects is too high, then it is determined that there is a serious defect and a non-compliant judgment result is generated.

[0093] Please see Figure 9 In some embodiments, step 803 may include, but is not limited to, steps 901 to 903.

[0094] Step 901: If the defect type is normal, then generate a third judgment result characterizing the weld quality.

[0095] Step 902: If the defect type is a local morphological defect and the defect confidence level is lower than the preset confidence threshold, then it is determined that there is no serious defect, and a third judgment result characterizing the weld seam is generated.

[0096] Step 903: If the defect type is other defect status, then a serious defect is determined to exist, and an unqualified third judgment result is generated.

[0097] In step 901 of some embodiments, the normal state refers to the "qualified" label among a pre-defined set of various defect classification labels, which represents no visual abnormalities or complete compliance with welding specifications. Specifically, after obtaining the defect prediction result output by the defect recognition model, a conditional comparison is performed on the defect type string contained therein. If the comparison finds that the defect type of the current target joint weld area directly matches the "qualified" state, it indicates that the model considers the surface quality of the area to be good and no known defect features have been detected. In this case, the system directly determines that the current weld does not have serious defects and generates a third judgment result representing that the visual defect inspection dimension has been successfully passed.

[0098] In step 902 of some embodiments, a local morphological defect refers to a minor surface variation that is non-structural and has a limited impact on the overall sealing strength of the workpiece, such as a "local protrusion" or "local depression" in a preset classification category. The preset confidence threshold refers to a reference probability boundary value set by the system to filter samples with ambiguous classification boundaries, for example, a default setting of 0.7. Specifically, considering that extremely slight surface unevenness is permissible in actual industrial welding manufacturing, if the defect type output by the defect identification model belongs to the aforementioned local morphological defects, but the accompanying defect confidence is below 0.7, it indicates that the model's certainty in classifying it as a defect is low, meaning the defect feature is extremely weak. Under this tolerance mechanism, the weak anomaly is determined not to constitute a serious defect affecting engineering quality, thus allowing it to proceed, and a third judgment result characterizing the weld's qualification is generated.

[0099] In step 903 of some embodiments, "other defect states" refers to all abnormal classification states other than the aforementioned explicit release conditions, including explicit defect categories that directly affect structural safety, such as "not welded," "partially welded," "too wide," "full circumference defect," "rusted carbon steel," and "inner side not fused," or definite defect samples belonging to "local protrusion" or "local depression" but whose corresponding confidence level is greater than or equal to a preset confidence threshold. Specifically, if the prediction result of the current weld falls into any of the above-mentioned other defect states, it indicates that there is a significant welding abnormality and the model's judgment confidence level is extremely high. At this time, the workpiece is judged to have a serious defect, and a third judgment result characterizing the defect inspection as unqualified is generated, blocking subsequent processes.

[0100] In step 804 of some embodiments, a logical AND operation is performed on the first judgment result, the second judgment result, and the third judgment result generated in the above steps. The overall quality of the current weld is finally determined to be up to standard, and a qualified quality judgment result for the weld area of ​​the target joint is output, only when the judgment results of the three dimensions of position compliance, size compliance, and defect inspection compliance are all simultaneously satisfied. If the judgment result of any of the above dimensions is not satisfied, an unqualified quality judgment result is output.

[0101] In some embodiments, after step 105, the following steps may also be included, but are not limited to: The center position of the tube sheet hole, the center position of the weld, the actual size data, the defect prediction results and the quality judgment results are superimposed on the two-dimensional image data for visualization. After obtaining the final inspection results, a copy of the original two-dimensional image data is used as a graphics rendering canvas. On this canvas, specific geometric shapes and colors are used; for example, green and red dots are used to mark the coordinates of the obtained tube sheet hole center and weld center, respectively, and an offset line is drawn between the two points to visually display the offset status. Simultaneously, in a preset area of ​​the image interface, the actual measured width and depth values, as well as the currently predicted defect type, corresponding confidence index, and final pass / fail quality judgment result are overlaid as a text layer. By simultaneously displaying these multi-dimensional inspection indicators on a unified image, on-site operators can intuitively obtain comprehensive quality status information of the workpiece.

[0102] Generate an inspection report containing quality assessment results, and associate and store the two-dimensional image data, three-dimensional scan data, and inspection report in the inspection database.

[0103] The inspection database refers to a structured data storage system used for persistently archiving historical inspection records of the equipment. Specifically, the current processing timestamp, processing time, center coordinate parameters, dimensional data, model prediction data, and quality judgment results are packaged to generate a structured inspection report. Subsequently, a corresponding index identifier is assigned to the current workpiece, and a mapping relationship is established between the collected underlying 2D image data and 3D scan data and the aforementioned structured inspection report. This set of associated data is then synchronously and persistently written into the local or remote inspection database, completing the data archiving of this inspection batch.

[0104] Through the aforementioned visualization and data association storage, this application embodiment, on the one hand, utilizes the intuitive image overlay of multi-dimensional detection indicators to improve the problem that traditional detection equipment only outputs a single qualified signal and lacks process basis, thereby improving the efficiency of on-site personnel's anomaly review and judgment; on the other hand, by associating and storing the original multimodal sensing data with the structured judgment report, a complete and traceable electronic archive of weld quality is constructed, which not only meets the quality traceability requirements in industrial manufacturing scenarios, but also retains historical data with high-value verification labels, thereby providing real working condition data for subsequent process analysis and continuous iterative optimization of defect identification models.

[0105] In some embodiments, after the association is stored in the detection database, the following steps may also be included, but are not limited to: When newly stored data in the detection database meets preset conditions, historical image data and corresponding historical verification tags are extracted from the detection database. The preset conditions refer to the set data accumulation threshold or time period that triggers the model's adaptive optimization and update, such as the accumulation of a specific number of new detection records in the database. Historical verification labels refer to the true defect classification results confirmed by manual review or a high-quality screening mechanism for the saved historical image data. Specifically, as the online detection task continues to execute, new operational detection data is continuously stored in the detection database. By periodically or in real-time monitoring the database's capacity changes, when the amount of newly stored data reaches the preset conditions, a targeted retrieval of historical records in the detection database is performed to extract the original historical two-dimensional image data to be used for optimization. Simultaneously, historical verification labels that establish a mapping relationship with these labels, representing the true defect classification status, are extracted as sample inputs for subsequent model iterations.

[0106] Historical image data and historical verification labels are added to the preset model training set. The defect recognition model is then retrained using the updated model training set to update the defect recognition model.

[0107] The model training set refers to a batch of historical samples used to provide the basis for forward feature inference and backward gradient propagation to the neural network. Retraining refers to the parameter optimization and update process using new data based on the existing model weight parameters or on a re-initialized basis. Specifically, the extracted historical image data and historical verification labels are uniformly formatted and added to the system's preset basic model training set, thereby constructing an updated model training set that incorporates the latest on-site working condition features. Subsequently, the current defect recognition model architecture is invoked, using the image matrix in the updated model training set as the network input and the historical verification labels as the desired classification output. The corresponding training parameters are set, and the network weight parameters of the model are retrained and fine-tuned. After the training meets the convergence requirements, the optimized model weight parameters are saved and the original old model file is replaced, thus completing the online upgrade and update deployment of the defect recognition model.

[0108] Through the aforementioned historical data extraction and model retraining operations, this embodiment utilizes a detection database to construct a continuous closed-loop iteration of defect identification data. This fully leverages the real-world working condition data continuously accumulated on actual production lines, supplementing the training network with defect samples possessing new features and morphologies. This approach overcomes the shortcomings of related technologies where static models experience significant accuracy degradation when faced with new materials, welding processes, or changes in ambient lighting. It effectively reduces the system's generalization cost, thereby maintaining and improving the model's feature adaptation capabilities and recognition accuracy in complex and variable industrial scenarios over the long term.

[0109] Through steps 101 to 105 above, this embodiment of the application utilizes multimodal data foundations of two-dimensional image data and three-dimensional scanning data to improve the accuracy of tube sheet hole and weld center positioning; at the same time, it combines three-dimensional dimensional measurement and pre-trained defect recognition model to perform multi-dimensional objective judgment covering physical spatial morphology indicators and image classification prediction, reducing the insufficient representation capability of a single data source and subjective judgment errors caused by human intervention in related technologies, thereby objectively and effectively improving the detection accuracy of weld defects.

[0110] In some embodiments, to ensure the effective operation of the above-described weld defect detection method and the real-time performance of model inference, the electronic device or computing unit on which this application embodiment relies needs to be configured with a specific hardware and software environment. Specifically, in terms of hardware environment configuration, the above method can be deployed in a computing device associated with a welding robot equipped with a vision acquisition module; to meet the massive matrix inference computation requirements of the deep network model, the central processing unit (CPU) of the computing device can be an Intel Core i7 or higher, and the graphics processing unit (GPU) can be an NVIDIA RTX 2060 or higher that supports parallel computing acceleration (such as CUDA acceleration); at the same time, the system's running memory is configured to be 16GB or higher, and it is equipped with 100GB or more of hard disk storage space for continuous and persistent storage of the accumulated detection database and pre-trained model parameter files. In terms of software environment configuration, the computing device can run mainstream operating systems such as Windows 10 or Linux (e.g., Ubuntu 20.04) and deploy a runtime environment of Python 3.8 or later. In addition, it requires the installation and configuration of basic functional libraries (including but not limited to numpy, opencv-python, tensorflow, matplotlib, pandas, and scikit-learn) for image matrix processing, depth tensor calculation, and machine learning data analysis. This comprehensive hardware and software environment setup provides a system-level foundation for real-time acquisition of multimodal sensing data under complex working conditions, efficient calculation of three-dimensional spatial physical dimensions, and smooth inference of deep defect recognition models.

[0111] Please see Figure 10 This application also provides a weld defect detection device, which can implement the above-mentioned weld defect detection method, including: The acquisition module is used to acquire two-dimensional image data and three-dimensional scan data of the target joint weld area, wherein the target joint weld area is the joint weld area formed by welding a tubular workpiece and a plate workpiece. The determination module is used to determine the center position of the tube sheet hole and the center position of the weld based on two-dimensional image data and three-dimensional scan data, and to calculate the center offset based on the center position of the tube sheet hole and the center position of the weld. The extraction module is used to extract weld profile data based on 3D scanning data and weld center position, and calculate the actual size data of the weld area of ​​the target joint based on the weld profile data; The input module is used to input two-dimensional image data into a pre-trained defect recognition model for recognition, and obtain the defect prediction results of the target joint weld area; The output module is used to output the quality judgment result of the weld area of ​​the target joint based on the center offset, actual size data and defect prediction results, combined with the preset weld standard parameters.

[0112] Reference Figure 11 , Figure 11 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 1101 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1102 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1102 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1102 and called and executed by the processor 1101 to execute the weld defect detection method of the embodiments of this application. Input / output interface 1103 is used to implement information input and output; The communication interface 1104 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, Wi-Fi, Bluetooth, etc.). Bus 1105 transmits information between various components of the device (e.g., processor 1101, memory 1102, input / output interface 1103, and communication interface 1104); The processor 1101, memory 1102, input / output interface 1103 and communication interface 1104 are connected to each other within the device via bus 1105.

[0113] This application also provides a computer program product, which includes a computer program. A processor of a computer device reads and executes the computer program, causing the computer device to perform the above-described weld defect detection method.

[0114] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0115] It should be understood that in this disclosure, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0116] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0117] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0120] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0122] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A method for detecting weld defects, characterized in that, The method includes: Two-dimensional image data and three-dimensional scan data of the target joint weld area are acquired; wherein, the target joint weld area is the joint weld area formed by welding a tubular workpiece and a plate-shaped workpiece; Based on the two-dimensional image data and the three-dimensional scan data, the center position of the tube sheet hole and the center position of the weld are determined, and the center offset is calculated according to the center position of the tube sheet hole and the center position of the weld. Based on the three-dimensional scanning data and the weld center position, weld profile data is extracted, and the actual size data of the target joint weld area is calculated based on the weld profile data. The two-dimensional image data is input into a pre-trained defect recognition model for recognition, and the defect prediction result of the target joint weld area is obtained. Based on the center offset, the actual size data, and the defect prediction results, combined with the preset weld standard parameters, the quality judgment result of the target joint weld area is output.

2. The weld defect detection method according to claim 1, characterized in that, The determination of the center position of the tube sheet hole and the center position of the weld based on the two-dimensional image data and the three-dimensional scan data includes: Edge extraction is performed on the two-dimensional image data to obtain image edge data; Perform circle detection on the image edge data to obtain candidate circles; If the candidate circle is not detected, the image edge data is used to extract the contour, find the largest circular contour, and take the center of the smallest circumcircle of the largest circular contour as the center position of the tube sheet hole. If the candidate circle is detected, the center of the candidate circle is taken as the center position of the tube sheet hole.

3. The weld defect detection method according to claim 1, characterized in that, The determination of the center position of the tube sheet hole and the center position of the weld based on the two-dimensional image data and the three-dimensional scan data includes: The two-dimensional image data is subjected to color space conversion and threshold segmentation to generate a two-dimensional feature mask; Height information is extracted from the three-dimensional scan data, and a three-dimensional height mask is generated based on the height information; The two-dimensional feature mask and the three-dimensional height mask are fused to obtain a fused mask; The contour of the fusion mask is extracted, and the center position of the weld is calculated based on the contour moments.

4. The weld defect detection method according to claim 1, characterized in that, The calculation of the center offset based on the center position of the tube sheet hole and the center position of the weld includes: Calculate the pixel distance between the center position of the tube sheet hole and the center position of the weld. Acquire the camera intrinsic parameter data of the vision system corresponding to the acquired two-dimensional image data; Based on the camera intrinsic data, the pixel distance is converted into an actual physical offset in physical space, and the actual physical offset is determined as the center offset.

5. The weld defect detection method according to claim 1, characterized in that, The step of extracting weld profile data based on the three-dimensional scanning data and the weld center position includes: Calculate the directional angle from the center position of the tube sheet hole to the center position of the weld; The three-dimensional scan data is smoothed by filtering to obtain smoothed three-dimensional scan data; Based on the weld center position and the direction angle, a first three-dimensional profile perpendicular to the direction angle and a second three-dimensional profile parallel to the direction angle are extracted from the smooth three-dimensional scan data. The first three-dimensional profile and the second three-dimensional profile are combined to determine the weld profile data.

6. The weld defect detection method according to claim 5, characterized in that, The actual dimension data includes the actual width and actual depth; the calculation of the actual dimension data of the target joint weld area based on the weld profile data includes: Extract the peak region span in the first three-dimensional profile, and convert the peak region span into the actual width; Calculate the height difference between the average height of the weld area and the average height of the surrounding area in the second three-dimensional profile, and convert the height difference into the actual depth.

7. The weld defect detection method according to claim 1, characterized in that, The defect prediction result includes the defect type and the corresponding defect confidence level; the step of inputting the two-dimensional image data into a pre-trained defect recognition model for recognition to obtain the defect prediction result of the target joint weld area includes: The two-dimensional image data is resized and normalized to obtain a standard input image; The standard input image is input into the defect recognition model for feature extraction and classification, and the probability distribution of various candidate defects is output. The candidate defect with the highest probability is identified as the defect type, and the corresponding probability value is identified as the confidence level.

8. The weld defect detection method according to claim 7, characterized in that, The preset weld standard parameters include center offset tolerance, nominal width tolerance, and nominal depth tolerance; based on the center offset, the actual size data, and the defect prediction results, combined with the preset weld standard parameters, the quality judgment result of the target joint weld area is output, including: Determine whether the center offset is within the center offset tolerance to obtain a first determination result; Determine whether the actual width is within the nominal width tolerance and whether the actual depth is within the nominal depth tolerance to obtain a second determination result; Based on the defect type and the defect confidence level, it is determined whether a serious defect exists, and a third judgment result is obtained; When the first judgment result, the second judgment result, and the third judgment result all meet the qualification conditions, the quality judgment result that the target joint weld area is qualified is output.

9. The weld defect detection method according to claim 8, characterized in that, The step of determining whether a serious defect exists based on the defect type and the defect confidence level to obtain a third determination result includes: If the defect type is normal, then the third judgment result characterizing the weld is qualified is generated; If the defect type is a local morphological defect and the defect confidence level is lower than the preset confidence level threshold, then it is determined that there is no serious defect, and the third judgment result characterizing the weld is qualified is generated. If the defect type is another defect state, then a serious defect is determined to exist, and an unqualified third judgment result is generated.

10. The weld defect detection method according to claim 1, characterized in that, After outputting the quality assessment result of the target joint weld area, the following is also included: The center positions of the tube sheet holes, the center positions of the welds, the actual size data, the defect prediction results, and the quality judgment results are superimposed on the two-dimensional image data for visualization. A test report containing the quality assessment results is generated, and the two-dimensional image data, the three-dimensional scan data, and the test report are associated and stored in the test database.

11. The weld defect detection method according to claim 1, characterized in that, Before inputting the two-dimensional image data into the pre-trained defect recognition model to obtain the defect prediction result for the target joint weld area, the method further includes: Obtain a historical set of two-dimensional image samples carrying defect type labels; Data augmentation processing is performed on the historical two-dimensional image sample set to obtain the training sample set; An initial convolutional neural network is constructed, with images in the training sample set as input and the corresponding defect type label as the expected output. The initial convolutional neural network is iteratively trained until the convergence condition is met, thereby obtaining the pre-trained defect recognition model.

12. The weld defect detection method according to claim 10, characterized in that, After storing the two-dimensional image data, the three-dimensional scan data, and the detection report in the detection database, the method further includes: When newly added stored data in the detection database meets preset conditions, historical image data and corresponding historical verification tags are extracted from the detection database. The historical image data and the historical verification labels are added to a preset model training set, and the defect recognition model is retrained using the updated model training set to update the defect recognition model.

13. A weld defect detection device, characterized in that, The device includes: The acquisition module is used to acquire two-dimensional image data and three-dimensional scan data of the target joint weld area, wherein the target joint weld area is the joint weld area formed by welding a tubular workpiece and a plate workpiece. The determination module is used to determine the center position of the tube sheet hole and the center position of the weld based on the two-dimensional image data and the three-dimensional scan data, and to calculate the center offset based on the center position of the tube sheet hole and the center position of the weld. The extraction module is used to extract weld profile data based on the three-dimensional scanning data and the weld center position, and to calculate the actual size data of the weld area of ​​the target joint based on the weld profile data; The input module is used to input the two-dimensional image data into a pre-trained defect recognition model for recognition, and obtain the defect prediction result of the target joint weld area; The output module is used to output the quality judgment result of the weld area of ​​the target joint based on the center offset, the actual size data and the defect prediction result, combined with the preset weld standard parameters.

14. An electronic device, characterized in that, include: The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the weld defect detection method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the weld defect detection method as described in any one of claims 1 to 12.

Images