Intelligent welding method based on deep learning and three-dimensional reconstruction

Abstract

The invention relates to the technical field of intelligent manufacturing and industrial robots, in particular to an intelligent welding method based on deep learning and three-dimensional reconstruction. Comprising the steps that point cloud data of a to-be-welded workpiece is obtained, a welding seam area is recognized through preprocessing and a point cloud segmentation network, and welding seam track information including welding seam space coordinates and a welding gun posture is obtained; optimizing the motion track of the robot through a global optimization algorithm and a local obstacle avoidance algorithm, and generating a planned track; welding seam position information is obtained in real time in the welding process, the real-time welding seam position is obtained through filtering of a state estimation algorithm and compared with a planned track, and position deviation is obtained and dynamically corrected; performing defect detection on the weld seam image, and combining process parameter analysis and case matching to generate a quality evaluation result; and closed-loop self-adaptive control is carried out based on the real-time welding seam position and the quality evaluation result. By means of the method, high-precision automatic extraction of the welding seam track, optimal planning of the movement track, real-time accurate tracking of the welding seam position and intelligent evaluation of the welding quality can be achieved.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing and industrial robot technology, and in particular to an intelligent welding method based on deep learning and three-dimensional reconstruction. Background Technology

[0002] Existing welding automation technologies largely rely on manual teaching and reproduction or traditional image processing algorithms, and often employ a single, fixed trajectory planning strategy. Manual teaching suffers from low programming efficiency, poor trajectory accuracy, and a lack of adaptability; traditional image processing algorithms are sensitive to lighting conditions and lack robustness; and single trajectory planning strategies struggle to adapt to complex and changing welding environments. Furthermore, existing solutions lack a complete closed loop from workpiece 3D perception, trajectory planning, real-time tracking to quality inspection, resulting in data flow breaks at each stage and hindering fully automated intelligent control of the welding process. Summary of the Invention

[0003] This invention addresses the problems in existing technologies, such as the lack of a complete data flow closed loop, low trajectory extraction accuracy, single planning strategy, and poor tracking and adaptive capabilities, by providing an intelligent welding method based on deep learning and 3D reconstruction.

[0004] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0005] This invention provides an intelligent welding method based on deep learning and 3D reconstruction, comprising:

[0006] The point cloud data of the workpiece to be welded is acquired, the point cloud data is preprocessed and the weld seam region is identified through a point cloud segmentation network, and the weld seam trajectory information is obtained through a feature extraction network based on the weld seam region. The weld seam trajectory information includes the weld seam spatial coordinates and the welding torch posture.

[0007] Based on the weld trajectory information, the motion trajectory of the welding robot is optimized by a global optimization algorithm and a local obstacle avoidance algorithm, and a planned trajectory is generated after trajectory smoothing.

[0008] During the welding process, the position information of the weld is acquired in real time. The position information is filtered by a state estimation algorithm to obtain the real-time weld position. The real-time weld position is compared with the planned trajectory to obtain the position deviation. The robot's motion trajectory is dynamically corrected based on the position deviation.

[0009] Defect detection is performed on weld images using an image recognition network, and quality assessment results are generated by combining process parameter analysis and case matching.

[0010] Based on the real-time weld position and the quality assessment results, closed-loop adaptive control is performed to automatically adjust welding parameters, forming a complete closed loop from point cloud data acquisition, trajectory planning, real-time tracking, quality detection, and parameter adjustment.

[0011] Optionally, point cloud data of the workpiece to be welded is acquired, and the point cloud data is preprocessed, including:

[0012] The point cloud data is acquired by laser 3D scanning and / or structured light projection.

[0013] A statistical outlier filtering algorithm is used to remove noise points. The number of neighborhood points k and the standard deviation multiple threshold σ are set. Points that deviate from the mean by more than σ times the standard deviation are identified as outliers and removed.

[0014] Voxel grid filtering is used to downsample the point cloud, and the voxel side length parameter is set.

[0015] Principal component analysis is used to estimate the normal vector of each point, and the search radius parameter is set.

[0016] Optionally, the weld seam region is identified through a point cloud segmentation network, and weld seam trajectory information is obtained based on the weld seam region through a feature extraction network, including:

[0017] The point cloud segmentation network employs a hierarchical point cloud processing network that includes a multi-scale feature extraction module and a feature propagation module, used to segment the point cloud data into weld seam regions and non-weld seam regions.

[0018] The feature extraction network employs a graph convolutional neural network to model the point cloud data of the weld area as a graph structure, where nodes represent sampling points and edges represent spatial adjacency relationships. The graph convolutional neural network includes an edge convolutional layer, a graph attention layer, and a global pooling layer. The edge convolutional layer is used to aggregate the geometric features of neighboring points, the graph attention layer is used to adaptively learn the weight coefficients of neighboring points, and the global pooling layer is used to integrate local features into a global descriptor.

[0019] The point cloud segmentation network outputs weld type classification results while segmenting the weld seam region. The weld seam types include at least butt welds, fillet welds, lap welds, and plug welds. The weld seam trajectory information includes weld seam spatial coordinates, welding torch posture, and the weld seam type classification results.

[0020] Optionally, the motion trajectory of the welding robot can be optimized using global optimization algorithms and local obstacle avoidance algorithms, including:

[0021] The global optimization algorithm employs an adaptive genetic algorithm, which optimizes based on a multi-objective fitness function, namely: where \(P\) represents the trajectory path, \(L(P)\) is the path length index, \(S(P)\) is the path smoothness index defined as the cumulative sum of the included angles of the tangent vectors of adjacent trajectory segments, \(E(P)\) is the joint energy consumption index defined as the integral of the square of the angular velocity of each joint, and \(D(P)\) is the obstacle avoidance safety margin index defined as the reciprocal of the minimum distance between each point on the trajectory and the nearest obstacle. and and and are weight coefficients and satisfy ;

[0022] The local obstacle avoidance algorithm adopts the artificial potential field method, and calculates the resultant force direction of the gravitational force generated by the target point and the repulsive force generated by the obstacle as the local optimization direction. The crossover probability and mutation probability of the adaptive genetic algorithm are dynamically adjusted according to the population fitness.

[0023] Optionally, the crossover probability and the mutation probability are dynamically adjusted according to the following calculation formulas:

[0024] when ; when ;

[0025] when ; when \(f < f_{avg}\);

[0026] where \(f_{max}\) is the maximum fitness value of the population, \(f_{avg}\) is the average fitness value of the population, \(f'\) is the larger fitness value of the two individuals participating in the crossover, and \(f\) is the fitness value of the individual to be mutated; \(P_{c1}\), \(P_{c2}\) are the upper and lower limits of the crossover probability respectively, and \(P_{m1}\), \(P_{m2}\) are the upper and lower limits of the mutation probability respectively.

[0027] The trajectory smoothing process adopts quintic polynomial interpolation and simultaneously constrains the boundary conditions of position, velocity and acceleration.

[0028] Optionally, during the welding process, the position information of the weld is obtained in real time, including:

[0029] Project line laser onto the weld area and synchronously collect images, and sequentially perform Gaussian filtering, adaptive threshold segmentation, morphological processing and laser stripe centerline extraction on the images;

[0030] Calculate the three-dimensional space coordinates of the weld based on the principle of laser triangulation ranging, and the calculation formula is: , where Z is the distance from the weld surface point to the camera optical center, f is the focal length of the camera lens, B is the baseline distance between the laser and the camera, d is the offset of the laser stripe on the image sensor, and θ is the angle between the laser plane and the camera optical axis.

[0031] The intrinsic parameter matrix and distortion coefficients are obtained through camera calibration, and the transformation relationship between the sensor coordinate system and the robot end effector coordinate system is obtained through hand-eye calibration.

[0032] Optionally, the position information is filtered using a state estimation algorithm to obtain the real-time weld position, and the robot's motion trajectory is dynamically corrected based on the position deviation, including:

[0033] The state estimation algorithm employs the Kalman filter algorithm, and its state equation is as follows: Observation equation Where Xk is the state vector at time k, including the position of the weld in the three coordinate axes and its rate of change; A is the state transition matrix, B is the control matrix, Uk-1 is the control vector, Wk-1 is the process noise; Zk is the observation vector at time k, H is the observation matrix, and Vk is the observation noise.

[0034] The dynamic correction uses a PID controller to decompose the positional deviation into vertical deviation and horizontal deviation, and compensates for them separately. The compensation results are then superimposed on the planned trajectory to generate the corrected trajectory command.

[0035] Optionally, defect detection is performed on the weld image using an image recognition network, including:

[0036] The image recognition network uses a convolutional neural network, which includes multiple convolutional blocks connected in sequence. Each convolutional block includes a convolutional layer, a batch normalization layer, an activation function, and a pooling layer. The convolutional features are fed into a fully connected layer after global average pooling to output the probability distribution of each defect category.

[0037] The defects to be detected include at least porosity, cracks, inclusions, lack of fusion, undercut, and poor surface finish.

[0038] Optionally, quality assessment results can be generated by combining process parameter analysis and case matching, including:

[0039] The process parameter analysis uses a recurrent neural network to detect anomalies in the timing data of welding current, arc voltage, and wire feed speed.

[0040] The case matching is based on a welding quality knowledge graph and a case database. A similarity calculation method optimized by a neural network is used to retrieve matching cases. The similarity calculation formula is as follows: Where C is the current case to be detected, and Ci is the i-th case in the case library. and Let J be the j-th feature attribute of C and Ci, respectively. These are the feature weight coefficients for neural network learning. This is a local similarity function.

[0041] Optionally, closed-loop adaptive control is performed based on the real-time weld position and the quality assessment results to automatically adjust welding parameters, including:

[0042] Welding process parameters are collected in real time, and a welding quality prediction model is established by combining the real-time weld position and the quality assessment results.

[0043] When welding deviations or quality abnormalities are detected, welding parameters are automatically adjusted based on reinforcement learning algorithms, including incremental adjustments to welding current, arc voltage, and welding speed. The adjustment strategy is as follows: for high and low deviations, the arc voltage and wire extension are mainly adjusted; for gap changes, the welding current and wire feed speed are mainly adjusted. Left and right deviations are compensated for by a dynamic correction step. The closed-loop adaptive control and the dynamic trajectory correction work together to form a dual closed-loop control from the two dimensions of welding process parameters and robot motion trajectory.

[0044] The adjusted welding parameters are fed back to the welding execution stage in real time. The point cloud data, planned trajectory, real-time tracking data, quality assessment results, and adjusted welding parameters of this welding process are associated and stored in the welding database. The accumulated welding data is analyzed and quality labeled regularly. High-quality welding cases are added to the training dataset to incrementally train and update the point cloud segmentation network, feature extraction network, and image recognition network. This realizes a complete closed loop from point cloud data acquisition, trajectory planning, real-time tracking, quality detection to parameter adjustment, as well as continuous iterative optimization based on data accumulation.

[0045] By implementing this invention, the following beneficial effects can be achieved:

[0046] By implementing this invention, it is possible to acquire point cloud data of the workpiece to be welded and perform preprocessing and segmentation, obtain weld trajectory information through a feature extraction network, and provide a high-precision geometric data foundation for subsequent trajectory planning. Compared with the traditional manual teaching method, the programming efficiency is improved by more than 10 times.

[0047] By implementing this invention, it is possible to generate a planned trajectory based on weld seam trajectory information using a global optimization algorithm and a local obstacle avoidance algorithm, breaking the limitations of a single strategy and allowing trajectory planning to simultaneously consider global optimization and local obstacle avoidance, thereby shortening the path length by more than 18% and improving motion smoothness by more than 45%.

[0048] By implementing this invention, the weld position can be acquired in real time during the welding process and compared and corrected with the planned trajectory, enabling the welding robot to have real-time adaptive capability, with a tracking accuracy better than ±0.3mm, effectively compensating for workpiece clamping errors and thermal deformation.

[0049] By implementing this invention, it is possible to generate quality assessment results through image recognition networks and case matching, thereby achieving intelligent identification of welding defects with a defect detection accuracy rate exceeding 96.2%, providing an effective technical means for welding quality control.

[0050] By implementing this invention, closed-loop adaptive control based on real-time weld position and quality assessment results can be achieved, automatically adjusting welding parameters and updating the database, enabling the welding system to have continuous learning and iterative optimization capabilities, and improving the welding quality pass rate to over 99%.

[0051] In summary, by implementing this invention, a complete closed-loop control of the welding process—from perception and planning to execution, detection, and adjustment—can be achieved. A clear data flow can be established, from point cloud data to weld trajectory information, planned trajectory, real-time weld position, quality assessment results, and parameter adjustment, significantly improving the level of welding automation and intelligence. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating an intelligent welding method based on deep learning and 3D reconstruction provided by the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0055] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0056] like Figure 1 As shown, this invention provides an intelligent welding method based on deep learning and 3D reconstruction, comprising the following steps:

[0057] S100. Obtain point cloud data of the workpiece to be welded, preprocess the point cloud data and identify the weld seam region through a point cloud segmentation network, and obtain weld seam trajectory information based on the weld seam region through a feature extraction network. The weld seam trajectory information includes weld seam spatial coordinates and welding torch posture.

[0058] S200. Based on the weld trajectory information, the motion trajectory of the welding robot is optimized by a global optimization algorithm and a local obstacle avoidance algorithm, and a planned trajectory is generated after trajectory smoothing.

[0059] S300. During the welding process, the position information of the weld is acquired in real time. The position information is filtered by a state estimation algorithm to obtain the real-time weld position. The real-time weld position is compared with the planned trajectory to obtain the position deviation. The robot's motion trajectory is dynamically corrected based on the position deviation.

[0060] S400 uses an image recognition network to detect defects in weld images and generates quality assessment results by combining process parameter analysis and case matching.

[0061] S500: Based on the real-time weld position and the quality assessment results, closed-loop adaptive control is performed to automatically adjust the welding parameters, forming a complete closed loop from point cloud data acquisition, trajectory planning, real-time tracking, quality detection, and parameter adjustment.

[0062] In this embodiment, the purpose of step S100 is to acquire precise geometric information of the workpiece to be welded using advanced three-dimensional sensing technology, providing a high-precision data foundation for subsequent trajectory planning and welding control. This step outputs weld trajectory information, which serves as the input to step S200, establishing the first loop data stream.

[0063] To achieve this goal, the point cloud data of the workpiece to be welded must first be obtained.

[0064] For example, for a certain type of engineering machinery box structure component, a Focus3D X330 laser 3D scanner was used to acquire point cloud data via laser 3D scanning. The scanning resolution was set to 1 / 4, the scanning quality to 4×, and the number of points in a single scan was approximately 40 million. Through multi-view scanning, a complete point cloud data with a total of approximately 200 million scan points was obtained. Structured light projection can also be used to acquire point cloud data, which is suitable for high-speed acquisition of large-area topographic information.

[0065] Next, the point cloud data is preprocessed.

[0066] For example, a statistical outlier filtering algorithm is used to remove noise points. The number of neighboring points is set to k=50, and the standard deviation threshold σ=1.0. The average distance from each point to its k nearest neighbors is calculated. Points deviating from the mean by more than 1.0 times the standard deviation are identified as outliers and removed, eliminating approximately 2% of the total points. A voxel grid filter is used to downsample the point cloud, with a voxel side length of 2mm. After downsampling, the point cloud size is reduced from 200 million points to approximately 5 million points. Principal component analysis is used to estimate the normal vector of each point, with a search radius of 5mm.

[0067] Then, the weld seam region is identified using a point cloud segmentation network.

[0068] For example, the point cloud segmentation network adopts the PointNet++ architecture, which includes a multi-scale feature extraction module and a feature propagation module. Through hierarchical point set abstraction and feature propagation mechanisms, it segments the preprocessed point cloud data into weld seam regions, base material regions, and background regions. The segmentation accuracy (IoU) reaches over 92%. The training dataset contains 3000 sets of welded workpiece point cloud data, of which 2400 sets are used for training and 600 sets are used for validation. The point cloud data is manually labeled with weld seam regions, base material regions, and background regions. The cross-entropy loss function and Adam optimizer are used, with an initial learning rate of 0.001, which decays to 0.5 times the original rate every 50 epochs. The batch size is 16, and the training time is approximately 8 hours on an NVIDIA RTX 3090 GPU after 200 epochs.

[0069] Based on the weld area, weld trajectory information is obtained through a feature extraction network.

[0070] For example, the feature extraction network employs a graph convolutional neural network to model the point cloud data of the weld region as a graph structure, where nodes represent sampling points and edges represent the spatial adjacency relationships between points. The network architecture includes edge convolutional layers, graph attention layers, and global pooling layers: the edge convolutional layers update node representations by aggregating the geometric features of neighboring points; the graph attention layers adaptively learn the contribution weights of different neighboring points through an attention mechanism; and the global pooling layers integrate local node features into global graph features. The weld trajectory information output by the network includes: weld type classification results (butt weld, fillet weld, lap weld, or plug weld), weld spatial coordinate sequence (centerline sampling interval 5mm), and welding torch posture angle at each sampling point. The average deviation between the extracted weld trajectory and the manually taught trajectory is less than 0.5mm. The training dataset for the graph convolutional feature extraction network contains 5000 weld trajectory samples, each trajectory containing 200-500 sampling points and their labeled weld type, spatial coordinates, and welding torch posture. Mean squared error loss function is used for coordinate regression, and cross-entropy loss function is used for type classification. The weight ratio of the multi-task loss is 1:0.5. The Adam optimizer is used with a learning rate of 0.0005 and training is performed for 300 epochs.

[0071] In this embodiment, the purpose of step S200 is to plan the optimal motion path of the welding robot based on the weld trajectory information extracted in step S100, while ensuring that the robot does not collide with the workpiece, fixture, or other obstacles during its movement. This step outputs the planned trajectory as a reference benchmark for real-time tracking in step S300, establishing the second loop of data stream.

[0072] To achieve this goal, it is first necessary to establish a mathematical model and an obstacle avoidance space model for the welding robot.

[0073] For example, for the ABB IRB 1410 six-DOF welding robot (maximum load 5kg, working radius 1.44m), a mathematical model of the link geometry and joint kinematics is established using the standard DH parameter method. The workpiece, fixture, and surrounding equipment are simplified into an axis-aligned bounding box model, and an octree is used to hierarchically organize the obstacle space, reducing the obstacle lookup time complexity to O(log n).

[0074] Next, the motion trajectory of the welding robot is optimized using a global optimization algorithm.

[0075] For example, the global optimization algorithm mentioned above employs an adaptive genetic algorithm based on a multi-objective fitness function. Optimize by setting weight coefficients. , , , Where L(P) is the path length index, S(P) is the path smoothness index, E(P) is the joint energy consumption index, and D(P) is the obstacle avoidance safety margin index. The population size is set to 100, the maximum number of iterations is 200, the crossover probability upper and lower bounds are Pc1=0.9 and Pc2=0.6, and the mutation probability upper and lower bounds are Pm1=0.1 and Pm2=0.001. The algorithm converges after approximately 150 generations, obtaining an approximately optimal trajectory.

[0076] Then, the trajectory is locally optimized using a local obstacle avoidance algorithm.

[0077] For example, the local obstacle avoidance algorithm uses the artificial potential field method to calculate the attraction generated by the target point and the repulsion generated by the obstacle for each path point. The direction of the resultant force of the two is used as the local optimization direction to ensure that the safety margin is greater than 50mm.

[0078] Finally, the planned trajectory is generated after trajectory smoothing.

[0079] For example, the trajectory smoothing process employs fifth-order polynomial interpolation, while simultaneously constraining the boundary conditions for position, velocity, and acceleration. After interpolation, the maximum velocity of the trajectory does not exceed 1 m / s, and the maximum acceleration does not exceed 5 m / s². Optimization results show that, compared to traditional manually taught trajectories, the optimized trajectory has a 18% shorter total length, a 22% shorter motion time, and a 15% lower joint energy consumption.

[0080] In this embodiment, the purpose of step S300 is to detect the position and orientation information of the weld in real time during the welding process, compare the detected real-time weld position with the planned trajectory output in step S200, calculate the position deviation, and feed it back to the robot control system to achieve precise tracking of the weld. This step outputs the real-time weld position, establishes the third-loop data stream, and provides position feedback information for step S500.

[0081] To achieve this goal, it is first necessary to obtain the location information of the weld in real time during the welding process.

[0082] For example, a composite vision sensing system can be formed by installing a line laser sensor (660nm laser wavelength, 0.1mm line width) and an industrial camera (1280×1024 resolution, 60fps) at the end effector of a welding robot. The line laser is projected onto the weld area while images are simultaneously acquired. These images are then processed sequentially: Gaussian filtering is performed using a 5×5 Gaussian kernel; the laser stripe region is extracted using the Otsu adaptive threshold segmentation algorithm; morphological opening and closing operations are performed using a 5×5 structuring element; and the center line of the laser stripe is extracted using the grayscale centroid method, achieving a sub-pixel positioning accuracy of 0.1 pixels.

[0083] Next, the three-dimensional spatial coordinates of the weld are calculated based on the principle of laser triangulation.

[0084] For example, using formulas The distance Z from a point on the weld surface to the camera's optical center is calculated, where f is the camera lens focal length, B is the baseline distance between the laser and the camera, d is the offset of the laser stripe on the image sensor, and θ is the angle between the laser plane and the camera's optical axis. The system's measurement accuracy is ±0.1 mm. The camera's intrinsic parameter matrix and distortion coefficients are obtained using the Zhang Zhengyou calibration method, and the transformation relationship between the sensor coordinate system and the robot's end effector coordinate system is obtained through hand-eye calibration, with a calibration error of less than 0.2 mm.

[0085] Then, the position information is filtered using a state estimation algorithm to obtain the real-time weld position.

[0086] For example, the state estimation algorithm uses the Kalman filter algorithm to establish a six-dimensional state-space model. State equations Observation equation The state vector Xk includes the position of the weld in the x, y, and z coordinate axes and its rate of change. The process noise variance was set to 0.01 mm², and the observation noise variance to 0.1 mm². After filtering, the root mean square error of the position estimation was reduced to 0.15 mm.

[0087] The position deviation is obtained by comparing the real-time weld position with the planned trajectory, and the robot's motion trajectory is dynamically corrected based on the position deviation.

[0088] For example, a dual-channel PID controller is used to decompose the positional deviation into vertical and horizontal deviations, which are compensated separately, with a control cycle of 10ms. When a positional deviation exceeding 0.5mm is detected, a correction is triggered, and the compensation result is superimposed on the planned trajectory to generate the corrected trajectory command. The correction response time is less than 50ms, and the steady-state tracking accuracy is better than ±0.3mm.

[0089] To achieve the fusion of the point cloud data acquired through offline scanning in step S100 and the real-time position information acquired through online detection in step S300 within the same coordinate system, this invention establishes a unified world coordinate system. Specifically, by scanning the calibration target on the welding worktable, the transformation matrix between the laser 3D scanner coordinate system and the world coordinate system is obtained. The transformation matrix between the line laser sensor coordinate system and the robot end effector coordinate system is obtained through hand-eye calibration. The transformation matrix between the robot's end effector coordinate system and the world coordinate system is obtained by combining the robot's forward kinematics model. Offline point cloud data is obtained through... Transform to the world coordinate system, online real-time position via Transform to the world coordinate system to achieve spatial registration of the two types of sensor data.

[0090] In this embodiment, the purpose of step S400 is to perform quality inspection on the welded structural components, identify welding defect types, assess welding quality levels, and generate improvement suggestions. This step outputs the quality assessment results, establishes the fourth-loop data stream, and provides quality feedback information for step S500.

[0091] To achieve this goal, the weld images are first inspected for defects using an image recognition network.

[0092] For example, the image recognition network employs a convolutional neural network with five convolutional blocks, taking a 256×256 pixel weld image as input. Each convolutional block includes two 3×3 convolutional layers, a batch normalization layer, a ReLU activation function, and a 2×2 max pooling layer, with the number of convolutional kernels being 64, 128, 256, 512, and 512 respectively. The convolutional features are fed into a fully connected layer after global average pooling, and the Softmax function is used to output the probability distribution of each defect category. The defects detected include porosity, cracks, slag inclusions, lack of fusion, undercut, and poor surface finish. The model achieves a classification accuracy of 96.2%. The training dataset for the defect detection convolutional neural network contains 12,000 weld images, covering six types of defects and normal welds, with the number of samples in each category balanced by oversampling. The cross-entropy loss function and SGD optimizer are used, with an initial learning rate of 0.01, momentum of 0.9, and weight decay of 0.0001. The batch size is 32, and the training lasts for 100 epochs.

[0093] Then, anomaly detection is carried out by combining process parameter analysis.

[0094] For example, the process parameter analysis employs a long short-term memory recurrent neural network to perform sequential modeling of the time-series data of the welding process parameters. By using multi-dimensional time series data of parameters such as welding current, arc voltage, and wire feed speed as input, the system learns the parameter variation patterns of a normal welding process and detects and issues early warnings for abnormal fluctuations.

[0095] Next, quality assessment results are generated based on case matching.

[0096] For example, the case matching is based on a welding quality knowledge graph and a case database. The entity types in the knowledge graph include welding defects, welding parameters, base material types, and welding methods; the relationship types include causal relationships, influence relationships, and applicability relationships. The case database contains 500 welding defect cases, each with 12 attributes. A similarity calculation formula optimized using a neural network is employed. Search for matching cases and generate welding quality assessment reports and improvement suggestions.

[0097] In this embodiment of the application, the purpose of step S500 is to perform closed-loop adaptive control based on the real-time weld position in step S300 and the quality assessment results in step S400, automatically adjust the welding parameters, form a complete closed loop from point cloud data acquisition, trajectory planning, real-time tracking, quality detection to parameter adjustment, and realize continuous iterative optimization of the welding process.

[0098] To achieve this goal, welding process parameters must first be collected in real time to establish a welding quality prediction model.

[0099] For example, process parameters such as welding current, arc voltage, wire feed speed, and welding speed are collected, with a sampling period of 10ms. Combining the real-time weld position (including position deviation information) from step S300 and the quality assessment results (including defect type and confidence level) from step S400, a multi-input multi-output welding quality prediction model is established.

[0100] When welding deviations or quality abnormalities are detected, welding parameters are automatically adjusted based on reinforcement learning algorithms.

[0101] For example, the adjustment parameters include the welding current increment ΔI, the arc voltage increment ΔU, and the welding speed increment Δv. The adjustment strategy is determined according to the type of deviation: for high and low deviations, the arc voltage and wire extension are mainly adjusted, because the arc voltage directly affects the arc length and penetration depth; for gap changes, the welding current and wire feed speed are mainly adjusted to adapt to changes in filler volume. Left and right directional deviations are spatial position deviations, which are compensated in real time by the PID controller in step S300 through dynamic trajectory correction, with a compensation period of 10ms. The closed-loop adaptive control in step S500 works in conjunction with the dynamic trajectory correction in step S300. The former is responsible for optimizing welding process parameters, and the latter is responsible for accurate spatial trajectory tracking. Together, they constitute a dual closed-loop control system for the welding process. The reinforcement learning algorithm adopts the proximal policy optimization method. The state space includes the real-time weld position and quality assessment results, the action space includes the incremental adjustment range of welding parameters, and the reward function comprehensively considers welding quality and efficiency indicators. The reinforcement learning algorithm adopts the proximal policy optimization (PPO) method. The policy network employs a three-layer fully connected neural network with 128, 64, and 32 neurons in each layer, using ReLU as the activation function. The value network structure is identical to the policy network. The state space is a 12-dimensional vector, including the weld's positional deviation and rate of change along the three coordinate axes (6-dimensional) and the one-hot encoding of the defect type (6-dimensional). The action space is a 3-dimensional continuous space, including the welding current increment. Arc voltage increment Welding speed increment The reward function is defined as: , where |Δp| ​​is the Euclidean distance of the positional deviation, and D is the defect severity score (0-1). For changes in energy consumption, the weighting coefficient is... , , The Adam optimizer was used for training, with a learning rate of 3×10^-4, a discount factor γ=0.99, a batch size of 64, and 5000 training epochs.

[0102] The adjusted welding parameters are fed back to the welding execution stage, and the new welding trajectory data is updated to the point cloud database to achieve continuous iterative optimization.

[0103] For example, the adjusted welding parameters are sent in real time to the welding power source and wire feeding mechanism via the robot controller. Simultaneously, the trajectory data, process parameters, and quality assessment results collected during the welding process are packaged and stored. Causal effect analysis and trajectory quality annotation are performed periodically (e.g., monthly) on new data. High-quality trajectories are added to the training dataset, and incremental updates are performed on the point cloud segmentation network, feature extraction network, and image recognition network. The closed-loop control response time is less than 50ms, and after three months of system operation, the welding quality pass rate increased from 95% to over 99%.

[0104] Through the above steps S100 to S500, the present invention establishes a complete data flow closed loop: step S100 outputs weld trajectory information → step S200 outputs planned trajectory based on weld trajectory information → step S300 outputs real-time weld position based on planned trajectory → step S400 outputs quality assessment result → step S500 outputs adjusted welding parameters based on real-time weld position and quality assessment result, and feeds back the new data to update the database of step S100, forming a complete closed loop of continuous iterative optimization.

[0105] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0106] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0107] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0108] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0109] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0110] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Clearly, those skilled in the art can make various alterations and variations to the invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the invention and its equivalents, the invention is also intended to include these modifications and variations.

Claims

1. An intelligent welding method based on deep learning and 3D reconstruction, characterized in that, include: The point cloud data of the workpiece to be welded is acquired, the point cloud data is preprocessed and the weld seam region is identified through a point cloud segmentation network, and the weld seam trajectory information is obtained through a feature extraction network based on the weld seam region. The weld seam trajectory information includes the weld seam spatial coordinates and the welding torch posture. Based on the weld trajectory information, the motion trajectory of the welding robot is optimized by a global optimization algorithm and a local obstacle avoidance algorithm, and a planned trajectory is generated after trajectory smoothing. During the welding process, the position information of the weld is acquired in real time. The position information is filtered by a state estimation algorithm to obtain the real-time weld position. The real-time weld position is compared with the planned trajectory to obtain the position deviation. The robot's motion trajectory is dynamically corrected based on the position deviation. Defect detection is performed on weld images using an image recognition network, and quality assessment results are generated by combining process parameter analysis and case matching. Based on the real-time weld position and the quality assessment results, closed-loop adaptive control is performed to automatically adjust welding parameters, forming a complete closed loop from point cloud data acquisition, trajectory planning, real-time tracking, quality detection, and parameter adjustment.

2. The intelligent welding method based on deep learning and 3D reconstruction according to claim 1, characterized in that, Acquire point cloud data of the workpiece to be welded, and preprocess the point cloud data, including: The point cloud data is acquired by laser 3D scanning and / or structured light projection. A statistical outlier filtering algorithm is used to remove noise points. Thresholds are set for the number of neighborhood points and the multiple of standard deviation. Points that deviate from the mean by more than one multiple of standard deviation are identified as outliers and removed. Voxel grid filtering is used to downsample the point cloud, and the voxel side length parameter is set. Principal component analysis is used to estimate the normal vector of each point, and the search radius parameter is set.

3. The intelligent welding method based on deep learning and 3D reconstruction according to claim 2, characterized in that, Weld seam regions are identified using a point cloud segmentation network, and weld seam trajectory information is obtained based on these regions using a feature extraction network, including: The point cloud segmentation network employs a hierarchical point cloud processing network that includes a multi-scale feature extraction module and a feature propagation module, used to segment the point cloud data into weld seam regions and non-weld seam regions. The feature extraction network employs a graph convolutional neural network to model the point cloud data of the weld area as a graph structure, where nodes represent sampling points and edges represent spatial adjacency relationships. The graph convolutional neural network includes an edge convolutional layer, a graph attention layer, and a global pooling layer. The edge convolutional layer is used to aggregate the geometric features of neighboring points, the graph attention layer is used to adaptively learn the weight coefficients of neighboring points, and the global pooling layer is used to integrate local features into a global descriptor. The point cloud segmentation network outputs weld type classification results while segmenting the weld seam region. The weld seam types include at least butt welds, fillet welds, lap welds, and plug welds. The weld seam trajectory information includes weld seam spatial coordinates, welding torch posture, and the weld seam type classification results.

4. The intelligent welding method based on deep learning and 3D reconstruction according to claim 1, characterized in that, The motion trajectory of the welding robot is optimized using global optimization algorithms and local obstacle avoidance algorithms, including: The global optimization algorithm employs an adaptive genetic algorithm, which optimizes based on a multi-objective fitness function, namely: Where P represents the trajectory path, L(P) is the path length index, S(P) is the path smoothness index defined as the cumulative sum of the angles between the tangent vectors of adjacent trajectory segments, E(P) is the joint energy consumption index defined as the integral of the square of the angular velocity of each joint, and D(P) is the obstacle avoidance safety margin index defined as the reciprocal of the minimum distance between each point on the trajectory and the nearest obstacle. , , , The weighting coefficients are and satisfy the following conditions: ; The local obstacle avoidance algorithm uses the artificial potential field method to calculate the resultant force direction of the attraction generated by the target point and the repulsion generated by the obstacle as the local optimization direction. The crossover probability and mutation probability of the adaptive genetic algorithm are dynamically adjusted according to the population fitness.

5. The intelligent welding method based on deep learning and 3D reconstruction according to claim 4, characterized in that, The dynamic adjustment formulas for the crossover probability Pc and mutation probability Pm are as follows: ,when hour; ,when hour; ,when hour ,when hour; Where fmax is the maximum fitness value of the population, favg is the average fitness value of the population, f' is the larger fitness value of the two individuals participating in the crossover, and f is the fitness value of the individual to be mutated; Pc1 and Pc2 are the upper and lower limits of the crossover probability, respectively, and Pm1 and Pm2 are the upper and lower limits of the mutation probability, respectively. The trajectory smoothing process employs fifth-order polynomial interpolation, while simultaneously constraining the boundary conditions for position, velocity, and acceleration.

6. The intelligent welding method based on deep learning and 3D reconstruction according to claim 1, characterized in that, The location information of the weld seam is acquired in real time during the welding process, including: A line laser is projected onto the weld area and images are acquired simultaneously. The images are then subjected to Gaussian filtering, adaptive threshold segmentation, morphological processing, and laser stripe centerline extraction in sequence. The three-dimensional spatial coordinates of the weld are calculated based on the principle of laser triangulation. The calculation formula is as follows: , where Z is the distance from the weld surface point to the camera optical center, f is the camera lens focal length, B is the baseline distance between the laser and the camera, d is the offset of the laser stripe on the image sensor, and θ is the angle between the laser plane and the camera optical axis. The intrinsic parameter matrix and distortion coefficients are obtained through camera calibration, and the transformation relationship between the sensor coordinate system and the robot end effector coordinate system is obtained through hand-eye calibration.

7. The intelligent welding method based on deep learning and 3D reconstruction according to claim 6, characterized in that, The position information is filtered using a state estimation algorithm to obtain the real-time weld position. The robot's trajectory is then dynamically corrected based on the position deviation, including: The state estimation algorithm employs the Kalman filter algorithm, and its state equation is as follows: The observation equation is Where Xk is the state vector at time k, including the position of the weld in the three coordinate axes and its rate of change; A is the state transition matrix, B is the control matrix, Uk-1 is the control vector, Wk-1 is the process noise; Zk is the observation vector at time k, H is the observation matrix, and Vk is the observation noise. The dynamic correction uses a PID controller to decompose the positional deviation into vertical deviation and horizontal deviation, and compensates for them separately. The compensation results are then superimposed on the planned trajectory to generate the corrected trajectory command.

8. The intelligent welding method based on deep learning and 3D reconstruction according to claim 1, characterized in that, Defect detection in weld images using image recognition networks includes: The image recognition network uses a convolutional neural network, which includes multiple convolutional blocks connected in sequence. Each convolutional block includes a convolutional layer, a batch normalization layer, an activation function, and a pooling layer. The convolutional features are fed into a fully connected layer after global average pooling to output the probability distribution of each defect category. The defects to be detected include at least porosity, cracks, inclusions, lack of fusion, undercut, and poor surface finish.

9. The intelligent welding method based on deep learning and 3D reconstruction according to claim 8, characterized in that, Quality assessment results are generated by combining process parameter analysis and case matching, including: The process parameter analysis uses a recurrent neural network to detect anomalies in the timing data of welding current, arc voltage, and wire feed speed. The case matching is based on a welding quality knowledge graph and a case database. A similarity calculation method optimized by a neural network is used to retrieve matching cases. The similarity calculation formula is as follows: Where C is the current case to be detected, and Ci is the i-th case in the case library. and Let J be the j-th feature attribute of C and Ci, respectively. These are the feature weight coefficients for neural network learning. This is a local similarity function.

10. The intelligent welding method based on deep learning and 3D reconstruction according to claim 1, characterized in that, Closed-loop adaptive control is performed based on the real-time weld position and the quality assessment results to automatically adjust welding parameters, including: Welding process parameters are collected in real time, and a welding quality prediction model is established by combining the real-time weld position and the quality assessment results. When welding deviations or quality abnormalities are detected, welding parameters are automatically adjusted based on reinforcement learning algorithms, including incremental adjustments to welding current, arc voltage, and welding speed. The adjustment strategy is as follows: for high and low deviations, the arc voltage and wire extension are mainly adjusted; for gap changes, the welding current and wire feed speed are mainly adjusted. Left and right deviations are compensated for by a dynamic correction step. The closed-loop adaptive control and the dynamic trajectory correction work together to form a dual closed-loop control from the two dimensions of welding process parameters and robot motion trajectory. The adjusted welding parameters are fed back to the welding execution stage in real time. The point cloud data, planned trajectory, real-time tracking data, quality assessment results, and adjusted welding parameters of this welding process are associated and stored in the welding database. The accumulated welding data is analyzed and quality labeled regularly. High-quality welding cases are added to the training dataset to incrementally train and update the point cloud segmentation network, feature extraction network, and image recognition network. This realizes a complete closed loop from point cloud data acquisition, trajectory planning, real-time tracking, quality detection to parameter adjustment, as well as continuous iterative optimization based on data accumulation.

Images