Multi-type welding seam structured light positioning method based on image synthesis
By generating structured light images of weld seams in a virtual environment and training a convolutional neural network, the adaptability and robustness issues of existing weld seam recognition methods are solved, achieving high-precision weld seam cross-section recognition and localization.
Patent Information
- Application Number
- CN202511051804.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-29
- Publication Date
- 2025-12-02
AI Technical Summary
Existing weld identification methods have shortcomings in terms of adaptability, accuracy, and robustness. They are difficult to adapt to diverse weld structures, and image recognition is susceptible to interference. The training data annotation error is large, making it difficult to achieve high-precision weld cross-section identification.
By constructing a virtual simulation environment to generate structured light images of welds, using a convolutional neural network to train a weld recognition model, adding simulation noise interference, and directly predicting the cross-sectional morphology of the weld, the recognition process is simplified and the model's adaptability and anti-interference ability are improved.
It improves the adaptability and accuracy of weld seam recognition, reduces the dependence on training data, and enhances the robustness and recognition efficiency of the model in complex environments.
Smart Images
Figure CN121053237A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a structured light recognition method for welds, specifically a method for locating welds from structured light stripe images of welds. Background Technology
[0002] With the increasing demands for welding quality, efficiency, and automation in industrial manufacturing, image-based weld seam positioning technology has become one of the key technologies for realizing high-precision automated welding systems. Structured light 3D imaging, due to its advantages such as high resolution, non-contact operation, and strong anti-obstruction capability, is widely used in weld seam inspection and positioning tasks. By projecting striped light or coded light patterns onto the workpiece surface and combining this with camera image acquisition and processing, the 3D contour information of the weld seam can be reconstructed, assisting in the extraction of the weld seam centerline, feature point recognition, and trajectory planning.
[0003] Current common weld structured light recognition methods typically rely on weld models with fixed geometric shapes, such as V-grooves, U-grooves, and fillet welds. These methods are mainly based on image processing or shallow machine learning techniques, extracting weld contours through regular template matching, edge detection, and grayscale gradient analysis. However, due to the diverse types of welds in industrial settings, the complex surface morphology of workpieces, and significant variations in weld size, groove angle, and fill height, traditional methods struggle to adapt to the diverse weld structures. They are particularly inadequate for irregular weld structures or curved weld surfaces, easily leading to detection failures or decreased accuracy.
[0004] Furthermore, existing methods rely heavily on manually collected, large amounts of real weld structured light images for model training. This process is limited by the difficulty in accurately measuring the cross-sectional shape of real welds, leading to significant annotation errors in the training data and thus restricting further improvements in the recognition accuracy of deep learning models. Simultaneously, the image acquisition process is susceptible to interference from welding sparks, reflections, and background stray light; image noise significantly interferes with the recognition and localization of weld structures, posing a challenge to the model's robustness.
[0005] In recent years, some studies have begun to introduce deep learning methods such as convolutional neural networks (CNNs) into the weld image recognition process, achieving some progress. However, these methods are often limited to specific types of weld scenes, and obtaining high-quality training samples is costly and limited in quantity, so the generalization ability and adaptability of the models still need to be improved. At the same time, existing neural networks mostly target the extraction of weld feature points or center lines, making it difficult to directly reconstruct or measure the complete cross-sectional shape of the weld.
[0006] To address the aforementioned issues, there is an urgent need for a weld image localization method that is adaptable to various weld structures, robust, and capable of direct cross-sectional identification, while also possessing high-precision and high-efficiency structured light image analysis capabilities. Summary of the Invention
[0007] To address the problems of poor adaptability to different weld types, difficulty in obtaining the true morphology of weld cross-sections, susceptibility to interference in image recognition, and complex recognition processes in existing methods, this invention provides a multi-type weld structured light localization method based on image synthesis. This method automatically synthesizes weld structured light images by constructing a virtual simulation environment and trains a convolutional neural network to achieve direct recognition and high-precision localization of complex weld cross-sections, effectively overcoming the bottlenecks of existing technologies in terms of adaptability, accuracy, and robustness.
[0008] The objective of this invention is achieved through the following technical solution:
[0009] A structured light localization method for multiple types of weld seams based on image synthesis includes the following steps:
[0010] Step S1: Forward synthesis of structured light image of weld seam:
[0011] Step S11: Generation of weld cross-section profile:
[0012] The weld morphology is described by a one-dimensional function of the cross-section, i.e. , The expression for the cross-sectional function is... The plane is the plane containing the cross-section of the weld. The direction is parallel to the welding plane. The direction is perpendicular to the welding plane;
[0013] Step S12: Determine the position of the structured light stripes using the extrinsic parameter matrix of the structured light probe.
[0014] Using structured light stripe point sets Indicates structured light stripes. Represents the i-th structured light stripe. , Indicates the number of sampling points, three-dimensional coordinates The position of the structured light stripes in the ground coordinate system;
[0015] The extrinsic parameters of the camera coordinate system include the rotation matrix relative to the ground coordinate system. and displacement vector The position of the structured light fringe point set in the camera coordinate system is represented as: ,in: ;
[0016] Let the position of the laser in the camera coordinate system be... The normal vector of the ray plane is ,but The following constraint equations must be satisfied:
[0017]
[0018] therefore, Satisfy the following equation:
[0019]
[0020] Step S13: Render the structured light image captured using the intrinsic parameter matrix of the structured light probe:
[0021] First, based on the structured light point set obtained in step S12... Calculate the position of the weld in the camera coordinate system ;
[0022] The pixel coordinate sequence of the structured light fringes in the photograph is then determined using the camera's intrinsic parameter matrix. ,in:
[0023]
[0024] in, Indicates the camera's focal length. , They represent the unit pixel in , Width in direction is the homogenization coefficient, used to adjust the third component of the vector on the right side of the equation to 1;
[0025] Assuming the intensity of the structured light stripes follows a two-dimensional Gaussian distribution, with its center located at... Then the image grayscale value Calculate using the following formula:
[0026]
[0027] in, Indicates the peak intensity of the light spot; The standard deviation of the control spot width;
[0028] To approximate real imaging conditions, the following three types of noise are added:
[0029] (1) Gaussian noise:
[0030]
[0031] (2) Salt and pepper noise:
[0032]
[0033] (3) Simulation of spark spatter rays
[0034] The pixel trajectory of each spark line segment is:
[0035]
[0036] in: The center point of the weld; The direction of firing; Control the spark length;
[0037] By adding lines of different directions and lengths, a welding spark effect diagram that conforms to actual visual observation is generated. The final structured light image is as follows:
[0038]
[0039] Step S2: Reverse solution of weld morphology features:
[0040] Step S21: The CNN network predicts the weld morphology.
[0041] (1) Image column Location code after broadcast By splicing along the channel dimension, the following is formed:
[0042]
[0043] (2) Input the CNN convolutional layer and perform joint feature extraction:
[0044]
[0045] in, This represents a one-dimensional convolutional layer. Indicates the activation layer. Indicates a fully connected layer;
[0046] Step S22: Calculate the weld morphology error loss function:
[0047] Using predicted location With real location The mean squared error is used as the loss function :
[0048]
[0049] Add surface continuity loss function :
[0050]
[0051] Final loss For weighted sum:
[0052]
[0053] In the formula, These are weighting coefficients;
[0054] Step S23: Iterate repeatedly until the model converges:
[0055] The loss function is optimized using gradient descent with an optimizer. , Convergence signifies the end of training. After training, this CNN network can be used to directly predict the positions of points on actual structured light images. This allows for the reconstruction of the weld morphology.
[0056] Compared with the prior art, the present invention has the following advantages:
[0057] 1. This invention automatically generates multi-type weld structured light image data by constructing a simulation environment, which helps to improve the model's adaptability and generalization ability to different weld shapes.
[0058] 2. The present invention utilizes the known and accurate cross-sectional geometric information of the weld seam image generated by simulation, without the need to measure the shape of the real workpiece, thereby improving the quality and annotation accuracy of the training sample and thus improving the overall weld seam positioning accuracy.
[0059] 3. This invention introduces various common noise interferences (Gaussian noise, salt and pepper noise, spark sputtering) in the structured light image simulation process, which improves the robustness and anti-interference ability of the recognition model in complex environments.
[0060] 4. This invention uses a convolutional neural network to directly predict the cross-sectional shape of the weld, simplifying the traditional identification process, avoiding the accumulation of errors in intermediate steps, and improving the efficiency and accuracy of weld positioning. Attached Figure Description
[0061] Figure 1 This is a flowchart of a multi-type weld structured light positioning method based on image synthesis;
[0062] Figure 2 This is a schematic diagram of the coordinate system between the weld and the probe.
[0063] Figure 3 Generate an image of the structured light pattern for a V-shaped weld.
[0064] Figure 4 The spatial distribution of structured light point set obtained by processing V-shaped weld seam images through a CNN neural network. Detailed Implementation
[0065] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.
[0066] This invention provides a multi-type weld structured light localization method based on image synthesis. To achieve weld morphology detection from weld structured light images, the method designs two modules: forward synthesis of weld structured light images and reverse solution of weld morphology features. The specific process is as follows: Figure 1 As shown, it includes the following steps:
[0067] Step S1: Forward synthesis of structured light image of weld seam:
[0068] The purpose of this step is to simulate the stripe pattern formed when structured light shines on the weld. The following descriptions are in superscript. Representing the ground coordinate system, use superscript. Represents the camera coordinate system. Figure 2 This is a schematic diagram showing the coordinate system of the ground where the weld is located and the coordinate system of the camera where the probe is located.
[0069] Step S11: Generation of weld cross-section profile:
[0070] The weld morphology can be described by a one-dimensional function of the cross-section, i.e. , The plane is the plane containing the cross-section of the weld. The direction is parallel to the welding plane. The direction is perpendicular to the welding plane. The examples provide cross-sectional functions for different types of welds. expression.
[0071] Step S12: Determine the position of the structured light stripes using the extrinsic parameter matrix of the structured light probe.
[0072] When structured light shines on the weld, it produces a continuous spatial stripe. The spatial distribution of this stripe is determined by the weld cross-sectional shape and the spatial position of the probe laser. This step aims to obtain the spatial position of the structured light stripe through the cross-sectional shape function and the structured light probe extrinsic matrix.
[0073] This invention employs structured light stripe point sets Indicates structured light stripes. Represents the i-th structured light point. , Indicates the number of sampling points, three-dimensional coordinates This represents the position of the structured light stripes in the ground coordinate system. (mm), This indicates the width of the spatial weld (mm), which can be adjusted according to specific circumstances. , The size and position of the stripes satisfy the weld section equation: .
[0074] The following calculations and solutions are performed. :
[0075] The extrinsic parameters of the camera coordinate system include the rotation matrix relative to the ground coordinate system. and displacement vector Then the position of the structured light fringe point set in the camera coordinate system can be represented as ,in: .
[0076] The laser is a linear light source, and the light it emits falls inside a fixed plane. Let the position of the laser in the camera coordinate system be denoted as . The normal vector of the ray plane is ( Determined by the probe structure (and is a fixed value), then The following constraint equations need to be satisfied:
[0077]
[0078] therefore, The following equation needs to be satisfied, and can be solved to obtain... :
[0079]
[0080] Step S13: Render the structured light image captured using the intrinsic parameter matrix of the structured light probe:
[0081] After generating the spatial location of the weld, the structured light image captured by the camera needs to be rendered. First, based on the structured light point set obtained in step S12... Calculate the position of the weld in the camera coordinate system ,Right now: .
[0082] The pixel coordinate sequence of the structured light fringes in the photograph is then determined using the camera's intrinsic parameter matrix. ,in:
[0083]
[0084] in, Indicates the camera's focal length. , They represent the unit pixel in , Width in direction is the homogenization coefficient, used to adjust the third component of the vector on the right side of the equation to 1.
[0085] To simulate a realistic weld image, the pixel coordinate sequence of structured light stripes in the photograph must be calculated. Surrounding the beams are light spot diffusion effects and image noise interference. It is assumed that the intensity of the structured light stripes follows a two-dimensional Gaussian distribution, with its center located at... Then the image grayscale value It can be calculated using the following formula:
[0086]
[0087] in, Indicates the peak intensity of the light spot; The standard deviation controlling the spot width depends on the laser divergence angle and camera resolution. To approximate realistic imaging conditions, the following three types of noise are added:
[0088] (1) Gaussian noise (image sensor noise):
[0089]
[0090] It follows a normal distribution. denoted as the standard deviation of Gaussian noise.
[0091] (2) Salt and pepper noise (simulating lens dust / interference):
[0092]
[0093] This indicates the probability of white spot noise occurring. This indicates the probability of black dot noise appearing.
[0094] (3) Simulation of spark spatter rays
[0095] The welding process generates bright, highly directional sparks. This invention simulates several radially diverging bright line segments added to the structured light image. Each spark segment is defined as a starting point. and direction angle ,length The pixel trajectory is as follows:
[0096]
[0097] in: The center point of the weld (e.g., the bottom of a V-shaped valley); For the direction of launch, obey Uniform distribution between them; Control the spark length, obey The uniform distribution of lines allows for numerical adjustments based on camera resolution. By adding multiple lines of different directions and lengths, a welding spark effect image that accurately reflects actual visual observation can be generated. This enhances the diversity and authenticity of data used in network training.
[0098] The final structured light image is as follows:
[0099]
[0100] Figure 3 A structured light effect image generated using the method described above is shown.
[0101] Step S2: Reverse solution of weld morphology features:
[0102] Step S21: The CNN network predicts the weld morphology.
[0103] Because structured light stripes are along in the image Extending along the axial direction, the network is a longitudinal slice of each structured light point in the image (i.e., with... Using the image column centered at the point as input, vertical features are extracted to predict the 3D coordinates of that point in the camera coordinate system.
[0104] Considering the significant influence of the horizontal position of structured light stripes in an image, this invention introduces positional encoding into the network input. The horizontal position of each stripe point is encoded as a vector and fused with the vector through joint convolution with image column features, thereby enhancing the network's spatial awareness.
[0105] Network input includes and ,in It is an image China and Israel The x-axis represents the entire column of pixels. This is the image length. Output the predicted 3D coordinates of this point in the camera coordinate system. .
[0106] The present invention adopts the following network structure design:
[0107] (1) Image column Location code after broadcast By splicing along the channel dimension, the following is formed:
[0108]
[0109] (2) Input the CNN convolutional layer and perform joint feature extraction:
[0110]
[0111] in, This represents a one-dimensional convolutional layer. Indicates the activation layer. This indicates a fully connected layer.
[0112] Step S22: Calculate the weld morphology error loss function:
[0113] Using predicted location With real location The mean squared error is used as the loss function:
[0114]
[0115] Add a surface continuity loss function:
[0116]
[0117] The final loss is the weighted sum:
[0118]
[0119] In the formula, These are weighting coefficients used to control the proportion of continuous loss in the overall loss function.
[0120] Step S23: Iterate repeatedly until the model converges:
[0121] We employ optimizers such as Adam for conventional gradient descent optimization. Convergence signifies the end of training. After training, the network can directly predict point positions on actual structured light images. This allows for the reconstruction of the weld morphology.
[0122] Example:
[0123] A welding task requires welding positioning for various welds, including V-shaped welds and stepped welds. Therefore, in step S11, different weld section functions are constructed. (V-shaped weld and stepped weld section morphology functions are then described.) It can be represented as:
[0124]
[0125] in, Indicates the slope of the V-shaped weld. Indicates the height of the stepped weld. This represents the width of the stepped weld, and floor() is the floor function that rounds down.
[0126] The probe will be in different poses during operation, so in step S12, the rotation matrix of the probe relative to the ground coordinate system needs to be randomly initialized according to the actual working conditions. and displacement vector To simulate working conditions in industrial settings and improve the robustness of the neural network, probe extrinsic parameters were adjusted. Sampling must be performed within a specified range, such as a rotation range of ±30° and a distance range not exceeding 100 mm. Furthermore, the specific requirements depend on the structured light probe model. Parameters. The remaining steps are consistent with the invention.
[0127] By processing a single image of structured light stripes captured by a camera using a trained CNN neural network, the corresponding set of structured light points can be predicted. , Figure 4 The spatial distribution of structured light point sets obtained by processing a V-shaped weld image through a CNN neural network is shown.
Claims
1. A structured light positioning method for multiple types of weld seams based on image synthesis, characterized in that... The method includes the following steps: Step S1: Forward synthesis of structured light image of weld seam: Step S11: Generate the weld cross-sectional shape; Step S12: Determine the position of the structured light stripes using the extrinsic parameter matrix of the structured light probe; Step S13: Render the captured structured light image using the intrinsic parameter matrix of the structured light probe; Step S2: Reverse solution of weld morphology features: Step S21: The CNN network predicts the weld morphology; Step S22: Calculate the weld morphology error loss function; Step S23: Iterate repeatedly until the model converges.
2. The multi-type weld seam structured light positioning method based on image synthesis according to claim 1, characterized in that... In step S11, the weld morphology is described by a one-dimensional function of the cross-section, i.e. , The expression for the cross-sectional function is... The plane is the plane containing the cross-section of the weld. The direction is parallel to the welding plane. The direction is perpendicular to the welding plane.
3. The multi-type weld seam structured light positioning method based on image synthesis according to claim 2, characterized in that... When the weld morphology is a V-shaped weld or a stepped weld, the cross-sectional morphology function Represented as: in, Indicates the slope of the V-shaped weld. Indicates the height of the stepped weld. This represents the width of the stepped weld, and floor() is the floor function that rounds down.
4. The multi-type weld seam structured light positioning method based on image synthesis according to claim 1, characterized in that... The specific steps of step S12 are as follows: Using structured light stripe point sets Indicates structured light stripes. Represents the i-th structured light stripe. , Indicates the number of sampling points, three-dimensional coordinates The position of the structured light stripes in the ground coordinate system; The extrinsic parameters of the camera coordinate system include the rotation matrix relative to the ground coordinate system. and displacement vector The position of the structured light fringes in the camera coordinate system is represented as ,in: ; Let the position of the laser in the camera coordinate system be... The normal vector of the ray plane is ,but The following constraint equations must be satisfied: therefore, Satisfy the following equation: 。 5. The multi-type weld seam structured light positioning method based on image synthesis according to claim 4, characterized in that... The , Indicates the width of the spatial weld.
6. The multi-type weld seam structured light positioning method based on image synthesis according to claim 4, characterized in that... The specific steps of step S13 are as follows: First, based on the structured light point set obtained in step S12... Calculate the position of the weld in the camera coordinate system ; The pixel coordinate sequence of the structured light fringes in the photograph is then determined using the camera's intrinsic parameter matrix. ,in: in, Indicates the camera's focal length. , They represent the unit pixel in , Width in direction is the homogenization coefficient, used to adjust the third component of the vector on the right side of the equation to 1; Assuming the intensity of the structured light stripes follows a two-dimensional Gaussian distribution, with its center located at... Then the image grayscale value Calculate using the following formula: in, Indicates the peak intensity of the light spot; The standard deviation of the control spot width; To approximate real imaging conditions, the following three types of noise are added: (1) Gaussian noise: In the formula, It follows a normal distribution. The standard deviation of Gaussian noise; (2) Salt and pepper noise: In the formula, This indicates the probability of white spot noise occurring. This indicates the probability of black dot noise occurring. (3) Simulation of spark spatter rays The pixel trajectory of each spark line segment is: in: The center point of the weld; The direction of the shot. Control the spark length; By adding lines of different directions and lengths, a welding spark effect diagram that conforms to actual visual observation is generated. The final structured light image is as follows: 。 7. The multi-type weld seam structured light positioning method based on image synthesis according to claim 6, characterized in that... The obey Uniform distribution between them; Control and obedience The uniform distribution between them.
8. The multi-type weld seam structured light positioning method based on image synthesis according to claim 1, characterized in that... The specific steps of step S21 are as follows: (1) Image column Location code after broadcast By splicing along the channel dimension, a structure is formed. ; (2) Input the CNN convolutional layer and perform joint feature extraction: in, This represents a one-dimensional convolutional layer. Indicates the activation layer. This indicates a fully connected layer.
9. The multi-type weld seam structured light positioning method based on image synthesis according to claim 1, characterized in that... The specific steps of step S22 are as follows: Using predicted location With real location The mean squared error is used as the loss function : Add surface continuity loss function : Final loss For weighted sum: In the formula, These are the weighting coefficients.
10. The multi-type weld seam structured light positioning method based on image synthesis according to claim 9, characterized in that... The specific steps of step S23 are as follows: The loss function is optimized using gradient descent with an optimizer. , Convergence signifies the end of training. After training, the CNN network can be used to directly predict the positions of points on actual structured light images. This allows for the reconstruction of the weld morphology.