Calibration-free welding method, device and equipment based on molten pool camera recognition

Abstract

The invention relates to the technical field of welding, and discloses a calibration-free welding method, device and equipment based on molten pool camera recognition, a welding robot drives a welding wire end point on a welding gun to move along a welding bead, a molten pool camera is installed on the welding gun, and the welding wire end point and the welding bead are within the view field of the molten pool camera; the method comprises the steps that a real-time image of a molten pool camera is obtained, and the welding bead center position and the welding wire end point position in the image are recognized according to the real-time image; based on the welding wire endpoint position and the welding bead center position, the relative position of the welding wire endpoint and the welding bead center position is determined; and the real-time track deviation of the welding wire motion track is determined based on the relative position, and the welding wire endpoint is controlled to move for welding according to the real-time track deviation. The welding bead center position and the welding wire end point position are visually recognized, real-time deviation correction is carried out based on the deviation of the two positions, the welding precision and stability are improved, and the consistency of the welding seam quality is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of welding technology, and specifically to a calibration-free welding method, apparatus, and equipment based on molten pool camera recognition. Background Technology

[0002] Welding is widely used in shipbuilding, bridge engineering, heavy machinery, and pressure vessels. For thick plate welding with complex weld bead morphology, relying on traditional manual teaching is time-consuming and labor-intensive. Therefore, many welding equipment systems now incorporate vision technology to assist manual teaching. However, a major problem with vision welding is that the accuracy of camera-guided welding torch welding depends on the accuracy of camera calibration. Traditional camera calibration methods typically require complex calibration plates and strict calibration environments, and are easily affected by adverse factors such as high temperatures, arc light interference, and smoke during welding, leading to insufficient calibration accuracy. Furthermore, in practical applications, due to the complex and variable welding scenarios, the camera's installation position and angle may change, resulting in unstable calibration results. This makes it difficult for vision inspection algorithms to maintain high accuracy and stability under complex welding conditions. Summary of the Invention

[0003] In view of this, the present invention provides a calibration-free welding method, apparatus and equipment based on molten pool camera recognition, so as to solve the problem that inaccurate camera calibration affects welding accuracy and stability.

[0004] In a first aspect, the present invention provides a calibration-free welding method based on molten pool camera recognition. A welding robot moves the tip of the welding wire on the welding torch along the weld bead. A molten pool camera is mounted on the welding torch, and the tip of the welding wire and the weld bead are within the field of view of the molten pool camera. The method includes:

[0005] Acquire real-time images from the molten pool camera, and identify the center position of the weld bead and the position of the welding wire tip in the image based on the real-time images;

[0006] The relative position of the welding wire tip and the weld center is determined based on the position of the welding wire tip and the position of the weld center.

[0007] The real-time trajectory deviation of the welding wire is determined based on the relative position, and the welding wire tip movement is controlled according to the real-time trajectory deviation to perform welding.

[0008] The calibration-free welding method based on molten pool camera recognition provided by this invention visually identifies the center position of the weld bead and the end position of the welding wire, and performs real-time correction based on the deviation between the two, eliminating the camera calibration step. This breaks through the dependence of visual inspection on calibration in traditional welding, can adapt to different welding environments and camera installation changes, significantly reduces debugging time and maintenance costs, improves welding accuracy and stability, and ensures the consistency of weld quality.

[0009] In one optional implementation, identifying the center position of the weld bead in the image based on real-time image recognition includes:

[0010] Acquire historical molten pool image data of the target welded object and construct an image dataset;

[0011] The weld bead recognition model was trained and optimized based on the image dataset.

[0012] Weld bead recognition models are used to identify weld beads in real-time images, and the center position of the weld bead is determined based on the identified weld beads.

[0013] The calibration-free welding method based on molten pool camera recognition provided by this invention improves the accuracy of weld bead positioning during the welding process by training and optimizing the weld bead recognition model, ensuring the accuracy of the welding trajectory, reducing manual inspection errors, and enhancing the adaptability to different welding scenarios by training the model based on historical data. This helps to stabilize automated welding, improve welding quality and production efficiency, and promote the advancement of welding technology towards precision and efficiency.

[0014] In one alternative implementation, training and optimizing a weld bead recognition model based on an image dataset includes:

[0015] Image data of molten pools at different times, different weld layers, and different weld passes were selected from the image dataset. Each image data of molten pools was preprocessed to obtain training samples.

[0016] The deep learning network model is trained and validated using training samples to obtain a well-trained weld bead recognition model.

[0017] Real-time images are acquired using a preset fixed window, and the weld bead recognition model is fine-tuned using the real-time images to obtain an optimized weld bead recognition model.

[0018] The calibration-free welding method based on molten pool camera recognition provided by this invention enriches the diversity of training samples and improves the generalization ability of the model by screening and preprocessing molten pool image data from multiple dimensions. After training and verification, a basic model is built to ensure the basic accuracy of weld bead recognition. Then, real-time image fine-tuning is performed using a preset window to adapt the model to the dynamic changes of the actual welding scene. After optimization, the model recognition is more accurate and stable, and can effectively cope with the complex situations of different times, weld layers and weld beads. This lays a solid foundation for the subsequent accurate identification of the weld bead center position and improves welding quality and production efficiency.

[0019] In one optional implementation, determining the relative position of the welding wire tip and the weld bead center position based on the welding wire tip position and the weld bead center position includes:

[0020] Establish an image coordinate system with the weld center position as the x-axis and the direction perpendicular to the weld center position as the y-axis.

[0021] The ordinate value of the wire tip position in the vertical direction is determined based on the wire tip position and the weld center position. This ordinate value is used as the relative position between the wire tip position and the weld center position. The sign of the ordinate value indicates the direction of the wire tip position offset from the weld center position, and the absolute value of the ordinate value indicates the magnitude of the offset of the wire tip position relative to the weld center position.

[0022] In one alternative implementation, the tip of the welding wire is located at the center of the field of view of the molten pool camera.

[0023] The calibration-free welding method based on molten pool camera recognition provided by this invention constructs a dedicated image coordinate system, accurately quantifying the relative position of the welding wire tip and the center of the weld bead through the vertical coordinate value. The positive and negative values ​​clearly indicate the offset direction, and the absolute value reflects the offset magnitude. This provides a clear and quantifiable basis for adjusting welding deviations, clearly defines the relative position, and promptly detects the deviation between the welding wire and the center of the weld bead. This facilitates real-time adjustment of welding parameters and paths, reduces welding deviations, and ensures that the welding wire tip is located at the center of the molten pool camera's field of view, avoiding the influence of edge distortion on detection. This allows for more accurate acquisition of the tip position and enhances the accuracy of position recognition and judgment.

[0024] In one optional implementation, welding is performed by controlling the movement of the welding wire tip based on real-time trajectory deviation, including:

[0025] Determine the direction and amount of deviation based on real-time trajectory deviation;

[0026] If the deviation of the real-time trajectory is greater than the preset deviation threshold, the welding wire tip is controlled to move in the opposite direction to the deviation direction by a preset correction amount.

[0027] If the deviation of the real-time trajectory is not greater than the preset deviation threshold, there is no need to adjust the welding wire movement trajectory.

[0028] The calibration-free welding method based on molten pool camera recognition provided by this invention accurately controls the movement of the welding wire tip by judging the trajectory deviation in real time, and corrects the deviation in a timely manner according to the direction and value of the deviation to ensure the accuracy of the welding trajectory. By setting a deviation threshold, it avoids excessive intervention, improves welding stability, reduces the impact of trajectory deviation on welding, adapts to the needs of automated welding, and enhances the reliability of welding process and the quality of finished products.

[0029] In one alternative implementation, if the center position of the weld bead cannot be identified based on the real-time image, the duration for which the center position of the weld bead cannot be identified is recorded.

[0030] If the duration exceeds the preset time threshold, the welding will be terminated.

[0031] The calibration-free welding method based on molten pool camera recognition provided by this invention records the duration for which the center position of the weld bead cannot be identified, and terminates the welding when the timeout period expires. This avoids empty welding and mis-welding of equipment, reduces invalid welding operations, protects equipment, saves welding materials, avoids uncontrolled welding deviations caused by continuous failure to identify the weld bead, and improves the reliability of automated welding.

[0032] Secondly, the present invention provides a calibration-free welding device based on molten pool camera recognition. A welding robot moves the welding wire tip on the welding torch along the weld bead. A molten pool camera is mounted on the welding torch, and the welding wire tip and weld bead are within the field of view of the molten pool camera. The device includes:

[0033] The image recognition module is used to acquire real-time images from the molten pool camera and identify the center position of the weld bead and the position of the welding wire tip in the image based on the real-time image.

[0034] The relative position determination module is used to determine the relative position between the welding wire tip and the weld center based on the welding wire tip position and the weld center position;

[0035] The real-time welding module is used to determine the real-time trajectory deviation of the welding wire movement trajectory based on the relative position, and to control the movement of the welding wire end point to perform welding based on the real-time trajectory deviation.

[0036] Thirdly, the present invention provides a computer device, including: a human-computer interaction interface, a memory, and a processor. The human-computer interaction interface is used to receive instructions and data input by the user and transmit them to the processor, and to feed back the processing results of the processor to the user.

[0037] The memory and the processor are interconnected and communicate with each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the method described in the first aspect or any of its corresponding embodiments.

[0038] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0039] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is a schematic flowchart of a calibration-free welding method based on molten pool camera recognition according to an embodiment of the present invention;

[0041] Figure 2 This is a schematic flowchart of another calibration-free welding method based on molten pool camera recognition according to an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of establishing an image coordinate system in the calibration-free welding method based on molten pool camera recognition according to an embodiment of the present invention;

[0043] Figure 4 This is a structural block diagram of a calibration-free welding device based on molten pool camera recognition according to an embodiment of the present invention;

[0044] Figure 5 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] This invention provides a calibration-free welding method based on molten pool camera recognition. By visually recognizing the deviation between the center position of the weld bead and the end position of the welding wire, and performing real-time correction, the method eliminates the camera calibration step and improves welding accuracy and stability.

[0047] According to an embodiment of the present invention, a calibration-free welding method based on molten pool camera recognition is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0048] This embodiment provides a calibration-free welding method based on molten pool camera recognition, which can be used in the aforementioned computer system. Figure 1 This is a flowchart of a calibration-free welding method based on a weld pool camera according to an embodiment of the present invention. A welding robot moves the welding wire tip on the welding torch along the weld bead. A weld pool camera is mounted on the welding torch, and the welding wire tip and weld bead are within the field of view of the weld pool camera. Figure 1 As shown, the process includes the following steps:

[0049] Step S101: Obtain real-time images from the molten pool camera, and identify the center position of the weld bead and the end position of the welding wire in the image based on the real-time images.

[0050] Specifically, this embodiment is applicable to application scenarios such as thick plate welding, thin plate welding, and pipe welding.

[0051] The equipment and related requirements for implementing this embodiment include: an industrial welding power supply with a standard RS232 / RS485 or Ethernet communication interface, supporting parameter output and real-time control; a six-axis industrial robot equipped with a high-precision position sensor and joint angle encoder, supporting teaching and closed-loop control, with a communication interface using TCP / IP or CAN bus; a dedicated industrial camera for observing the molten pool, using an industrial camera interface standard based on Gigabit Ethernet technology (Gigabit Ethernet, GigE) or a USB 3.0 interface, with a resolution of not less than 1280×1024 and a continuous acquisition frame rate set to 30fps; and an industrial control computer (IPC) pre-installed with dedicated control software, image acquisition and processing programs, and an operating system using Windows or Linux.

[0052] The industrial welding power supply is connected to the industrial robot control system via a dedicated data interface. Using the industrial robot's teach pendant, the communication parameters of the industrial welding power supply (including but not limited to: baud rate, data format, fixed data packet header, and cyclic redundancy check) are rigorously set and verified to ensure accurate data transmission. A welding torch is installed at the flange position of the industrial robot, and a molten pool camera is fixedly mounted on the torch, ensuring that the center of the camera's field of view coincides with the tip of the welding wire as much as possible.

[0053] The industrial robot is stably connected to the industrial control computer via TCP / IP or CAN bus, enabling bidirectional communication. The robot system pre-records the position of the welding torch end effector, tool offset, and calibration parameters to ensure precise correspondence between the welding torch end position (welding wire tip position) and the actual coordinates. The weld pool camera is directly connected to the industrial control computer via a GigE / USB 3.0 interface, controlled by a dedicated driver and software development kit (SDK), with a resolution set to 1280×1024 and continuous acquisition at 30fps.

[0054] The welding process is captured by a molten pool camera, which captures real-time images of the weld center and the wire tip. There are no obstructions within the camera's field of view. During normal welding, the wire tip should move along the weld center. To ensure the weld quality after welding, the trajectory deviation of the wire tip needs to be controlled. Therefore, to monitor the trajectory deviation of the wire tip, the weld center and the wire tip position need to be identified based on the real-time images.

[0055] Step S102: Determine the relative position between the welding wire tip and the weld center position based on the welding wire tip position and the weld center position.

[0056] Specifically, the positions of the welding wire tip and the weld center, identified in the real-time image, are placed in the same two-dimensional coordinate system. An image coordinate system can be established based on the camera's image coordinates. When identifying the positions of the welding wire tip and the weld center, the coordinates A(x1, y1) and B(x2, y2) of the welding wire tip and the weld center in the image coordinate system are used as the corresponding positions. The weld center position includes multiple weld center points that make up the welding trajectory; this is just an example and not a limitation. By comparing the difference between the two coordinates, CHA(x1-x2, y1-y2), the relative position of the welding wire tip and the weld center is determined.

[0057] It should be noted that before welding begins, the operator moves the welding wire tip to the starting point of the welding trajectory by operating the industrial robot and determines the direction of movement of the welding wire tip through instructions. When welding begins, the welding wire tip moves and welds according to the direction of movement. During the welding process, the center position of the weld bead is identified by the molten pool camera, so that the welding wire tip welds along the weld bead. Therefore, during the welding process, it is only necessary to determine the relative position of the welding wire tip and the center position of the weld bead in the two-dimensional image.

[0058] Step S103: Determine the real-time trajectory deviation of the welding wire movement trajectory based on the relative position, and control the movement of the welding wire end point to perform welding according to the real-time trajectory deviation.

[0059] Specifically, the real-time trajectory deviation of the welding wire movement trajectory is determined based on the relative position. In this embodiment, the welding wire tip is assumed to maintain a constant direction of movement along the center of the weld bead. Only the lateral deviation of the welding wire tip relative to the center of the weld bead needs to be adjusted in real time. The real-time trajectory deviation is usually a positive or negative number, where positive and negative indicate the direction of deviation and magnitude indicates the distance of deviation. The welding wire tip is adjusted according to the real-time trajectory deviation. By controlling the movement of the industrial robot, the position of the welding wire tip is adjusted so that the deviation of the center position of the weld bead is within the preset deviation range, thereby ensuring the quality of the weld.

[0060] The calibration-free welding method based on molten pool camera recognition provided in this embodiment visually identifies the center position of the weld bead and the end position of the welding wire, and performs real-time correction based on the deviation between the two. This eliminates the need for camera calibration, breaks through the dependence of visual inspection on calibration in traditional welding, can adapt to different welding environments and camera installation changes, significantly reduces debugging time and maintenance costs, improves welding accuracy and stability, and ensures the consistency of weld quality.

[0061] This embodiment provides a calibration-free welding method based on molten pool camera recognition, which can be used in the aforementioned computer system. Figure 2This is a flowchart of a calibration-free welding method based on a weld pool camera according to an embodiment of the present invention. A welding robot moves the welding wire tip on the welding torch along the weld bead. A weld pool camera is mounted on the welding torch, and the welding wire tip and weld bead are within the field of view of the weld pool camera. Figure 2 As shown, the process includes the following steps:

[0062] Step S201: Obtain real-time images from the molten pool camera, and identify the center position of the weld bead and the position of the welding wire tip in the image based on the real-time images.

[0063] Specifically, step S201 includes:

[0064] Step S2011: Obtain historical molten pool image data of the target welding object and construct an image dataset.

[0065] Specifically, a camera driver was independently developed to control image acquisition parameters, enabling continuous image acquisition, preprocessing (using median filtering, histogram equalization, and geometric correction), and real-time storage. Based on the image acquisition parameters, molten pool image data for various weld types in steel structures, including H-beams, was accurately acquired. Each image was accompanied by a precise timestamp, welding machine status, and robot motion data.

[0066] Based on the calibrated coordinate system, key features in the image, such as the molten pool boundary (referring to the outline of the molten pool during the welding process, i.e. the boundary between the molten and unmolten areas, which can be identified in real time through image processing technology for evaluating welding quality) and the weld center, are strictly labeled. The labeled data is uniformly saved in XML or JSON file format, and corresponds strictly one-to-one with the collected data.

[0067] Historical molten pool image data containing the weld center and / or wire tip are uniformly saved to construct an image dataset for training and optimizing the weld recognition model and / or wire tip recognition model. It should be noted that the training and optimization process of the weld recognition model and the wire tip recognition model is exactly the same, the only difference being the recognition target. In this embodiment, the training and optimization of the weld recognition model is used as an example, and the training and optimization process of the wire tip recognition model will not be described in detail.

[0068] Step S2012: Train and optimize the weld bead recognition model based on the image dataset.

[0069] Specifically, based on an image dataset containing weld seams, a weld seam recognition model is designed and trained using a convolutional neural network target detection algorithm (such as YOLOv3, Faster R-CNN, or U-Net).

[0070] Step S2013: Use the weld bead recognition model to identify weld beads in the real-time image and determine the center position of the weld bead based on the identified weld beads.

[0071] Specifically, the real-time image undergoes grayscale conversion, Gaussian filtering for noise reduction, and histogram equalization to enhance contrast. The preprocessed image is then input into a trained convolutional neural network, which extracts features such as weld edges and textures through multiple convolutional and pooling operations to generate feature maps.

[0072] The model outputs weld candidate regions based on feature maps, eliminates redundant boxes through non-maximum suppression, and determines the final weld boundary boxes. For segmentation models such as U-Net, pixel-level weld masks are directly generated, and boundary point sets are extracted through contour detection algorithms.

[0073] The center position of the weld can be determined by methods such as geometric center method, morphological thinning, and ellipse fitting. Using geometric center method, the centroid coordinates (x0, y0) of the bounding box or contour point set are calculated as the center position. Using morphological thinning, the skeleton of the binarized weld region is extracted, and the center point is calculated after obtaining the central axis. Using ellipse fitting, the boundary points are fitted into an ellipse by the least squares method, and the center of the ellipse is used as the weld center.

[0074] A Kalman filter is used to predict and correct the center position of consecutive frames, eliminating position jitter caused by molten pool fluctuations or arc light interference, and outputting a smooth trajectory of the weld bead center position.

[0075] The calibration-free welding method based on molten pool camera recognition provided in this embodiment improves the accuracy of weld bead positioning during welding by training and optimizing the weld bead recognition model, ensuring the accuracy of welding trajectory, reducing manual inspection errors, and enhancing the adaptability to different welding scenarios by training the model based on historical data. This helps stabilize automated welding, improves welding quality and production efficiency, and promotes the advancement of welding technology towards precision and efficiency.

[0076] In some optional implementations, step S2012 above includes:

[0077] Step a1: Select molten pool image data from different times, different weld layers, and different weld passes from the image dataset, and preprocess each molten pool image data to obtain training samples.

[0078] In this embodiment, the U-Net deep learning network model is used as an example for illustration. U-Net is a classic image segmentation network with an encoder-decoder structure, which is very suitable for pixel-level weld line recognition tasks.

[0079] Based on the sequence of molten pool images (and image dataset) acquired during the welding process, molten pool image data from different times, different weld layers, and different weld passes were selected to form multiple training samples. For each molten pool image, the weld area was manually labeled using the LabelMe tool in a segmented mask format. The centerline coordinates were used as auxiliary data. Data augmentation was performed on the labeled images by simulating actual disturbances through random rotation, cropping, flipping, and lighting changes. The preprocessed images yielded training samples for model training.

[0080] Step a2: Use training samples to train and validate the deep learning network model to obtain a trained weld bead recognition model.

[0081] Specifically, the training samples are proportionally divided into a training set (80%) and a validation set (20%). The training and validation sets are loaded, and the deep learning network model is trained using the training set and validated using the validation set. A uniform input size (e.g., 640×480) and fixed training parameters (initial learning rate 0.001, batch size 32, and processing the entire training dataset once for every 200 samples) ensure that the model's accuracy in identifying the molten pool boundary and weld center reaches or exceeds 95%, and the positional information regression error is controlled within ±5%.

[0082] After each training round, metrics such as Intersection over Union (IoU), Precision, Recall, and weld centerline deviation are evaluated. Early stopping and a learning rate scheduler can be used to prevent overfitting. U-Net outputs a weld area mask. The weld centerline is further extracted through image processing of the weld area, then smoothed, and continuous coordinates are extracted for welding trajectory generation. The specific training process is a mature existing technique and will not be elaborated here.

[0083] Step a3: Real-time images are acquired using a preset fixed window, and the weld bead recognition model is fine-tuned using the real-time images to obtain an optimized weld bead recognition model.

[0084] Specifically, the trained model is compiled and deployed to an industrial control computer. Real-time image data is collected at a fixed window of 30 frames per second, and the model is fine-tuned using the real-time image data to optimize the model and ensure that the weld bead recognition model can adapt to actual working conditions in the long term.

[0085] The calibration-free welding method based on molten pool camera recognition provided in this embodiment enriches the diversity of training samples and improves the generalization ability of the model by screening and preprocessing molten pool image data from multiple dimensions. After training and verification, a basic model is built to ensure the basic accuracy of weld bead recognition. Then, real-time image fine-tuning is performed using a preset window to adapt the model to the dynamic changes of the actual welding scene. After optimization, the model recognition is more accurate and stable, and can effectively cope with the complex situations of different times, weld layers, and weld beads. This lays a solid foundation for the subsequent accurate identification of the weld bead center position and improves welding quality and production efficiency.

[0086] Step S202: Determine the relative position between the welding wire tip and the weld center position based on the welding wire tip position and the weld center position.

[0087] Specifically, step S202 includes:

[0088] Step S2021: Establish an image coordinate system with the center position of the weld bead as the abscissa and the direction perpendicular to the center position of the weld bead as the ordinate.

[0089] Specifically, such as Figure 3 The image shown is a schematic diagram of the molten pool during the real-time acquisition of images. The target object being welded is an L-shaped workpiece. Taking one side of the L-shaped workpiece as an example, an image coordinate system is established with the center of the weld bead as the x-axis and the direction perpendicular to the center of the weld bead as the y-axis. The origin of the image coordinate system is not restricted and can be set at any point. Figure 3 The welding start point O is shown.

[0090] Step S2022: Determine the ordinate value of the welding wire tip position in the vertical direction based on the welding wire tip position and the weld center position. This ordinate value is used as the relative position between the welding wire tip position and the weld center position. The sign of the ordinate value indicates the offset direction of the welding wire tip position relative to the weld center position, and the absolute value of the ordinate value indicates the magnitude of the offset of the welding wire tip position relative to the weld center position.

[0091] Specifically, the relative position of the welding wire tip and the weld center is the difference between their coordinates: BA = (x2 - x1, y2 - y1). In this embodiment, the weld center is on the x-axis, so its ordinate is always 0. The ordinate value y1 of the welding wire tip position in the ordinate direction represents the relative position of the welding wire tip and the weld center. When the ordinate is positive, it indicates that the welding wire tip position is biased towards the positive y-axis relative to the weld center, with the welding trajectory direction as positive, meaning the welding wire tip is biased to the left. When the ordinate is negative, it indicates that the welding wire tip position is biased towards the negative y-axis relative to the weld center, meaning the welding wire tip is biased to the right. The absolute value of the ordinate of the welding wire tip position represents the magnitude of the offset relative to the weld center. It should be noted that in the image coordinate system, pixels are usually used as the unit of measurement. For example, a welding wire tip position of -10 indicates that the welding wire tip is biased to the left by 10 pixels. This is just an example and is not a limitation.

[0092] In some alternative implementations, the tip of the welding wire is located at the center of the field of view of the molten pool camera.

[0093] The calibration-free welding method based on molten pool camera recognition provided in this embodiment constructs a dedicated image coordinate system. The relative relationship between the welding wire tip and the center of the weld bead is precisely quantified through the vertical coordinate value. The positive and negative values ​​clearly indicate the offset direction, and the absolute value reflects the offset magnitude. This provides a clear and quantifiable basis for adjusting welding deviations, clarifies the relative position, and allows for timely detection of the deviation between the welding wire and the center of the weld bead. This facilitates real-time adjustment of welding parameters and paths, reduces welding deviations, and ensures that the welding wire tip is located at the center of the molten pool camera's field of view. This avoids the influence of edge distortion on detection, more accurately obtains the tip position, and enhances the accuracy of position recognition and judgment.

[0094] Step S203: Determine the real-time trajectory deviation of the welding wire movement trajectory based on the relative position, and control the movement of the welding wire end point to perform welding according to the real-time trajectory deviation.

[0095] Specifically, step S203 includes:

[0096] Step S2031: Determine the direction and amount of deviation based on the real-time trajectory deviation.

[0097] Specifically, in the real-time image, the real-time relative position between the weld bead center and the welding wire tip is the real-time trajectory deviation of the welding wire. For example, if the real-time relative position is (0, y2-y1), then the real-time trajectory deviation of the welding wire is y2-y1. The sign of the actual trajectory deviation indicates the direction of the deviation. Taking the direction of the welding wire's trajectory as forward, when the real-time trajectory deviation y2-y1>0, the welding wire moves to the left; when the real-time trajectory deviation y2-y1<0, the welding wire moves to the right. The magnitude of the actual trajectory deviation is the deviation amount.

[0098] Step S2032: If the deviation of the real-time trajectory is greater than the preset deviation threshold, the welding wire tip is controlled to move in the opposite direction to the deviation direction by a preset correction amount.

[0099] Step S2033: If the deviation of the real-time trajectory is not greater than the preset deviation threshold, then there is no need to adjust the welding wire movement trajectory.

[0100] Specifically, a preset deviation threshold is set according to the welding process requirements. Adjustment of the welding trajectory is triggered only when the real-time trajectory deviation exceeds the preset threshold. For example, if the preset deviation threshold is 3, trajectory adjustment is triggered when the deviation exceeds 3 pixels. An industrial robot interpolation algorithm controls the welding wire tip to move in the opposite direction to the deviation by a preset correction amount. The correction amount is the amount of movement of the welding wire tip in the world coordinate system controlled by the industrial robot. The interpolation algorithm is a mature existing technology and will not be elaborated here. In this embodiment, the unit of real-time trajectory deviation is pixels, and the unit of correction amount is mm. The relationship between the two is related to the relative height between the molten pool camera and the welding wire tip position. The greater the relative height, the greater the actual correction amount corresponding to 1 pixel in the image. For example, taking 3 pixels as approximately equal to 1 mm, if the real-time trajectory deviation is 10 pixels, it means that the welding trajectory is 10 pixels to the left (positive y-axis direction). The industrial robot moves the welding wire tip to the right (negative y-axis direction) by 0.5 mm (relative to 1.5 pixels). After multiple adjustments, after the first correction, the real-time trajectory deviation is again obtained as 8.5 pixels, which is still greater than the preset deviation threshold. Then, the second correction is performed... until the deviation is no more than 3 pixels, and there is no need to adjust the welding wire movement trajectory.

[0101] The calibration-free welding method based on molten pool camera recognition provided in this embodiment accurately controls the movement of the welding wire tip by judging the trajectory deviation in real time. It corrects the deviation in a timely manner according to the direction and value of the deviation, ensuring the accuracy of the welding trajectory. By setting a deviation threshold, it avoids excessive intervention, improves welding stability, reduces the impact of trajectory deviation on welding, adapts to the needs of automated welding, and enhances the reliability of welding process and the quality of finished products.

[0102] In some alternative implementations, if the center position of the weld bead cannot be identified based on the real-time image, the duration for which the center position of the weld bead cannot be identified is recorded.

[0103] If the duration exceeds the preset time threshold, the welding will be terminated.

[0104] Specifically, after the robot completes calibration, the industrial control computer software prompts the operator to select welding process parameters and processing direction. After the user confirms the process, the system issues welding instructions to the welding machine, and the robot starts the welding operation according to the preset trajectory, with all parameters strictly executed according to the process document.

[0105] During welding, the molten pool camera continuously acquires real-time images at 30fps. After preprocessing, it detects changes in the weld position (welding trajectory deviation) in real time. The control software uses a PID control algorithm to instantly generate robot fine-tuning instructions based on the weld offset calculated for each frame of image, and continuously sends control signals to ensure that the welding torch always precisely coincides with the weld.

[0106] If no weld information is detected (the center position of the weld bead cannot be identified) within 500ms (or 15 consecutive frames) in the current welding direction, the system strictly determines that the weld area has ended and immediately issues a termination command to end the current welding process.

[0107] The calibration-free welding method based on molten pool camera recognition provided in this embodiment records the duration for which the center position of the weld bead cannot be identified, and terminates the welding when the timeout period expires. This avoids empty welding and mis-welding, reduces invalid welding operations, protects equipment, saves welding materials, avoids uncontrolled welding deviations caused by the continuous inability to identify the weld bead, and improves the reliability of automated welding.

[0108] This embodiment also provides a calibration-free welding device based on molten pool camera recognition. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0109] This embodiment provides a calibration-free welding device based on a weld pool camera. A welding robot moves the welding wire tip on the welding torch along the weld bead. The weld pool camera is mounted on the welding torch, and the welding wire tip and weld bead are within the field of view of the weld pool camera. Figure 4 As shown, it includes:

[0110] The image recognition module 401 is used to acquire real-time images from the molten pool camera and identify the center position of the weld bead and the end position of the welding wire in the image based on the real-time image.

[0111] The relative position determination module 402 is used to determine the relative position between the welding wire tip and the weld center position based on the welding wire tip position and the weld center position.

[0112] The real-time welding module 403 is used to determine the real-time trajectory deviation of the welding wire movement trajectory based on the relative position, and to control the movement of the welding wire end point to perform welding based on the real-time trajectory deviation.

[0113] In some alternative implementations, the image recognition module 401 includes:

[0114] The sample construction unit is used to acquire historical molten pool image data of the target welded object and construct an image dataset.

[0115] The model training and optimization unit is used to train and optimize the weld recognition model based on the image dataset.

[0116] The position recognition unit is used to identify welds in real-time images using a weld recognition model and determine the center position of the welds based on the identified welds.

[0117] In some alternative implementations, the relative position determination module 402 includes:

[0118] The coordinate system establishment unit is used to establish an image coordinate system with the weld center position as the abscissa and the direction perpendicular to the weld center position as the ordinate.

[0119] The offset determination unit is used to determine the ordinate value of the wire tip position in the vertical axis direction based on the wire tip position and the weld center position. This ordinate value is used as the relative position between the wire tip position and the weld center position. The sign of the ordinate value indicates the offset direction of the wire tip position relative to the weld center position, and the absolute value of the ordinate value indicates the magnitude of the offset of the wire tip position relative to the weld center position.

[0120] In some alternative implementations, the real-time welding module 403 includes:

[0121] The real-time deviation determination unit is used to determine the direction and amount of deviation based on the real-time trajectory deviation.

[0122] The real-time correction unit is used to control the welding wire tip to move in the opposite direction to the deviation direction by a preset correction amount if the deviation of the real-time trajectory is greater than a preset deviation threshold.

[0123] The stable welding unit is used so that if the deviation of the real-time trajectory does not exceed the preset deviation threshold, there is no need to adjust the welding wire movement trajectory.

[0124] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0125] In this embodiment, the calibration-free welding device based on molten pool camera recognition is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0126] This invention also provides a computer device having the above-described features. Figure 4 The image shows a calibration-free welding device based on molten pool camera recognition.

[0127] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 5 As shown, the computer device includes: a human-machine interface, one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components communicate with each other using different buses and can be installed on a common motherboard or in other ways as needed. The human-machine interface is developed based on C++, LabVIEW, or other industrial software platforms, integrating robot control, image acquisition, model calling, and welding process parameter distribution. The interface displays real-time images of the molten pool, weld seam recognition results, the current position and deviation value of the welding torch, and provides a process parameter selection interface (welding current, voltage, wire feed rate, oscillation angle, processing direction). The software internally implements PID or higher-level control algorithms to ensure the robot completes fine-tuning according to preset instructions, ensuring precise alignment of the welding torch and weld seam, and dynamically issuing offset adjustment instructions with a delay of less than 100ms. The processor can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of the GUI on external input / output devices (such as display devices coupled to the interface). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory units, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 5 Take a processor 10 as an example.

[0128] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0129] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0130] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0131] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0132] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0133] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0134] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A calibration-free welding method based on molten pool camera recognition, characterized in that, A welding robot moves the welding wire tip on a welding torch along the weld bead. A weld pool camera is mounted on the welding torch, and the welding wire tip and weld bead are within the field of view of the weld pool camera. The method includes: Acquire real-time images from the molten pool camera, and identify the center position of the weld bead and the position of the welding wire tip in the image based on the real-time images; The relative position of the welding wire tip and the weld center is determined based on the position of the welding wire tip and the position of the weld center. The real-time trajectory deviation of the welding wire movement trajectory is determined based on the relative position, and the welding wire end point movement is controlled according to the real-time trajectory deviation to perform welding.

2. The method according to claim 1, characterized in that, Based on the real-time image recognition, the center position of the weld bead in the image is identified, including: Acquire historical molten pool image data of the target welded object and construct an image dataset; The weld bead recognition model was trained and optimized based on the aforementioned image dataset; The weld bead recognition model is used to identify weld beads in the real-time image, and the center position of the weld bead is determined based on the identified weld beads.

3. The method according to claim 2, characterized in that, Training and optimizing the weld bead recognition model based on the aforementioned image dataset includes: From the image dataset, molten pool image data of different times, different weld layers, and different weld passes are selected, and each molten pool image data is preprocessed to obtain training samples. The deep learning network model is trained and validated using the training samples to obtain a trained weld bead recognition model. Real-time images are acquired using a preset fixed window, and the weld bead recognition model is fine-tuned using the real-time images to obtain an optimized weld bead recognition model.

4. The method according to claim 1, characterized in that, Determining the relative position of the wire tip and the weld center based on the wire tip position and the weld center position includes: Establish an image coordinate system with the weld center position as the x-axis and the direction perpendicular to the weld center position as the y-axis. The ordinate value of the welding wire tip position in the vertical direction is determined based on the welding wire tip position and the weld center position. This ordinate value is used as the relative position between the welding wire tip position and the weld center position. The sign of the ordinate value indicates the offset direction of the welding wire tip position relative to the weld center position, and the absolute value of the ordinate value indicates the magnitude of the offset of the welding wire tip position relative to the weld center position.

5. The method according to claim 1, characterized in that, The tip of the welding wire is located at the center of the field of view of the molten pool camera.

6. The method according to claim 1, characterized in that, Welding is performed by controlling the movement of the welding wire tip based on the real-time trajectory deviation, including: The direction and amount of deviation are determined based on the real-time trajectory deviation. If the deviation of the real-time trajectory is greater than the preset deviation threshold, the welding wire tip is controlled to move in the opposite direction to the deviation direction by a preset correction amount. If the deviation of the real-time trajectory is not greater than the preset deviation threshold, then there is no need to adjust the welding wire movement trajectory.

7. The method according to any one of claims 1 to 6, characterized in that, If the center position of the weld cannot be identified based on the real-time image, the duration of the period during which the center position of the weld cannot be identified is recorded. If the duration exceeds a preset time threshold, the welding process is terminated.

8. A calibration-free welding device based on molten pool camera recognition, characterized in that, A welding robot moves the welding wire tip on the welding torch along the weld bead. A weld pool camera is mounted on the welding torch, and the welding wire tip and weld bead are within the field of view of the weld pool camera. The device includes: The image recognition module is used to acquire real-time images from the molten pool camera and identify the center position of the weld bead and the position of the welding wire tip in the image based on the real-time images. The relative position determination module is used to determine the relative position between the welding wire tip and the weld center based on the welding wire tip position and the weld center position; The real-time welding module is used to determine the real-time trajectory deviation of the welding wire movement trajectory based on the relative position, and to control the movement of the welding wire end point to perform welding according to the real-time trajectory deviation.

9. A computer device, characterized in that, include: Human-computer interface, memory and processor, The human-computer interaction interface is used to receive user input instructions and data and transmit them to the processor, and to feed back the processor's processing results to the user. The memory and the processor are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method according to any one of claims 1 to 7.

Images