A weld tracking method and system based on arc sound composite laser vision

Abstract

The present invention discloses a weld tracking method based on arc sound composite laser vision, including: a system mainly composed of an image acquisition module, an acoustic signal acquisition module, an acoustic signal processing module, an image processing module, a welding robot and supporting welding equipment, and an industrial computer. Collect welding arc sound, obtain arc sound information, perform time delay estimation on the arc sound information, and obtain a relationship prediction model between time delay and weld deviation; obtain original image information of the weld, perform median filtering on the original image information to obtain a spare weld image, extract spare weld information pixel points, and fuse them with the relationship prediction model between time delay and weld deviation using robust least squares method, screen available weld pixel points, and obtain real-time weld tracking trajectory by fitting, effectively eliminating the interference of noise such as arc light, and ensuring real-time weld tracking with high accuracy and strong anti-interference ability.

Description

Technical Field

[0001] The invention relates to a welding seam tracking method and system based on arc sound composite laser vision, belonging to the technical field of welding trajectory intelligent tracking. Background Art

[0002] Welding is an indispensable part of the manufacturing industry and plays a vital role in it. In the process of continuously deepening the integration of informatization and industrialization, welding technology is developing towards intelligence and digitalization, which continuously improves the quality of welding products. At present, most robot welding production still adopts the teaching and reproduction method, that is, the robot repeats welding according to the pre-set welding path and speed. This welding method is simple to operate, but when the welding workpiece changes, the teaching program needs to be reset, which consumes a lot of manpower and time costs, resulting in a significant reduction in welding efficiency. In actual production, the welding environment and welding targets are often changeable, and the teaching and reproduction mode cannot perceive these changes in the welding environment, so the welding quality of the workpiece cannot be guaranteed. Therefore, the traditional teaching and reproduction welding robot cannot realize the flexibility and intelligence of the welding process, and automatic weld tracking is an important aspect of welding intelligence research.

[0003] The key to automatic weld tracking is sensor technology, which uses sensors to accurately obtain weld data, and further uses the data detected by the sensors to feedback and adjust through the controller. At present, voltage, current sensors and visual sensors have been used for weld tracking. Among them, visual sensors are the most widely used. They have the advantages of large amount of information, non-contact, high sensitivity, accuracy and automation, but they are limited by strong arc light, smoke, splash and other noise. The arc sound signal contains a lot of welding dynamic information and can be used to identify the dynamic process of welding. At the same time, since the arc sound signal originates from the arc heat and the vibration of the molten pool and is not easily affected by the shape change of the workpiece, it can be used to monitor the dynamic process of welding.

[0004] Chinese patent CN111390439B discloses a weld detection method, device, welding robot and storage medium, including: scanning the parent material in a preset area around the weld by radar to obtain distance information; determining the parent material cross-sectional curvature and the relative position information between the weld and the radar according to the distance information; determining the relative position information between the weld and the radar and the relative position information between the radar and the laser camera according to the relative position information between the weld and the radar; determining the region of interest ROI from the image acquired by the laser camera according to the relative position information between the weld and the laser camera; determining the center position of the weld and the width of the welding groove according to the line laser pattern in the ROI based on the parent material cross-sectional curvature. However, the weld detection technology method mainly has the following problems: due to the influence of high temperature, groove condition, processing error, surface condition and workpiece thermal deformation, the welding torch will deviate from the weld, lack of real-time weld detection, resulting in reduced welding quality or even failure. Moreover, the weld detection method cannot effectively eliminate the influence of strong arc light, smoke, splash and other noise on the laser camera, and thus cannot perform real-time weld tracking well.

[0005] In summary, in real-time weld tracking, it is necessary to design a real-time weld tracking method and system with high accuracy and strong anti-interference ability to eliminate the interference of noise such as arc, smoke, and spatter. Summary of the invention

[0006] The purpose of the present invention is to collect weld information in real time through laser vision, and to establish a prediction model for the relationship between time delay and weld deviation by collecting and analyzing arc sound signals, and to obtain a real-time weld tracking trajectory after screening and fitting the visual collection information with the prediction model. The invention can effectively eliminate the influence of strong arc light, smoke, splash and other noise on the laser vision sensor, and is a real-time weld tracking method with high accuracy and strong anti-interference. The system includes an image acquisition module, an acoustic signal acquisition module, an acoustic signal processing module, an image processing module, a welding robot and supporting welding equipment, and an industrial computer.

[0007] The image acquisition module is a CCD camera (2) equipped with a filter, which is fixed at the front end of the welding gun (1) in the moving direction; the sound signal acquisition module is composed of a dual-microphone microphone (3), which is fixedly installed on the left and right diagonal sides of the welding workbench (4) and is used to collect and amplify the arc sound.

[0008] A weld tracking method and system based on arc sound combined with laser vision, characterized in that: the real-time weld tracking method comprises the following steps:

[0009] The arc sound is received simultaneously by a dual-microphone microphone, and then the arc sound information is obtained by an acoustic signal processing module;

[0010] The arc sound information is subjected to noise reduction processing to obtain a first sound signal and a second sound signal, including: weighting the arc sound information by a generalized correlation method (GCC) in a functional spectrum domain to highlight the signal and suppress noise, and the definition of the GCC function is as follows:

[0011]

[0012] Where: g (ω) is the weight function, Φ 12 (ω) is R 12 (τ) is the power spectrum, ω is the angular frequency;

[0013] The time delay of the first sound signal and the second sound signal is estimated to obtain a relationship model between the time delay and the weld deviation, including: the time delay estimation is performed on the original signal frame by frame with 20 ms as one frame; and the weighted least squares linear regression is used to obtain the relationship prediction model between the time delay and the weld deviation;

[0014] Obtain the original image information of the weld through the laser vision sensor;

[0015] De-noising the original image information to obtain a spare weld image, including: performing a de-noising process on the original weld image using a median filter;

[0016] Sharpening the weld edge of the spare weld image and extracting the spare weld information pixel points, including: using Canny edge detection to identify the weld contour;

[0017] The robust least squares method is used for the relationship prediction model between the spare weld information pixel points and the time delay and weld deviation, the available weld pixel points are screened, and a real-time weld tracking trajectory is fitted, including: fusing the spare weld information pixel points and the relationship prediction model between the time delay and weld deviation, and obtaining the weld deviation value X from the relationship prediction model of the weld deviation i , thus determining the residual of the i-th spare weld information pixel point as δ i And δ i =X i , with weight W i , using the robust estimation function Huber function to get the weight W i Value:

[0018]

[0019] in is a parameter used to measure the normal range of residuals. Alternative weld information pixels with a value smaller than this value are screened as available weld pixels, and alternative weld information pixels with a value larger than this value are abnormal points. The abnormal values ​​are penalized and their weights are reduced, so that the model parameters are not easily affected by abnormal values.

[0020] Use the gradient descent method to minimize the objective function. The formula of the gradient descent method is as follows:

[0021]

[0022] Where: θ i represents the parameter vector of the i-th iteration, J(θ) represents the loss function, Represents the gradient vector of the loss function, α represents the learning rate, and controls the step size of each iteration of the parameter vector;

[0023] Thus, the robustly estimated model parameters are obtained and the real-time weld tracking trajectory is fitted.

[0024] Compared with the prior art, the present invention has achieved the following technical effects:

[0025] (1) In the actual welding process, the arc sound signal is complex. A dual-microphone array is used, and the array redundant information is used to eliminate the influence of noise and extract high-quality arc sound signal analysis. A prediction model for the relationship between time delay and weld deviation is established.

[0026] (2) The present invention uses a weld tracking method and system based on arc sound combined with laser vision to fuse laser vision information with arc sound signals for real-time weld tracking. This effectively overcomes the shortcomings of using only laser vision cameras being affected by strong arc light, smoke, splash and other noises, and using only arc sound signals being easily affected by arc stability, current size and other factors, thereby improving the anti-interference ability and achieving high-precision weld tracking and control. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 The schematic diagram of a weld tracking system based on arc sound and laser vision includes a welding gun (1), a CCD camera (2), a dual-microphone microphone (3), and a welding workbench (4);

[0029] Figure 2A workflow diagram of a real-time weld tracking method using dual microphone array and laser vision;

[0030] Figure 3 Schematic diagram for GCC time delay estimation. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The purpose of the present invention is to provide a weld tracking method and system based on arc sound combined with laser vision. In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0033] Embodiment 1:

[0034] A weld tracking method and system based on arc sound composite laser vision, such as Figure 1 , Figure 2 As shown, the weld tracking system includes an image acquisition module, an acoustic signal acquisition module, an acoustic signal processing module, an image processing module, a welding robot and supporting welding equipment, and an industrial computer. The image acquisition module is a CCD camera (2) equipped with a filter, which is fixed at the front end of the welding gun (1) in the moving direction. After collecting the original image information, it is transmitted to the image processing module for processing the original image information; the acoustic signal acquisition module is composed of a dual-microphone microphone (3), which is fixedly installed on the left and right diagonal sides of the welding workbench (4), and is used to collect and amplify the arc sound, and transmit it to the acoustic signal processing module for processing the arc sound information; the processed original image information and the processed arc sound information are transmitted to the industrial computer via an Ethernet cable.

[0035] The industrial computer uses median filtering to denoise the original weld image and obtain a backup weld image. The median filter first defines a (2n+1)*(2n+1), (n=1,2,3,…) mask, and then arranges the grayscale values ​​of the pixels in the mask in ascending or descending order. The center of the sorted mask is taken out to replace the grayscale value of the pixel to achieve filtering. The expression is as follows:

[0036] g(x,y)=med{x,y}

[0037] Canny edge detection is used to identify the weld contour, so that the weld edge of the spare weld image is sharpened and the spare weld information pixel points are extracted; the industrial computer performs noise reduction processing on the arc sound information to obtain the first sound signal and the second sound signal, and the time delay of the first sound signal and the second sound signal is estimated to obtain the relationship model between the time delay and the weld deviation; the robust least squares method is used to predict the relationship between the spare weld information pixel points and the time delay and the weld deviation, the available weld pixel points are screened, the weld tracking trajectory is fitted, and then the corresponding actions of the welding robot and the supporting welding equipment are controlled through feedback.

[0038] Embodiment 2:

[0039] In this embodiment, the arc sound information is subjected to noise reduction processing, such as Figure 3 As shown, the arc sound information is weighted by the generalized correlation method (GCC) in the functional spectrum domain to highlight the signal and suppress the noise, thereby obtaining the first sound signal and the second sound signal. The definition of the GCC function is as follows:

[0040]

[0041] Where: g (ω) is the weight function, Φ 12 (ω) is R 12 (τ) is the power spectrum, ω is the angular frequency.

[0042] The time delay of the first sound signal and the second sound signal is estimated to obtain a relationship model between the time delay and the weld deviation, including: the time delay estimation is performed on the original signal frame by frame with 20 ms as one frame; the weighted least squares linear regression is used to obtain the relationship prediction model between the time delay and the weld deviation, specifically, the robustfit in MATLAB is used to estimate the model parameters using the weighted least squares method, and different weights are assigned to the arc sound data points according to the severity of their deviation from the overall change trend, thereby reducing the influence of outliers on the fitting and better characterizing the overall change trend of the data.

[0043] Embodiment 3:

[0044] In this embodiment, the robust least squares method is used for the relationship prediction model between the spare weld information pixel points and the time delay and weld deviation, the available weld pixels are screened, and the real-time weld tracking trajectory is fitted, including: fusing the spare weld information pixel points and the relationship prediction model between the time delay and weld deviation, and obtaining the weld deviation value X from the relationship prediction model of the weld deviation. i , thus determining the residual of the i-th spare weld information pixel point as δ i And δ i =Xi , with weight W i , using the robust estimation function Huber function to get the weight W i Value:

[0045]

[0046] in is a parameter used to measure the normal range of residuals. Alternative weld information pixels smaller than this value are screened as available weld pixels, and alternative weld information pixels larger than this value are abnormal points. Abnormal values ​​are penalized and their weights are reduced, so that model parameters are not easily affected by abnormal values.

[0047] Use the gradient descent method to minimize the objective function. The formula of the gradient descent method is as follows:

[0048]

[0049] Where: θ i represents the parameter vector of the i-th iteration, J(θ) represents the loss function, represents the gradient vector of the loss function, and α represents the learning rate, which controls the step size of each iteration of the parameter vector. Thus, the model parameters are robustly estimated and the real-time weld tracking trajectory is fitted.

[0050] The present specification uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A weld tracking method based on arc sound composite laser vision is mainly used to solve the problem that when a single laser vision is used for real-time weld tracking, the characteristic image is easily disturbed by a large amount of noise such as arc light and spatter due to the close installation distance between the laser vision sensor and the welding torch, which affects the automatic welding accuracy. The method is characterized by: The system mainly comprises an image acquisition module, an acoustic signal acquisition module, an acoustic signal processing module, an image processing module, a welding robot and supporting welding equipment, and an industrial computer; the image acquisition module is a CCD camera (2) equipped with a filter, fixed at the front end of the welding gun (1) in the moving direction; the acoustic signal acquisition module is composed of a dual-microphone microphone (3), fixedly installed on the left and right diagonal sides of a welding workbench (4), and used to collect and amplify arc sound; wherein the acoustic signal acquisition module collects welding arc sound, uses the acoustic signal processing module to obtain arc sound information, transmits it to the industrial computer, performs time delay estimation on the arc sound information, and obtains a prediction model of the relationship between time delay and weld deviation; while the acoustic signal acquisition module collects welding arc sound, uses the image acquisition module to obtain original image information of the weld, transmits it to the industrial computer, performs median filtering on the original image information to obtain a spare weld image, extracts spare weld information pixel points, fuses them with the prediction model of the relationship between time delay and weld deviation using a robust least square method, screens available weld pixel points, and obtains a real-time weld tracking trajectory by fitting.

2. The weld tracking method based on arc sound combined with laser vision according to claim 1, characterized in that: The arc sound information is denoised to obtain a first sound signal and a second sound signal. The arc sound information is weighted in the functional spectrum domain by the generalized correlation method (GCC) to highlight the signal and suppress noise. The time delay of the first sound signal and the second sound signal is estimated, and the time delay of the original signal is estimated frame by frame with 20 ms as one frame to obtain a prediction model of the relationship between the time delay and the weld deviation. The weighted least squares linear regression is used to obtain the prediction model of the relationship between the time delay and the weld deviation.

3. The weld tracking method based on arc sound combined with laser vision according to claim 1, characterized in that: The spare weld information pixel points and the relationship prediction model between the time delay and the weld deviation are fused, the weld deviation value is obtained by the relationship prediction model of the weld deviation, the residual of the spare weld information pixel points is determined, and the weight W is obtained by using the robust estimation function Huber function i The objective function is minimized using the gradient descent method to obtain robustly estimated model parameters and fit the real-time weld tracking trajectory.

Images