An image processing method, storage medium and processor for a narrow-gap U-shaped groove

Abstract

The present application relates to the field of welding manufacturing, in particular to the automatic control and design in the welding process. The present application is realized by the following technical scheme: an image processing method of narrow-gap U-shaped groove, comprising target detection image acquisition step, further comprising the following steps: S01, image selection step; in the image data of the groove, selecting continuous M frame images, each frame image being an original image; S02, image enhancement step; obtaining an enhanced image P based on the continuous M frame images, S03, binary processing step; obtaining a binary image P1; S04, combination filtering step; performing combination filtering processing on the binary image P1 to obtain a clear image P2. The present application aims to provide an image processing method of narrow-gap U-shaped groove, through the cooperation processing of three-dimensional spatial domain image enhancement and morphological filtering, obtaining a clear picture of U-shaped groove laser stripe, providing a good data basis for subsequent automatic intelligent welding.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of welding manufacturing, in particular to the automation control and design in the welding process. BACKGROUND

[0002] With the continuous progress of modern industrial production level, the industrial enterprises put forward new requirements for welding technology in terms of welding quality, production efficiency, manufacturing cost, batch supply capacity and the like. In the welding technology, full penetration welding is a common welding technology requirement, that is, the weld of single-sided welding must achieve penetration everywhere, and a uniform-sized weld must be formed on the back of the workpiece. The connection strength of such a weld is high compared with the weld without penetration or without uniform penetration.

[0003] In terms of weld penetration control technology means, there are currently two main solutions in the industry:

[0004] Solution one: using the method of "feature online detection + real-time feedback control": this is also the main research direction of scholars at home and abroad, that is, extracting relevant feature information of the arc or molten pool in the welding process to judge whether the penetration is achieved or not, such as temperature field method, arc light method, arc voltage method, arc sound method, molten pool oscillation method, etc. For example, a new real-time arc voltage tracking method for electric arc welding is disclosed in Chinese patent document No. CN202110735100, which includes an industrial computer, a motion control unit and a welding torch on the hardware. The welding torch generates an arc voltage signal during welding, and the industrial computer collects and analyzes the arc voltage signal in real time. According to the analysis result, the motion control unit is input to realize real-time adjustment of the welding parameters of the welding torch.

[0005] However, in this technical solution, light, electricity, magnetism and heat in the welding process will cause strong interference to the extraction of feature information, and these feature information is strictly synchronized with the welding process, lacking predictability, and thus having insufficient prediction ability for the upcoming welding condition mutation, making it difficult to achieve good penetration prediction and control effect.

[0006] Solution two: using the method of "slope synchronous detection + lag feedback control": laser vision method is used synchronously in the welding process to detect the slope size before the welding torch reaches the position, and according to the feedback result, lag algorithm is used to control the process parameters, so as to achieve good forming. For example, a robot automatic welding device for single-sided welding of unequal gap workpieces and double-sided forming is disclosed in Chinese patent document No. CN201810621606, which includes a laser tracking system, a robot system and a welding system. According to the slope gap recognized by the laser vision tracking system, the process parameters under the standard gap are compared, and the matched process parameters are called to control the robot to perform automatic welding of the unequal gap workpiece, so as to realize single-sided welding and double-sided forming.

[0007] The technical scheme has one "early detection", that is, there is a time and space advance between image extraction and formal welding, so that the image extraction is often not disturbed by the welding strong light as described in the first scheme. However, the technical scheme is often focused on the detection and control of thin plate butt joints, V-groove butt joints and corner joints. As mentioned in the above Chinese patent document with the publication number CN201810621606, it is also applied in the weld penetration control of V-groove.

[0008] However, in actual welding, in addition to the V-groove, the common groove type is the narrow gap U-groove. Compared with the ordinary groove, the narrow gap U-groove has a larger depth-width ratio, a narrower groove width and a steeper groove side wall. When welding, less filler material can be used to fill the groove, the heat input of the welded joint is small, and the weld quality is high, so it is often used for welding of important structural parts.

[0009] However, due to the shape characteristics of the above-mentioned groove, the extraction of feature information during detection is much more difficult than that of the conventional groove, and it is not suitable for automatic welding by the second scheme described above. Specifically, during detection, it is difficult to automatically extract feature information by machine vision, which is due to the following two aspects:

[0010] Reason one: uneven imaging brightness. For a conventional groove, such as a V-shaped groove, the light emitted by the laser forms a normal incidence on the workpiece and the groove surface, and the imaging effect is good. For a narrow gap U-groove, due to the very narrow groove width and the very steep groove side wall, the light emitted by the laser is still normally incident on the workpiece surface and the groove bottom, but it forms a grazing incidence on the groove side wall, and there is less effective reflected light, resulting in uneven imaging in the image, with high brightness in some areas and low brightness in some areas. The direction of the laser beam is nearly parallel to the groove side wall, so the brightness of the laser stripe at the position of the groove side wall is obviously insufficient, and even the brightness is lower than that of some stray light at the bottom of the groove, and the laser stripe has many discontinuities. This situation is extremely unfavorable for the extraction of assembly gap and other feature information.

[0011] Reason two: random noise interference with information extraction. In normal actual production, the workpiece surface sometimes rusts and sometimes the local position is exposed to metal luster due to manual polishing, resulting in changes in surface roughness. Since the roughness change is random, and natural light contains all wavelengths of light, when the laser vision detection scans to these positions, part of the light will pass through the filter and appear as random and large-area reflections on the collected picture, causing serious interference with the extraction of the laser stripe. Traditional laser stripe image processing algorithms are difficult to filter out these stray light well.

[0012] Due to the shape characteristics of the above type of groove, the feature information extraction is much more difficult than the conventional groove during detection, and it still belongs to the research blank. In actual industrial production, the welding of narrow gap U-shaped groove still relies on robot repeated teaching and manual monitoring of welding process to ensure the welding quality, and it is difficult to avoid the problems of low welding efficiency, high production cost and subjectivity of manual supervision. SUMMARY

[0013] The purpose of the present application is to provide a narrow gap U-shaped groove image processing method, storage medium and processor, which obtains a clear picture of U-shaped groove laser stripe through the cooperation of three-dimensional spatial domain image enhancement and morphological filtering, and provides a good data basis for subsequent automatic intelligent welding.

[0014] The above technical purpose of the present application is realized by the following technical scheme:

[0015] A narrow gap U-shaped groove image processing method, comprising a target detection image acquisition step, in which a laser and an industrial camera move along the weld direction to take pictures and obtain image data of the groove, the image data being time-continuous pictures; after the target detection image acquisition step, the following steps are further included:

[0016] S01, image selection step;

[0017] In the image data of the groove, M consecutive images are selected, and each image is an original image I 原 ;

[0018] S02, image enhancement step;

[0019] Based on the M consecutive images, an enhanced image P is obtained, and this step specifically includes:

[0020] S021, low-frequency component image acquisition step,

[0021] Gaussian filtering is adopted to perform smoothing convolution operation processing on each frame of original image, and the low-frequency component I 低 of the image is extracted,

[0022] I 低 = I 原 ★G(s, t),

[0023] Wherein, is the convolution operation operator, is the Gaussian filter kernel. e is a natural constant, r is the radius size of the Gaussian filter kernel, and sigma is the standard deviation of the Gaussian filter kernel;

[0024] S022, low-frequency component removal step,

[0025] The original image I原 Subtract I obtained in the previous step 低 to obtain a difference image I, I = I 原 - I 低 ;

[0026] S023, an enhanced image obtaining step,

[0027] Summing the difference images I of the continuous M frames after removing the low-frequency components to obtain an enhanced image P, i is the frame number of the difference image;

[0028] S03, a binarization processing step;

[0029] Binarizing the image P based on a global threshold segmentation method to obtain a binarized image P1;

[0030] S04, a combination filtering step;

[0031] Combination filtering processing the binarized image P1 to obtain a clear image P2,

[0032] The combination filtering is a combination filtering mode of primary particle filtering, dilation, secondary particle filtering, and corrosion, wherein the particle area removed by the primary particle filtering is smaller than the particle area removed by the secondary particle filtering.

[0033] As a preferred embodiment of the present application, in the S01, the value range of M is set to 7-14.

[0034] As a preferred embodiment of the present application, in the S021, the Gaussian filter kernel size is selected to be 27*27.

[0035] As a preferred embodiment of the present application, in the S04, the corrosion and the dilation are processed using the same size template.

[0036] As a preferred embodiment of the present application, in the S04, the corrosion and the dilation are processed using a 9x9 circular template.

[0037] As a preferred embodiment of the present application, in the S03, the global threshold segmentation method is the Otsu global threshold segmentation method.

[0038] As a preferred embodiment of the present application, after the S04, there is a step of judging the out-of-tolerance, in which the gap size of the groove is obtained based on the clear image P2, and the out-of-tolerance is judged, if the gap size of the groove exceeds the preset range, the automatic welding is not performed, and manual intervention is performed.

[0039] A storage medium, which contains a stored program, the program executes the image processing method of the narrow-gap U-shaped groove.

[0040] A processor for running a program, the program performing the image processing method of the narrow-gap U-shaped groove.

[0041] In summary, the present application has the following beneficial effects:

[0042] 1. After image enhancement, binarization processing and combination filtering operation, a clear image is finally obtained, even for a narrow-gap U-shaped groove, on the one hand, the laser stripes of the side wall are clearly visible and the lines are obvious, on the other hand, the whole image is clear in black and white, without random and large-area reflection, and without noise particles to interfere with the laser stripes.

[0043] 2. The clear image brings convenience for feature extraction of the narrow-gap U-shaped groove, so that the groove size information can be extracted by machine vision, and automatic welding operation can be performed by automatic welding equipment.

[0044] 3. The global threshold segmentation method is Otsu global threshold segmentation method, which has high automation degree.

[0045] 4. The same size template is used for erosion and expansion processing, so that the expansion processing can better ensure that the feature point position does not drift, and the negative influence of information extraction error is avoided. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 is a schematic diagram of the binarized image P1 in Example 1;

[0047] Figure 2 is a schematic diagram of the clear image P2 finally obtained in Example 1;

[0048] Figure 3 is a flowchart of Example 3. DETAILED DESCRIPTION

[0049] The present application will be further described in detail below with reference to the accompanying drawings.

[0050] The present embodiment is only an explanation of the present application, and is not a limitation of the present application, and those skilled in the art can make modifications to the present embodiment without creative contribution according to the needs after reading the present specification, but as long as the present application is within the scope of the claims, it is protected by the patent law.

[0051] Example 1, an image processing method of a narrow-gap U-shaped groove, which is consistent with the existing automatic welding system in hardware implementation, and contains three module units of image acquisition unit, data analysis processing unit and control unit. The functions and working modes of the three module units are explained below.

[0052] Image acquisition unit: including laser, narrowband filter, industrial camera, lens and image acquisition card, mainly for collecting laser stripe image. The laser and the industrial camera are installed at a certain distance and angle. During the measurement process, the laser emits a "one" type laser projection on the groove surface, the filter selects the corresponding central wavelength of the laser to filter out the stray light entering the camera, and the laser stripe projected on the groove surface is photographed into a gray image by the camera, which is transmitted to the industrial computer through the image acquisition card.

[0053] Image processing and analysis unit: including industrial computer, its embedded image processing algorithm, data filtering algorithm and process parameter prediction model, mainly for processing the acquired laser stripe image, extracting the assembly gap size value, and obtaining the corresponding welding current prediction value through data filtering and process model prediction and transmitting to the control unit.

[0054] Control unit: including PLC lower computer, electric welder and welding gun, according to the received welding current prediction value, sequentially output at the corresponding position after the start of welding, get the weld with uniform back penetration, realize single-sided welding double-sided forming.

[0055] In this case, the two major module units of image acquisition unit and control unit have not been changed or only slightly adapted, and the main technical improvement and innovation are concentrated in the image processing and analysis unit. Details are described below.

[0056] According to the time step division, the use method of the groove automatic detection and welding system based on machine vision contains the following steps:

[0057] First, the target detection image acquisition step.

[0058] In this step, the image data of the groove is acquired by the image acquisition unit, and the image data is sent to the image processing and analysis unit. Specifically, the laser and the industrial camera are installed at a certain angle and position relationship, and the narrowband filter selects the central wavelength corresponding to the laser emission (such as 670±10nm) to filter out the stray light entering the camera. The laser emits a "one" type laser projection on the groove, and the industrial camera is aligned with the groove to adjust the focal length to the clear laser stripe image. The industrial camera is connected with the industrial computer through the image acquisition card. After the system is built, the field of view of the industrial camera is calibrated, and the corresponding relationship between the world coordinate system and the image coordinate system is established.

[0059] After the detection starts, the laser and the industrial camera move forward along the weld direction at a certain speed V detect, and the industrial camera takes pictures continuously at a certain frequency f, collects the groove laser stripe gray image, and transmits it to the industrial computer through the image acquisition card.

[0060] For example, in the present embodiment, V detect = 5mm / s, image sampling rate f = 10Hz. Collect 600 frames of laser stripe gray scale images, and transmit to the industrial computer through the image acquisition card.

[0061] At this time, the industrial computer has obtained the gray scale image of the groove.

[0062] This step is the same as the prior art. After this step is completed, the industrial computer obtains the laser stripe gray scale image of the groove and the corresponding visual system calibration.

[0063] Subsequently, the next step, the actual innovative step of the present application, the image processing step, is entered, which specifically includes the following steps.

[0064] S01, image selection step.

[0065] M consecutive images are selected, and M is determined by the user. If M is too large, the subsequent welding unit control length will be relatively long, thereby resulting in insufficient welding accuracy. If M is too small, the data features in the subsequent image recognition process will not be obvious enough. Generally, M is set to 7-14, and in the present embodiment, 10 is selected, i.e., 10 consecutive images are obtained.

[0066] S02, image enhancement step.

[0067] As described in the background art, the narrow gap U-shaped groove has a very narrow groove width and a very steep groove side wall, and grazing incidence will be formed on the groove side wall, resulting in less effective reflected light. This results in a significant lack of laser stripe brightness at the position of the groove side wall, and even the brightness is lower than that of some stray light at the bottom of the groove, and the laser stripe has many discontinuities. The present step is a three-dimensional spatial domain image enhancement, which serves to increase the uniformity of the brightness of the laser stripe.

[0068] Specifically, S021, low-frequency component image acquisition step.

[0069] Gaussian filtering is used to perform smoothing convolution operation processing on the original image, and the low-frequency component I 低 of the image is extracted.

[0070] I 低 = I 原 ★G(s, t).

[0071] Wherein, ★ is the convolution operation operator, is the Gaussian filter kernel. e is a natural constant, r is the radius size of the Gaussian filter kernel, and σ is the standard deviation of the Gaussian filter kernel. These two values are determined by the type of Gaussian filter kernel selected. Through experimental testing, in the present embodiment, the size of the Gaussian filter kernel is preferably 27X27; therefore σ = 4.5.

[0072] S022, low-frequency component removing step.

[0073] The original image I 原 Subtract I 低 obtained in the previous step, to obtain a difference image I.

[0074] I = I 原 - I 低 .

[0075] S023, enhanced image obtaining step.

[0076] This step is to obtain a three-dimensional spatial domain enhanced image P. The method is to add the pixels of the continuous M frames of images after removing the low-frequency components, that is, to add the continuous 10 frames of difference images I in this embodiment.

[0077] i is the frame number of the difference image, that is, 0, 1, 2-8, 9, a total of 10 frames in this embodiment.

[0078] S03, binary processing step.

[0079] Based on the global threshold segmentation method in the prior art, the image P is binary processed, that is, image segmentation processing, to obtain a binary image P1. After the image P is binary processed, there is no gray scale, only black and white, which is 0 or 1 in data. There are many global threshold segmentation methods in the prior art, such as Huang threshold segmentation method, InterModes threshold segmentation method, IsoData threshold segmentation method, Moments (geometric moment threshold) segmentation method, etc.; in this embodiment, the Otsu global threshold segmentation method is selected.

[0080] The operation formula is:

[0081]

[0082] Wherein, x, y represent the coordinate values of a pixel in the image, k * is the optimal threshold value, in the Otsu global threshold segmentation method, the value of k * is automatically determined according to the gray scale distribution of the image, without user specification, and the automation degree is higher.

[0083] S04, combination filtering step.

[0084] This step is one of the key steps of the present application.

[0085] By the operation of S02, the image has been removed a part of the noise, but not completely. In this step, the morphological particle filtering is used to remove the small and isolated noise particles which are not filtered by S02. As shown in Fig. 3, the laser stripe has been enhanced, and the non-uniformity of the gray scale has been improved. However, there are still many white noise particles in the image. Figure 1

[0086] Here, the particle refers to a region formed by a group of non-zero pixels which are connected to each other in the image. The morphological particle filtering is used to remove / retain the corresponding particle according to a certain morphological criterion (such as the maximum diameter, the center of mass coordinate, the area, etc.). In the present application, the morphological particle filtering is used to remove the particles with an area less than a certain threshold. The filtering threshold K is user-defined. For example, if K is defined as 100, the particles with an area less than 100 (such as the particles with an area of 89, 72 and 95) will be removed, and the particles with an area of 102, 200 and 310 will be retained.

[0087] If the filtering method of the ordinary particle filtering is used, when the value of K is large, the discontinuous part of the laser stripe may be removed by mistake. If the value of K is small, it is impossible to remove all the particles which should be removed, and many particle noises will be retained in the image, resulting in errors in the subsequent feature collection.

[0088] In the present embodiment, the combined filtering method is used, and specifically, the combined filtering method of "one-time particle filtering, dilation, two-time particle filtering and erosion" is used to process the binary image P1.

[0089] It should be noted that the particle filtering processing, the dilation processing and the erosion processing are all existing technologies in the prior art, and can be directly selected by those skilled in the art.

[0090] S041, one-time particle filtering. The filtering threshold K1 is user-defined, and the particles with an area less than K1 in the binary image P1 are removed.

[0091] S042, dilation. The white particles are dilated, and the area is increased. The black background is not affected.

[0092] S043, two-time particle filtering. The filtering threshold K2 is also user-defined.

[0093] The one-time particle filtering is used to remove the particles with a small area, and the two-time particle filtering is used to remove the particles with a large area. Therefore, the filtering threshold K2 of the two-time particle filtering is greater than the filtering threshold K1 of the one-time particle filtering.

[0094] ​The inflation processing is performed before the secondary particle filtering, so as to connect the discontinuous part of the laser stripe particle, and avoid being deleted by mistake in the secondary particle filtering. Meanwhile, the inflation processing avoids connecting the small noise particles around the laser stripe particle, so as to avoid the change of the stripe shape and the drift of the feature point.

[0095] S044, corrosion. The corrosion processing is an operation opposite to the inflation processing, and the particle which has been inflated and enlarged is reduced. The same size template is used in the corrosion and inflation processing, so as to avoid the drift of the feature point position in the inflation processing, and avoid the negative influence of the information extraction error.

[0096] In the embodiment, K1 and K2 can be 300 and 600 respectively, that is, the primary particle filtering removes the particle with the area below 300 pixels, and the secondary particle filtering removes the particle with the area below 600 pixels. The circular template with the size of 9*9 can be used in the inflation and corrosion processing.

[0097] Up to now, the combined filtering processing is completed. After the image enhancement, the binary processing and the combined filtering operation, the clear image P2 is obtained, as shown in the following figure, which is the final output of the present application. Figure 2

[0098] As shown in the figure, even for the narrow gap U-shaped groove, on the one hand, the laser stripe of the side wall is clear and visible, and on the other hand, the whole image is clear, without random and large area reflection, and without noise particle to interfere with the laser stripe. Such clear image P2 brings convenience for the feature extraction of the narrow gap U-shaped groove, so that the groove size information can be extracted by the machine vision, and the automatic welding operation can be performed by the automatic welding equipment.

[0099] Embodiment 2: On the basis of embodiment 1, the stripe information extraction, the gap value extraction and the data filtering operation are added.

[0100] Specifically, after the S04, the combined filtering step is completed, the clear image P2 is obtained. At this time, the stripe information extraction operation is performed on P2, the column with the sum of the pixel gray value of 0 is marked, and these columns with the sum of the pixel gray value of 0 are the positions of the assembly gap.

[0101] Subsequently, the gap value extraction operation is performed, the pixel size of the gap is obtained by subtracting the leftmost marked column from the rightmost marked column, and finally the actual gap size value is obtained by converting the pixel size to the world coordinate system by using the calibration coefficient.

[0102] ​Data filtering, the measured whole gap size results of the assembly of the groove are filtered by a one-dimensional median filter with a kernel size of mx1, to weaken the influence of outliers. In order to keep the data quantity unchanged before and after filtering, the first and last data of the data sequence set need to be copied

[0103] Example 3: On the basis of example 2, a step of judging whether it is out of tolerance is added.

[0104] As Figure 3 shown, after the data filtering operation, the groove size information is judged, that is, the gap size of the groove is judged for out of tolerance. If the gap size of the groove is out of the preset range, the automatic welding is not performed, but a prompt is issued, and manual intervention is converted.

[0105] A storage medium, the storage medium contains a stored program, wherein the program executes the image processing method suitable for narrow gap U-shaped groove as described above.

[0106] A processor for running a program, wherein the program executes the image processing method suitable for narrow gap U-shaped groove as described above.​

Claims

1. An image processing method for a narrow-gap U-shaped bevel, comprising a target detection image acquisition step, wherein a laser and an industrial camera move along the weld direction to take pictures, thereby obtaining image data of the bevel, wherein the image data is a time-continuous image; characterized in that, Following the target detection image acquisition step, the following steps are also included: S01, Image selection steps; From the image data of the bevel, select M consecutive frames, each frame being the original image I. 原 ; S02, Image Enhancement Steps; The enhanced image P is obtained based on M consecutive frames of images. This step specifically includes: S021, Low-frequency component image acquisition steps. Gaussian filtering is used to perform smooth convolution operations on each frame of the original image to extract the low-frequency component I in the image. 低 , I 低 =I 原 ★G(s,t), Where ★ represents the convolution operator. σ is the Gaussian filter kernel; e is the natural constant, r is the radius of the Gaussian filter kernel, and σ is the standard deviation of the Gaussian filter kernel. S022, Low-frequency component removal step. Original image I 原 Subtract the I obtained in the previous step 低 The difference image I is obtained, I = I 原 -I 低 ; S023, Enhanced image acquisition steps The enhanced image P is obtained by summing the difference images I of M consecutive frames after removing low-frequency components. i is the frame number of the difference image; S03, Binarization process; Based on the global threshold segmentation method, the image P is binarized to obtain a binarized image P1; S04, Combined Filtering Steps; The binarized image P1 is subjected to combined filtering to obtain a clear image P2. The combined filtering is a combination of primary particle filtering, dilation, secondary particle filtering, and erosion, wherein the particle area removed by primary particle filtering is smaller than the particle area removed by secondary particle filtering.

2. The image processing method for a narrow-gap U-shaped bevel according to claim 1, characterized in that: In S01, the value range of M is set to 7-14.

3. The image processing method for a narrow-gap U-shaped bevel according to claim 1, characterized in that: In S021, the Gaussian filter kernel size is selected as 27*27.

4. The image processing method for a narrow-gap U-shaped bevel according to claim 1, characterized in that: In S04, corrosion and expansion are treated using templates of the same size.

5. The image processing method for a narrow-gap U-shaped bevel according to claim 4, characterized in that: In S04, both corrosion and expansion are treated using a 9×9 circular template.

6. The image processing method for a narrow-gap U-shaped bevel according to claim 1, characterized in that: In S03, the global threshold segmentation method is the Otsu global threshold segmentation method.

7. The image processing method for a narrow-gap U-shaped bevel according to any one of claims 1-6, characterized in that: After S04, there is a step to determine if the tolerance is exceeded. In this step, the gap size of the bevel is determined based on the clear image P2, and a tolerance judgment is made. If the gap size of the bevel exceeds the preset range, automated welding will not be performed, and manual intervention will be used instead.

8. A storage medium comprising a stored program, characterized in that, The program executes the image processing method for the narrow-gap U-shaped bevel as described in any one of claims 1-7.

9. A processor for running programs, characterized in that, The program executes the image processing method for the narrow-gap U-shaped bevel as described in any one of claims 1-7.

Images