Welding seam position detection method in strong noise environment

Through improved segmentation network and edge detection methods, combined with filtering and morphological processing, the accuracy of weld image recognition in a strong noise environment is solved, and high-precision detection of weld information is achieved to ensure the optimization of welding quality and production cost. .

CN118961710APending Publication Date: 2024-11-15ANHUI UNIVERSITY OF TECHNOLOGY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411005838.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In a highly noise environment, there is noise interference in the weld image, which makes it difficult to ensure the accuracy of weld quality recognition. Traditional segmentation networks cannot effectively detect noise problems in the image, affecting welding quality and production costs.

Method used

The improved segmentation network combined with edge detection method is adopted to gradually remove noise through filtering and noise reduction processing, morphological processing and edge detection algorithms to ensure the authenticity and accuracy of weld information. Specific steps include image preprocessing, segmentation network segmentation, morphological processing and edge detection fusion, edge detection using Canny and Sobel operators, non-maximum suppression and weighted fusion, and improve the detection accuracy of weld edge information.

Benefits of technology

The intelligent weld position detection in a highly noise environment is achieved, the accuracy and detection accuracy of weld information identification are improved, and the welding quality and production cost are reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118961710A_ABST
    Figure CN118961710A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of welding seam edge position detection, and particularly relates to a welding seam position detection method in a strong noise environment. The method is used for improving the welding positioning precision in the welding seam position detection and welding seam recognition processes. In the welding process, a real-time welding seam surface image is collected and preprocessed, the obtained image is further segmented to obtain an image only with welding seam information, and meanwhile, an edge detection algorithm is used for recognizing the welding seam image to judge whether specific welding seam edge information can be displayed or not; and meanwhile, corresponding data can be provided for weld joint track planning of the industrial robot in the welding process. The method has the characteristics of high recognition degree, good learning performance, high accuracy and the like, and can be applied to the fields of trajectory planning and the like in the industrial robot welding process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of weld edge position detection, and in particular relates to a weld position detection method in a strong noise environment. Background Art

[0002] In industrial environments, the quality, safety and reliability of welds are of vital importance. In order to improve the quality of welds, it is necessary to detect and identify the weld quality during the welding process. Usually, weld images are often accompanied by noise interference from the welding environment. For example, the mirror reflection of stainless steel during welding and the splashing of welding slag during welding will affect the accuracy of our weld quality identification, making it difficult for welders to identify the true situation of the weld, which will lead to problems such as welding quality and weld trajectory identification, and even cause great losses to the industrial environment and increase the production costs of enterprises.

[0003] For the problem of weld identification, we must first eliminate the influence of noise in the weld image on weld quality identification. The segmentation network in deep learning can process the weld image very well and completely segment the noise in the weld image. However, the segmentation algorithm is not smart enough and cannot detect whether the segmented image contains other noise problems.

[0004] In view of this, the inventor hopes to provide a method for detecting the weld position in a strong noise environment. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned problems existing in traditional technologies, provide a weld position detection method in a strong noise environment, improve the traditional segmentation network, and use edge detection for fusion to achieve good recognition results and ensure that the identified weld information is authentic and reliable.

[0006] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions:

[0007] The present invention provides a method for detecting a weld position in a strong noise environment, the method comprising the following steps:

[0008] S1, using a camera to collect the original image of the workpiece containing weld information in the welding area during the welding process;

[0009] S2, pre-processing the collected image by filtering and denoising, filtering out general speckle noise, and obtaining an image of uniform size and outputting it as a grayscale image;

[0010] S3, packing the preliminarily processed images, segmenting them using a segmentation network, extracting images containing only weld information, and filtering out other image areas within the workpiece range except for the weld information;

[0011] S4. Put the obtained weld information image into the edge detection algorithm for processing, and perform morphological processing on it to obtain the result, and determine whether it only contains weld information.

[0012] Furthermore, the specific operation of step S1 is: install the camera in a fixed position relatively still with the welding gun, and use the camera to collect the original image of the workpiece in the welding area during the welding process. The image is often accompanied by reflected light of the stainless steel workpiece and complex noise conditions, but it is only necessary to ensure that there is an obvious weld in the image, and real-time information can be fed back at the same time.

[0013] Furthermore, the specific operation of step S2 is as follows: for the pictures collected in step S1, the images are often accompanied by various noises due to different welding environments; in order to segment the noise in the image, the image is first preprocessed, and the input image is normalized to obtain a normalized image and an image without mirror reflection; at the same time, logarithmic and applied differential operations are performed on the normalized diffuse pixels; compared with the normalized image, corresponding pixels are obtained in the image without mirror reflection, and logarithmic and differential operations are performed at the same time to obtain the normalized image and the image without mirror reflection. Differential equations are calculated on pixels, and the two are subtracted based on the intensity logarithmic differential theory. If the result is zero, the adjacent pixels are diffuse reflection pixels, otherwise, the two adjacent pixels can be mirror reflection pixels, noise or two truncated pixels.

[0014] Furthermore, since the chromaticity peak of noise can be regarded as a constant, while the chromaticity peak of the mirror reflection pixel changes, it is only necessary to determine whether the chromaticity peaks of the two pixels are equal. If they are the same, it is noise, and if they are different, at least one of them is a mirror reflection pixel. Repeating the operation on the pixels can obtain the mirror reflection pixels of the entire image; since the chromaticity peak of the diffuse reflection component does not change with the change of image intensity, while the mirror reflection component changes with the image intensity, the mirror reflection component of the adjacent pixel with the smaller maximum chromaticity is reduced by changing the adjacent pixel with the smaller maximum chromaticity to make it equal to the adjacent pixel with the larger maximum chromaticity, and so on, iterating all pixels and mirror reflections in the surrounding mirror reflection area can achieve the purpose of eliminating mirror reflection.

[0015] Furthermore, the image after highlight removal is repaired through an image inpainting algorithm to repair the image that has lost some information due to the removal of highlight noise; at the same time, feature analysis and windowing processing can effectively remove the surrounding useless information, and median filtering is performed on the image to further remove noise and spot information in the image and output the image.

[0016] Furthermore, the specific operation of step S3 is: using the segmentation function in the segmentation network to segment the significant feature area of ​​the weld, and the specific steps are: inputting the image after highlight pixel filtering, performing information processing on the image with highlight pixels filtered out, outputting the spectral data of each part of the image, obtaining the regional wavelength when the spectral intensity value is the largest and taking out the average value of the wavelength; extracting the size of the spectral intensity value of each area at the average wavelength and calculating the size of the average spectral intensity value of each area; because the spectral intensity between the weld and the workpiece is different, the spectral intensity of the weld area is relatively low, and the welded workpiece is a stainless steel plate, and the spectral intensity of this type of material is relatively high during the welding process, so the size of the average spectral value is used as a benchmark, and the pixel points greater than the average spectral value are eliminated to obtain the weld area, that is, the area of ​​interest.

[0017] Furthermore, the GrabCut algorithm is executed on the selected area of ​​interest, and the image is divided into four parts: background, foreground, possible background and possible foreground to form a segmentation mask; in order to obtain more accurate mask information, the pixels of the background and possible background pixels are assigned a value so that their pixel information is 0, and the pixel information of the remaining foreground and possible foreground is assigned a value of 1, so as to obtain a mask with only foreground and background information; the foreground information can be replaced by the white part, and the remaining part is replaced by the black part to display an image that separates the foreground and background into different areas, and then the image is multiplied with the original image to segment the weld image as the foreground, that is, to obtain a weld salient feature map.

[0018] Further, the specific operation of step S4 is: by performing morphological processing on the weld image after processing, edge images in different directions are obtained, wherein the Canny operator and the Sobel operator are used to process the image areas in different directions, because the Sobel operator is a gradient operator of oblique deviation points, and the gradient here refers to the difference in the strength of the effect of the weld edge displayed in the image during the edge detection process, and the direction of the gradient here is perpendicular to the direction of the edge; and the Canny operator is considered to be an operator that optimally approximates the signal-to-noise ratio and the positioning product; therefore, these two operators are selected to perform morphological processing on the weld image after processing, and better weld edge information of the weld image is obtained, and non-maximum suppression is performed on the segmented image, wherein all the obtained windows are sorted according to the suppression values, and the window with the highest value is selected. The initial value is used as the initial value, and the remaining windows are counted as suppression windows. Then, the weld information SVM data set trained in YOLO is used for training. The output window is the probability size of the weld and the overlap rate between the initial window and the suppression window. The above operation is iterated repeatedly to compare the overlap rate and the probability size of the window weld. The result of the suppression window is statistically calculated. If the value of the suppression window is not much different from the value of the training window, and the scale is not much different, it can be determined that the repetition rate is large. The Canny operator and the Sobel operator are directly used to perform morphological processing on the input image. Different operators are applied to image areas in different directions for processing. After completion, weighted fusion is performed according to the images in different directions to obtain the weld edge image. If the window values ​​differ greatly and the window scales differ greatly, it is determined that the segmentation effect is not good, and the segmentation network is used to re-segment.

[0019] The beneficial effects of the present invention are:

[0020] 1. The weld position detection method in a strong noise environment provided by the present invention is an intelligent detection method. For the reflected light spots caused by the strong reflective workpiece environment during the welding process, the image is subjected to highlight filtering using a two-color reflection model, thereby avoiding the influence of highlight pixels in the image on the weld position detection. At the same time, the corresponding segmentation algorithm is used to further process the weld position information, thereby greatly increasing the accuracy of weld information recognition.

[0021] 2. On the basis of the above, in order to verify whether the weld image obtained by the segmentation network meets the conditions, the training window is judged by non-maximum suppression of the image through the idea of ​​edge detection until a clear weld information image can be obtained. At the same time, the edge image obtained by morphological fusion processing of the weld image greatly improves the detection accuracy of the weld edge information. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0023] Figure 1 It is a flow chart of the weld position detection method of the present invention;

[0024] Figure 2 Flowchart for image preprocessing;

[0025] Figure 3 Flowchart for obtaining the significant areas of a weld. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Example

[0028] like Figure 1 As shown, this embodiment provides a method for detecting the weld position in a strong noise environment, which mainly serves the recognition and detection of weld information in a strong noise environment, can detect the specific situation of the weld information in real time, can timely identify the location of the weld defect and provide reliable real data for the weld trajectory planning of the industrial robot. Specifically, it includes the following steps:

[0029] S1, using a camera to collect the original image of the workpiece containing weld information in the welding area during the welding process;

[0030] S2, pre-processing the collected image by filtering and denoising, filtering out general speckle noise, and obtaining an image of uniform size and outputting it as a grayscale image;

[0031] S3, packing the preliminarily processed images, segmenting them using a segmentation network, extracting images containing only weld information, and filtering out other image areas within the workpiece range except for the weld information;

[0032] S4, putting the obtained weld information image into the edge detection algorithm for processing, and performing morphological processing on it to obtain the result to determine whether it only contains weld information;

[0033] In step S1, the following steps are included:

[0034] The camera is installed in a fixed position relatively still with the welding gun. During the welding process, the camera is used to collect the original image of the workpiece in the welding area. The image is often accompanied by reflected light from the stainless steel workpiece and complex noise. However, it is sufficient to ensure that there is an obvious weld in the image. At the same time, real-time information feedback can be performed.

[0035] In step S2, the following steps are included:

[0036] For the pictures collected in step S1, there are often various noises in the images due to different welding environments. In order to segment the noise in the image, the image is preprocessed first, and the input image is normalized to obtain a normalized image and an image without mirror reflection. At the same time, logarithmic and differential operations are performed on the normalized diffuse pixels. Compared with the normalized image, we can get the corresponding pixels in the image without mirror reflection, and perform logarithmic and differential operations at the same time. In this way, we can get the differential equations on the pixels of the normalized image and the image without mirror reflection. Based on the intensity logarithmic differential theory, we will subtract the two. If the result is zero, the adjacent pixels are diffuse reflection pixels. Otherwise, the two adjacent pixels can be mirror reflection pixels, noise, or two truncated pixels.

[0037] Since the chromaticity peak of noise can be regarded as a constant, while the chromaticity peak of the specular reflection pixel changes, we only need to determine whether the chromaticity peaks of the two pixels are equal. If they are the same, it is noise, and if they are different, at least one of them is a specular reflection pixel. Repeating the operation on the pixels can obtain the specular reflection pixels of the entire image. Because the chromaticity peak of the diffuse reflection component does not change with the change of image intensity, while the specular reflection component changes with the image intensity, we can change the adjacent pixel with the smaller maximum chromaticity to make it equal to the adjacent pixel with the larger maximum chromaticity. In this way, we can reduce the specular reflection component of the adjacent pixel with the smaller maximum chromaticity. By analogy, we can iterate all pixels and specular reflections in the surrounding specular reflection area to achieve our goal of eliminating specular reflection.

[0038] The image after highlight removal is repaired by image inpainting algorithm to repair the image that has lost some information due to the removal of highlight noise. At the same time, feature analysis and windowing can effectively remove the surrounding useless information, and median filtering is performed on the image to further remove the noise and spot information in the image and output the image;

[0039] In step S3, the following steps are included:

[0040] Further segment the noise information in the image, perform information processing on the image with the highlight pixels filtered out, output the spectral data of each part of the image, obtain the regional wavelength when the spectral intensity value is the largest, and take out the average value of the wavelength. Extract the size of the spectral intensity value of each region at the average wavelength and calculate the size of the average spectral intensity value of each region. Because the spectral intensity between the weld and the workpiece is different, the spectral intensity of the weld area is lower, while the welded workpiece is a stainless steel plate, which has a higher spectral intensity during the welding process. Therefore, the size of the average spectral value is used as a benchmark, and the pixel points greater than the average spectral value are eliminated to obtain the weld area of ​​the weld, that is, the region of interest.

[0041] The GrabCut algorithm is executed on the selected area of ​​interest, and the image is divided into four parts: background, foreground, possible background, and possible foreground to form a segmentation mask. In order to obtain more accurate mask information, the background pixels and possible background pixels are assigned a value of 0, and the remaining foreground and possible foreground pixel information are assigned a value of 1, and a mask with only foreground and background information is obtained. The foreground information can be replaced by the white part, and the rest of the parts are replaced by black to display an image that separates the foreground and background into different areas. Then, the image can be multiplied with the original image to segment the weld image as the foreground, that is, to obtain the weld salient feature map.

[0042] In step S4, the following steps are included:

[0043] By performing morphological processing on the weld image after processing, edge images in different directions are obtained, wherein the Canny operator and the Sobel operator are used to process the image areas in different directions, because the Sobel operator is a gradient operator of oblique deviation points, and the gradient here refers to the difference in the strength of the weld edge displayed in the image during the edge detection process, and the direction of the gradient here is perpendicular to the direction of the edge; and the Canny operator is considered to be an operator that optimally approximates the signal-to-noise ratio and the positioning product; therefore, these two operators are selected to perform morphological processing on the weld image after processing, and the weld edge information of the weld image can be obtained better, and the image obtained by segmentation is subjected to non-maximum suppression, wherein all the obtained windows are sorted according to the suppression value, and the window with the highest value is selected as the initial value, and the remaining windows are counted as the suppression window; and then the weld information SVM data set trained in YOLO is used for training, and the output window is the probability size of the weld and the overlap rate between the initial window and the suppression window, and the above operation is iterated cyclically to compare the overlap rate and the probability size of the window weld. The results of the suppression window are counted. If the value of the suppression window is not much different from the value of the training window and the scale is not much different, it can be determined that the repetition rate is large. The Canny operator and the Sobel operator are directly used to perform morphological processing on the input image. Different operators are applied to image areas in different directions for processing. After completion, weighted fusion is performed based on the images in different directions to obtain the weld edge image. If the window values ​​differ greatly and the window scales differ greatly, it is determined that the segmentation effect is not good, and the segmentation network is used to re-segment.

[0044] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for detecting weld position in a strong noise environment, characterized in that: The method comprises the following steps: S1, using a camera to collect the original image of the workpiece containing weld information in the welding area during the welding process; S2, pre-processing the collected image by filtering and denoising, filtering out general speckle noise, and obtaining an image of uniform size and outputting it as a grayscale image; S3, packing the preliminarily processed images, segmenting them using a segmentation network, extracting images containing only weld information, and filtering out other image areas within the workpiece range except for the weld information; S4. Put the obtained weld information image into the edge detection algorithm for processing, and perform morphological processing on it to obtain the result, and determine whether it only contains weld information.

2. The method for detecting weld position in a strong noise environment according to claim 1, characterized in that: The specific operation of step S1 is: install the camera in a fixed position relatively still with the welding gun, and use the camera to collect the original image of the workpiece in the welding area during the welding process. The image is often accompanied by reflected light of the stainless steel workpiece and complex noise conditions, but it is only necessary to ensure that there is an obvious weld in the image, and real-time information can be fed back at the same time.

3. The method for detecting weld position in a strong noise environment according to claim 2, characterized in that: The specific operation of step S2 is as follows: for the pictures collected in step S1, there are often various noises in the images due to different welding environments; the images are preprocessed to separate the noise in the images, and the input images are normalized to obtain normalized images and images without mirror reflections; at the same time, logarithmic and applied differential operations are performed on the normalized diffuse pixels; Compared with the normalized image, the corresponding pixels are obtained in the image without mirror reflection, and logarithmic and differential operations are performed at the same time to obtain the differential equations of the normalized image and the image without mirror reflection on the pixels. The two are subtracted based on the intensity logarithmic differential theory. If the result is zero, the adjacent pixels are diffuse reflection pixels. Otherwise, the two adjacent pixels may be mirror reflection pixels, noise, or two truncated pixels.

4. The method for detecting weld position in a strong noise environment according to claim 3, characterized in that: Since the chromaticity peak of noise can be regarded as a constant, while the chromaticity peak of the mirror reflection pixel changes, it is only necessary to determine whether the chromaticity peaks of the two pixels are equal. If they are the same, it is noise, and if they are different, at least one of them is a mirror reflection pixel. Repeating the operation on the pixels can obtain the mirror reflection pixels of the entire image. Since the chromaticity peak of the diffuse reflection component does not change with the change of image intensity, while the mirror reflection component changes with the image intensity, the mirror reflection component of the adjacent pixel with the smaller maximum chromaticity is reduced by changing the adjacent pixel with the smaller maximum chromaticity to make it equal to the adjacent pixel with the larger maximum chromaticity. By analogy, all pixels and mirror reflections in the surrounding mirror reflection area are iterated to achieve the purpose of eliminating mirror reflection.

5. The method for detecting weld position in a strong noise environment according to claim 4, characterized in that: The image after highlight removal is repaired through an image inpainting algorithm to repair the image that has lost some information due to the removal of highlight noise; at the same time, feature analysis and windowing can effectively remove the surrounding useless information, and median filtering is performed on the image to further remove noise and spot information in the image and output the image.

6. The method for detecting weld position in a strong noise environment according to claim 5, characterized in that: The specific operation of step S3 is: using the segmentation function in the segmentation network to segment the significant feature area of ​​the weld, and the specific steps are: inputting the image after highlight pixel filtering, performing information processing on the image with highlight pixels filtered out, outputting the spectral data of each part of the image, obtaining the regional wavelength when the spectral intensity value is maximum and taking out the average value of the wavelength; extracting the size of the spectral intensity value of each area at the average wavelength and calculating the size of the average spectral intensity value of each area; because the spectral intensity between the weld and the workpiece is different, the spectral intensity of the weld area is relatively low, and the welded workpiece is a stainless steel plate, and the spectral intensity of this type of material is relatively high during the welding process, so the size of the average spectral value is used as a benchmark, and the pixel points greater than the average spectral value are eliminated to obtain the weld area, that is, the area of ​​interest.

7. The method for detecting weld position in a strong noise environment according to claim 6, characterized in that: The GrabCut algorithm is executed on the selected region of interest to divide the image into four parts: background, foreground, possible background and possible foreground to form a segmentation mask; in order to obtain more accurate mask information, the background pixels and possible background pixels are assigned a value of 0, and the remaining foreground and possible foreground pixel information are assigned a value of 1 to obtain a mask with only foreground and background information; The foreground information can be replaced by a white part, and the rest can be replaced by a black part to display an image that separates the foreground and background into different areas. By multiplying the image with the original image, the weld image can be segmented as the foreground, thus obtaining a weld salient feature map.

8. The method for detecting weld position in a strong noise environment according to claim 7, characterized in that: The specific operation of step S4 is as follows: by performing morphological processing on the weld image after processing, edge images in different directions are obtained, wherein the Canny operator and the Sobel operator are used to process the image areas in different directions, because the Sobel operator is a gradient operator of oblique deviation points, and the gradient here refers to the difference in the strength of the effect of the weld edge displayed in the image during the edge detection process, and the direction of the gradient here is perpendicular to the direction of the edge; and the Canny operator is considered to be an operator that optimally approximates the signal-to-noise ratio to the positioning product; therefore, these two operators are selected to perform morphological processing on the weld image after processing, and the weld edge information of the better weld image is obtained, and the image obtained by segmentation is subjected to non-maximum suppression, wherein all the obtained windows are sorted according to the suppression value, and the window with the highest value is selected as the initial value, and the remaining windows are counted as the suppression window; and then the weld information SVM data set trained in YOLO is used for training, and the output window is the probability size of the weld and the overlap rate of the initial window and the suppression window, and the above operation is iterated cyclically to compare the overlap rate and the probability size of the window weld; The results of the suppression window are counted. If the value of the suppression window is not much different from the value of the training window and the scale is not much different, it can be determined that the repetition rate is large. The Canny operator and the Sobel operator are directly used to perform morphological processing on the input image. Different operators are applied to image areas in different directions for processing. After completion, weighted fusion is performed based on the images in different directions to obtain the weld edge image. If the window values ​​differ greatly and the window scales differ greatly, it is determined that the segmentation effect is not good, and the segmentation network is used to re-segment.