A method, device and medium for detecting weld surface defects based on machine vision

Abstract

The present invention discloses a method, device and medium for detecting weld surface defects based on machine vision. The method comprises: obtaining a weld image of a boiler pipe collected by an industrial camera, preprocessing the image: image restoration, image denoising, image enhancement and image rotation correction; binarizing the image obtained by the preprocessing, and then performing morphological processing; and intercepting the ROI area in the image obtained after morphological processing by determining the position of the weld edge and the center line; intercepting the ROI area in the image obtained after morphological processing by determining the position of the weld edge and the center line; extracting feature information from the ROI area image; inputting the feature information into a pre-trained weld surface defect detection model for detection, and outputting the weld surface defect type; the structure of the weld surface defect detection model adopts a partial binary tree support vector machine. The present invention improves the recognition accuracy and efficiency of weld surface defects.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision, and in particular relates to a method, equipment and medium for detecting weld surface defects based on machine vision. Background Art

[0002] Domestically, the surface defect detection of thermal pipe welds in furnaces mainly relies on the most primitive manual regular detection method, which often causes false detection and missed detection, and workers work in dusty environments for a long time, which makes them prone to occupational diseases. Machine vision is a non-contact, non-destructive automatic detection technology with the advantages of safety, reliability, long-term operation in harsh environments, and high detection efficiency. At present, there are many methods for detecting weld defects using machine vision, such as neural network algorithms and support vector machine algorithms. Through the debugging of defect recognition models, it is found that the effective extraction of feature areas will directly affect the recognition time and accuracy of the model.

[0003] Therefore, it is particularly important to extract the area of ​​the collected weld image and eliminate irrelevant background.

[0004] Patent No.: CN201810709776.9 discloses a weld defect recognition method based on an improved convolutional neural network. However, this method does not include weld area extraction. The collected images can only be manually cropped and the convolutional neural network is used to complete defect recognition, which has extremely low work efficiency. CN201711043638.3 discloses a method for detecting surface defects of ship plates based on machine vision. The method uses a global threshold segmentation method to extract the area of ​​interest from the preprocessed ship plate image to be detected, and obtains multiple sub-areas. However, the light inside the boiler is dim and dusty. The global threshold segmentation method cannot effectively identify the pipeline weld area, and the weld area cannot be extracted from the image alone. Patent No.: CN201910396121.5 discloses a method and system for extracting feature areas of cylindrical longitudinal welds based on structured light. The method uses two lasers to generate laser stripes on both sides, and performs row projection operation on the pixels of the weld image to find the row with the smallest pixel point, select rows with a fixed width h upward and downward, and finally realize the tracking of the weld area. This method has two disadvantages: First, it can only realize local tracking of the weld area, and cannot extract the entire weld area; second, the above method extracts the area through the minimum pixel value, and due to the high dust inside the boiler, many "snowflakes" will appear when shooting the pipeline image, so the area corresponding to the minimum pixel value is not the pipeline weld area.

[0005] In summary, existing devices and methods cannot accurately identify and extract weld feature areas, which results in limited accuracy in weld surface defect detection. Summary of the invention

[0006] In order to solve the accuracy problem of weld surface defect detection in the prior art, the present invention proposes a weld surface defect detection method, equipment and medium based on machine vision, which realizes weld area division and extraction by using the method of gray value accumulation and then averaging, and then completes defect detection by extracting weld area features, thereby reducing the workload of workers in inspecting weld defects in furnace pipelines and improving detection efficiency.

[0007] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0008] A method for detecting weld surface defects based on machine vision comprises the following steps:

[0009] Step 1: Obtain the weld image of the boiler pipe collected by the industrial camera and preprocess the image. The preprocessing mainly includes: image restoration, image denoising, image enhancement and image rotation correction;

[0010] Step 2, binarizing the image obtained by the preprocessing, and then performing morphological processing; and intercepting the ROI area in the image obtained after the morphological processing by determining the position of the weld edge and the center line;

[0011] Step 3, by determining the position of the weld edge and center line, intercepting the ROI area in the image obtained after morphological processing;

[0012] Step 4, extracting feature information from the ROI area image;

[0013] Step 5, input the feature information into a pre-trained weld surface defect detection model for detection, and output the weld surface defect type; the pre-trained weld surface defect detection model adopts a partial binary tree support vector machine and is trained using feature information samples with defect type labels.

[0014] Furthermore, Wiener filtering is used for image restoration; adaptive mean filtering is used for image denoising; and Retinex method is used for image enhancement.

[0015] Furthermore, Hough transform is used to detect straight lines and calculate the weld inclination angle θ. Image rotation correction is performed based on the weld inclination angle θ, specifically:

[0016] Step A1, using Hough line detection to identify all straight lines in the weld image;

[0017] Step A2, calculating the lengths of all straight lines, and removing straight lines whose lengths are shorter than a preset length value;

[0018] Step A3, calculating the average slope K of all remaining straight lines, and calculating the corresponding inclination angle θ based on the average slope K;

[0019] Step A4, rotating and correcting the weld image according to the tilt angle θ.

[0020] Furthermore, the method for intercepting the ROI area is:

[0021] Step B1, the grayscale average value of each row of the image obtained by morphological processing;

[0022] Step B2, in the range of the [δ, L2-δ]th row of the image, traverse and find the minimum value C1 of all grayscale average values, and record the corresponding behavior as the x1th row of the image; where L2 is the total number of pixel rows in the image;

[0023] Step B3, in the image and In the range of the row interval, traverse and find the minimum value C2 of all grayscale average values, and record the corresponding row as the x2th row of the image; where L0 is the number of pixel rows occupied by the weld width, which is a preset empirical value;

[0024] Step B4, determine that the row where the center line of the weld is located is x0=(x1+x2) / 2, and the rows where the upper and lower edges of the weld are located are: [x0-L0, x0+L0];

[0025] Step B5, extract the area [x0-L0, x0+L0] between the upper and lower edges of the weld from the weld image, and obtain the ROI area image.

[0026] Furthermore, the feature information extracted from the ROI region image includes: image texture features, defect location and area, and the perimeter, rectangularity, and circularity of the defect contour.

[0027] Further, defect types include: undercut, pores and cracks.

[0028] Furthermore, a weld surface defect detection model with a partial binary tree support vector machine structure is used to identify the weld surface defect type as follows:

[0029] The image texture features are input into the first support vector machine SVM1 to determine whether the weld has defects;

[0030] The position and area of ​​the defect are input into the second support vector machine SVM2 to determine whether there is an undercut defect in the weld;

[0031] The circularity and rectangularity of the defect contour are input into the third support vector machine SVM3 to determine whether there is a porosity defect in the weld;

[0032] The position and area of ​​the defect as well as the perimeter, circularity and rectangularity of the defect contour are input into the fourth support vector machine SVM4 to determine whether there is a crack defect in the weld.

[0033] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor implements the method described in any one of the above technologies.

[0034] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of the above technologies.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] 1. The detection results of the method of the present invention are reliable and can identify a variety of defects. After experiments, the accuracy rate is as high as over 90%;

[0037] 2. The method of the present invention only takes 40ms from data collection to data output, which greatly improves the detection efficiency while ensuring the accuracy of the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of weld defect identification according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of a weld after enhancement according to an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of Hough line detection according to an embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram of a weld after rotation according to an embodiment of the present invention;

[0042] Figure 5 This is a schematic diagram of averaging after grayscale accumulation according to an embodiment of the present invention;

[0043] Figure 6 A schematic diagram of determining the edge position of a weld according to an embodiment of the present invention;

[0044] Figure 7 This is a schematic diagram of ROI region capture according to an embodiment of the present invention.

[0045] Figure 8 A schematic diagram of defect recognition results according to an embodiment of the present invention; DETAILED DESCRIPTION

[0046] The following is a detailed description of an embodiment of the present invention. This embodiment is based on the technical solution of the present invention, and provides a detailed implementation method and a specific operation process to further explain the technical solution of the present invention.

[0047] One embodiment of the present invention provides a method for detecting weld surface defects based on machine vision, referring to Figure 1 As shown, the following steps are included:

[0048] Step 1: Obtain the weld image of the boiler pipe collected by the industrial camera and preprocess the image. The preprocessing mainly includes: image restoration, image denoising, image enhancement and image rotation correction;

[0049] Among them, Wiener filtering is used for image restoration; adaptive mean filtering is used for image denoising; Retinex method is used for image enhancement. After enhancement, the weld image is as follows Figure 2 shown.

[0050] Hough transform is used to detect straight lines and calculate the weld inclination angle θ. Image rotation correction is performed based on the weld inclination angle θ, specifically:

[0051] Step A1, using Hough line detection to identify all straight lines in the weld image, such as Figure 3 As shown;

[0052] Step A2, calculating the lengths of all straight lines, and removing straight lines whose lengths are shorter than a preset length value;

[0053] Step A3, calculating the average slope K of all remaining straight lines, and calculating the corresponding inclination angle θ based on the average slope K;

[0054] Step A4, with the dot as the center, the weld image is rotated and corrected according to the tilt angle θ. The rotated weld image is as follows: Figure 4 shown.

[0055] The preset length value in this embodiment can be preset as an empirical ratio value of the image size width. For example, if the original image length is 200 mm and the empirical ratio is 50%, the preset length value is 80 mm.

[0056] Step 2: Binarize the preprocessed image and then perform morphological processing.

[0057] Specifically, the K-means clustering method can be used to classify the foreground and background in the image to achieve image binarization; the morphological processing specifically uses an open operation followed by a close operation.

[0058] Step 3, by determining the position of the weld edge and center line, intercepting the ROI area in the image obtained after morphological processing; specifically:

[0059] Step B1, the grayscale average of each row of the image obtained by morphological processing is calculated. Figure 5 As shown;

[0060] Step B2, in the range of the [δ, L2-δ]th row of the image, traverse and find the minimum value C1 of all grayscale average values, and record the corresponding behavior as the x1th row of the image; where L2 is the total number of pixel rows in the image;

[0061] Step B3, in the image and Within the range of rows, traverse to find the minimum value C2 of all grayscale averages, and record the corresponding behavior as the x2th row of the image. Figure 6 As shown; wherein L0 is the number of pixel rows occupied by the weld width, which is a preset empirical value; wherein δ is a preset weld edge background estimate, which is preset to 50 in this embodiment.

[0062] Step B4, determine that the row where the center line of the weld is located is x0=(x1+x2) / 2, and the rows where the upper and lower edges of the weld are located are: [x0-L0, x0+L0];

[0063] Step B5, the area between the upper and lower edges of the weld [x0-L0, x0+L0] is cut out from the weld image, that is, the ROI area image is obtained, such as Figure 7 shown.

[0064] After the above image preprocessing and ROI area extraction, the number of image pixels is reduced, which can improve the efficiency of subsequent defect type identification. In addition, irrelevant background areas are eliminated, and only the weld area is retained. The subsequent extracted feature information is also more accurate, thereby improving the accuracy of defect type identification.

[0065] Step 4: extract feature information from the ROI area image, including: image texture features, defect location and area, and the perimeter, rectangularity, and circularity of the defect outline.

[0066] Among them, the LBP operator is used to extract image texture features, the ContourArea and ContourPerimeter functions are used to extract the area and contour perimeter of the weld defect respectively, the image moment is used to extract the defect location information, and the following formula is used to extract the rectangularity D and circularity C of the defect:

[0067] D=A W / A,

[0068] In the formula, A Wis the minimum circumscribed rectangular area of ​​the defect, A is the defect area; P is the perimeter of the defect area. If the defect area is circular, the C value is greater than 1. If it is other shapes, C is between (0,1).

[0069] Step 5, input the feature information into a pre-trained weld surface defect detection model for detection, and output the weld surface defect type; the pre-trained weld surface defect detection model adopts a partial binary tree support vector machine and is trained using feature information samples with defect type labels.

[0070] The types of weld surface defects involved in this embodiment include: undercut, pores and cracks.

[0071] The partial binary tree support vector machine is composed of a plurality of support vector machines. The present invention is implemented with respect to the weld surface defect type and is configured to be composed of four support vector machines.

[0072] like Figure 1 As shown, the input of the first support vector machine is the image texture feature, and the output is a label indicating whether there is a defect in the weld; the input of the second support vector machine is the position and area of ​​the defect, and the output is a label indicating whether there is an undercut defect in the weld; the input of the third support vector machine is the circularity and rectangularity of the defect contour, and the output is a label indicating whether there is a porosity defect in the weld; the input of the fourth support vector machine is the position and area of ​​the defect, as well as the circumference, circularity and rectangularity of the defect contour, and the output is a label indicating whether there is a crack defect in the weld.

[0073] When the above-mentioned partial binary tree support vector machine is trained with labeled training samples, the weld surface defect detection model obtained can be used to detect defects on weld surfaces with unknown defects and unknown defect types. In this embodiment, regardless of whether the previous support vector machine detects a defect or detects a corresponding defect, the remaining support vector machines are used to continue to perform other defect detections based on the corresponding feature information. Therefore, each support vector machine outputs a detection result accordingly. By combining the four detection results, it is finally determined whether there are several defects on the weld surface at the same time. The recognition results of this embodiment are compared with the true value. Figure 8 shown.

[0074] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor implements the weld surface defect detection method based on machine vision described in the above embodiment.

[0075] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for detecting weld surface defects based on machine vision described in the above-mentioned embodiment is implemented.

[0076] The above embodiments are preferred embodiments of the present application. Ordinary technicians in this field can also make various changes or improvements on this basis. Without departing from the overall concept of the present application, these changes or improvements should fall within the scope of protection required by the present application.

Claims

1. A weld surface defect detection method based on machine vision, characterized in that: The following steps are involved: Step 1: Obtain the weld image of the boiler pipe collected by the industrial camera and preprocess the image. The preprocessing mainly includes: image restoration, image denoising, image enhancement and image rotation correction; Step 2, binarizing the image obtained by the preprocessing, and then performing morphological processing; and intercepting the ROI area in the image obtained after the morphological processing by determining the position of the weld edge and the center line; The method for extracting the ROI area is as follows: Step B1, the grayscale average value of each row of the image obtained by morphological processing; Step B2, in the range of the [δ, L2-δ]th row of the image, traverse and find the minimum value C1 of all grayscale average values, and record the corresponding behavior as the x1th row of the image; where L2 is the total number of pixel rows in the image; Step B3, in the image and In the range of the row interval, traverse and find the minimum value C2 of all grayscale average values, and record the corresponding row as the x2th row of the image; where L0 is the number of pixel rows occupied by the weld width, which is a preset empirical value; Step B4, determine that the row where the center line of the weld is located is x0=(x1+x2) / 2, and the rows where the upper and lower edges of the weld are located are: [x0-L0, x0+L0]; Step B5, extracting the area [x0-L0, x0+L0] between the upper and lower edges of the weld from the weld image, thus obtaining a ROI area image; Step 3, by determining the position of the weld edge and center line, intercepting the ROI area in the image obtained after morphological processing; Step 4, extracting feature information from the ROI area image; Step 5, inputting the feature information into a pre-trained weld surface defect detection model for detection, and outputting the weld surface defect type; the pre-trained weld surface defect detection model adopts a partial binary tree support vector machine and is trained using feature information samples with defect type labels; Among them, defect types include: undercut, pores and cracks; The weld surface defect detection model with a partial binary tree support vector machine structure is used to identify the weld surface defect type as follows: The image texture features are input into the first support vector machine SVM1 to determine whether the weld has defects; The position and area of ​​the defect are input into the second support vector machine SVM2 to determine whether there is an undercut defect in the weld; The circularity and rectangularity of the defect contour are input into the third support vector machine SVM3 to determine whether there is a porosity defect in the weld; The position and area of ​​the defect as well as the perimeter, circularity and rectangularity of the defect contour are input into the fourth support vector machine SVM4 to determine whether there is a crack defect in the weld.

2. The weld surface defect detection method based on machine vision according to claim 1 is characterized in that: Wiener filtering is used for image restoration; adaptive mean filtering is used for image denoising; and Retinex method is used for image enhancement.

3. The weld surface defect detection method based on machine vision according to claim 1, characterized in that: Hough transform is used to detect straight lines and calculate the weld inclination angle θ. Image rotation correction is performed based on the weld inclination angle θ, specifically: Step A1, using Hough line detection to identify all straight lines in the weld image; Step A2, calculating the lengths of all straight lines, and removing straight lines whose lengths are shorter than a preset length value; Step A3, calculating the average slope K of all remaining straight lines, and calculating the corresponding inclination angle θ based on the average slope K; Step A4, rotating and correcting the weld image according to the tilt angle θ.

4. The weld surface defect detection method based on machine vision according to claim 1, characterized in that: The feature information extracted from the ROI area image includes: image texture features, defect location and area, and the perimeter, rectangularity, and circularity of the defect outline.

5. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the computer program is executed by the processor, the processor implements the method according to any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images