A method and system for detecting surface quality of stainless steel welds based on machine vision

Abstract

The present invention provides a method and system for inspecting the surface quality of stainless steel welds based on machine vision, belonging to the technical field of weld visual inspection technology. The method comprises the following steps: Step 1: Image acquisition; Step 2: Image preprocessing; Step 3: Determining whether the weld surface is qualified: The image characteristics of a good weld surface are: The outer annular area of ​​the weld is basically intact, without cracks or excess metal, and the inner area is flat and free of holes; Step 4: Defect detection: This includes defect detection when the weld surface is unqualified and defect detection when the weld surface is qualified. The system comprises an image acquisition device, a host, and a control unit. The image acquisition device includes a camera, a lens, a light source, and an assembly. The assembly includes a camera fixture, a light source fixture, a robotic arm interface, a top plate, a left plate, a right plate, a front plate, and a rear plate. The present invention overcomes the shortcomings of manual inspection methods, such as high subjectivity, easy fatigue, and the inability to quantify inspection results.

Description

Technical Field

[0001] The present invention relates to the technical field of weld spot visual inspection, and in particular to a method and system for inspecting the surface quality of stainless steel weld spots based on machine vision. Background Art

[0002] Resistance spot welding, a pressure welding process with high production efficiency, low heat input, and easy automation, has become a reliable manufacturing technology in the automotive and rail vehicle sectors. Currently, for high-volume resistance welding applications, there is a need for simple, non-destructive quality detection of spot weld defects and a reduction in the destructive sampling currently used to improve production efficiency and reduce costs. Common appearance inspection methods primarily rely on visual inspection, which can lead to misjudgments due to human fatigue, lack of responsibility, and lack of experience. Furthermore, there are management challenges associated with the large number of personnel involved, and a lack of standards for judging different evaluation results. In recent years, with the rapid development of machine vision and other technologies in the field of artificial intelligence, machine vision has become a popular research area in spot weld defect detection. Many researchers have applied machine vision technology to spot weld defect detection and achieved promising results. However, resistance spot welding is a complex process characterized by the interaction of multiple factors. Furthermore, the surface of stainless steel products is susceptible to environmental fluctuations, such as changes in lighting and target position, which complicate the extraction of weld information. This further complicates the application of machine vision inspection technology to identify surface defect types in welds. Summary of the Invention

[0003] In view of the deficiencies of the prior art, the present invention proposes a high-efficiency and high-precision method and system for detecting the surface quality of stainless steel spot welds based on machine vision.

[0004] The specific technical solutions are:

[0005] A method for detecting surface quality of stainless steel welds based on machine vision, the method comprising:

[0006] Step 1: Image acquisition;

[0007] Step 2: image preprocessing;

[0008] Step 3: Determine whether the solder joint surface is qualified:

[0009] The image characteristics of a good solder joint surface are: the outer ring area of ​​the solder joint is basically intact, without cracks and excess metal, and the inner area is flat and has no holes;

[0010] Step 4 Defect Detection:

[0011] It includes defect detection when the solder joint surface is unqualified and defect detection when the solder joint surface is qualified.

[0012] A stainless steel weld surface quality inspection system based on machine vision includes an image acquisition device, a host and a control unit. The system is characterized in that the image acquisition device includes a camera, a lens, a light source and an assembly part. The assembly part includes a camera fixture, a light source fixture, a robotic arm interface, a top plate, a left plate, a right plate, a front plate and a rear plate. The camera is fixed to the top plate by the camera fixture, and the light source is adjusted in height by the light source fixture. The host is used to control the image acquisition device to acquire images and receive images transmitted by the image acquisition device through the data transmission interface. The control unit is used to perform image preprocessing, determine whether the weld surface is qualified and perform defect detection operations.

[0013] The beneficial effects of the above technical solution of the present invention are as follows:

[0014] This invention, based on the inspection object and requirements, builds a machine vision-based weld defect detection method and system. This system includes hardware selection, fixture design, and image processing module development. It supports real-time acquisition and processing of stainless steel spot weld images to obtain relevant information such as weld defects and nugget diameter. Using the Otsu algorithm for image extraction, pixel-based calculation of the nugget area, and weld defect determination, this system overcomes the shortcomings of manual inspection methods, such as high subjectivity, fatigue, and the inability to quantify test results. It accurately measures the weld nugget size and effectively distinguishes between cracks, spatter, missing weld area, dents, and weld penetration defects, enabling real-time detection of stainless steel spot weld surface quality, demonstrating considerable practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Flow chart of the detection method of the present invention;

[0016] Figure 2 Schematic diagram of the image acquisition device of the present invention;

[0017] Figure 3 Schematic diagram of the light source illumination principle.

[0018] The reference numerals are as follows:

[0019] 1. Camera; 2. Lens; 3. Light source; 4. Camera fixture; 5. Light source fixture; 6. Robotic arm interface; 7. Top plate; 8. Left side plate; 9. Right side plate; 10. Front side plate; 11. Rear side plate. DETAILED DESCRIPTION

[0020] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0021] Example 1

[0022] This embodiment provides a method for detecting the surface quality of stainless steel spot welding based on machine vision, such as Figure 1 As shown, the details are as follows:

[0023] Step 1 Image acquisition

[0024] Before image acquisition, camera calibration is required to correct for lens distortion and improve the accuracy of camera results. Zhang's calibration method is used to obtain camera intrinsic parameters, which offers high accuracy and robustness. The Zhang calibration method follows these steps: First, prepare calibration images. Capture 20 calibration images by placing a calibration plate at different positions, ensuring that the plate occupies approximately two-thirds of the camera's field of view. Calibrate using OpenCV's built-in corner detection function to obtain the camera's intrinsic and extrinsic parameter coefficients and evaluate the calibration results. Finally, determine the position and brightness of the light source based on the specific situation. This can be adjusted using the fixture and controller. After completing these steps, the host computer can control the image acquisition device to capture images, which are then transmitted to the host computer via a USB 3.0 data transmission interface.

[0025] Step 2: Image preprocessing

[0026] Image preprocessing includes grayscale conversion, median filtering, and extraction of weld target objects based on the Otsu algorithm to reduce noise interference and minimize image processing computational complexity. Image acquisition, transmission, and processing can be subject to noise interference, resulting in the loss of important information, affecting the accuracy of subsequent feature extraction and image recognition, and causing numerous problems. First, grayscale conversion is performed to convert the three-channel image into a single-channel image. Next, a median filter with a 3×3 filter kernel is used for noise reduction. This method eliminates noise while preserving image details and preventing edge blurring. It is also time-efficient and suitable for filtering and denoising stainless steel spot weld images. Finally, the Otsu algorithm is used to process the image. An appropriate threshold is selected to maximize the variance between the foreground and background classes. The current threshold is then used as the optimal segmentation threshold to obtain the weld target. The weld target is the surface image of the weld formed by spot welding, which is a circular feature region. The interior of the weld is sunken, while the inner arc is sloped.

[0027] Step 3: Determine whether the solder joint surface is qualified

[0028] Image features of a good solder joint surface: The outer annular area of ​​the solder joint is basically intact, without cracks or excess metal, and the inner area is flat without holes.

[0029] The resulting solder joint target image is divided into two parts through template matching: solder joints with intact surfaces are grouped into one category, and solder joints with defects are grouped into another category. In actual testing, due to the changes in the distance between the image to be tested and the camera, and the influence of the template size factor, in order to obtain better matching results, adaptive size template matching is adopted. First, a solder joint with intact surface is selected as the template image, which is put into a loop and scaled by 10% each time. Template matching is then performed again, and finally, images of solder joints with intact surfaces at different scales (ROI areas) are obtained. All ROI areas are compared with the template image for similarity, and the matching image with the highest similarity is selected. At the same time, the best matching ratio is obtained. This is then used as a template for matching training with the image to be tested to obtain the ROI area, which is used to complete the classification of whether the solder joint surface is qualified.

[0030] Step 4 Defect Detection

[0031] (1) Detection of unqualified defects on the solder joint surface

[0032] First, the image of the unqualified solder joint surface is enhanced, and the image is corrected using the histogram equalization method. The grayscale value is "adjusted" using the cumulative distribution function to achieve contrast enhancement, which can highlight certain information in the image for better identification and facilitate the extraction of better feature information in the later stage.

[0033] Cracks: Weld cracks can be divided into internal cracks and peripheral cracks. Internal cracks are located in the internal area and are a thin straight line, while peripheral cracks are cracks surrounding the circumference of the weld. There is a difference in grayscale between the crack and the annular feature area, and the grayscale value of the crack is relatively small. Therefore, based on the difference in grayscale between the crack and the background, an appropriate threshold and number of cycles are selected. The threshold and number of cycles need to be determined manually after multiple attempts. The crack in the image is extracted using regional growth with automatically selected seeds. The extracted crack is a thin straight line with a background grayscale value of 255.

[0034] Spatter: The surface is rough in appearance and the image shows a line segment with width. Some parts are thick, some are thin, and some are broken. The length of the burr is generally much smaller than the specified distance between two welds. The weld moment can be determined according to the thickness of the component, and the length range of the burr line can be determined accordingly. Then, the Hough line transform is used to find a straight line that meets the preset length at the edge of the weld.

[0035] Missing solder joint area: The outer annular feature area is partially missing. The inner and outer circles of the solder joint are found based on circle detection using Hough transform. The circle detected when the solder joint area is missing is incomplete and has gaps.

[0036] Concave: The inside of the solder joint is sunken, while the outside is flat. The inside of the solder joint arc is sloped. The normal internal area is flat, and the pit that appears is a concave. The grayscale value of the concave part of the solder joint is different from the grayscale value of other internal areas. It is darker and has a smaller grayscale value on the image display. The difference in grayscale values ​​can be used to determine the appropriate threshold and the appropriate number of cycles. The threshold and number of cycles need to be determined manually after multiple attempts. The concave area is extracted by the region growing method with automatic seed selection.

[0037] Weld penetration: Weld penetration is the phenomenon of penetration in the solder joint, resulting in holes. The solder joint produced by weld penetration has uniform internal grayscale and no obvious bright areas. The appropriate threshold and number of cycles are determined by the change in the internal grayscale value of the solder joint. The threshold and number of cycles need to be determined manually after multiple attempts. The weld penetration defect can be distinguished by the region growing method with automatic seed selection.

[0038] (2) Qualified defect detection of solder joint surface

[0039] Cold welding: By comparison, it is found that the area of ​​the weld spot produced by cold welding is smaller than that of the weld spot produced under normal circumstances, that is, the diameter of the weld core is smaller than the standard. The standard is determined by looking up the spot welding quality standard based on the workpiece material and its thickness. According to the weld core calculation formula: Where D is the longest distance between the two ends of the weld nugget edge, and d is the shortest distance between the two ends of the weld nugget edge. The area of ​​the weld nugget can also be estimated by calculating the pixel points in the weld nugget area to obtain the relative diameter of the weld nugget. According to the diameter calculation formula of a circle: Where S is the calculated nugget area.

[0040] First, the qualified image of the solder joint surface is morphologically processed to eliminate noise. The background noise is first eroded and the foreground noise is expanded. To avoid affecting other places, the foreground noise can be removed by closing operation and the background noise can be removed by opening operation. Then the solder joint contour information is extracted, but its contour may be incomplete and its pixel information cannot form a complete circle. In order to complete its contour, a minimum circumscribed circle is made for it. Next, the solder joint is filled and its pixel area is calculated. Finally, the area of ​​the qualified solder joint is calculated from the molten core, and then compared with the actual area to obtain the conversion ratio. Then, the pixel area range of the qualified solder joint is determined according to the values ​​of grade A and grade BF of the quality standard. This is used as the threshold to judge whether the solder joint molten core diameter meets the standard. The one smaller than the range is a virtual solder joint and unqualified, and the one larger than the range is also unqualified.

[0041] Example 2

[0042] This embodiment provides a machine vision-based stainless steel spot weld surface quality inspection system, comprising an image acquisition device, a host computer, and a control unit. The image acquisition device is mounted on a robotic arm to capture spot weld images in real time, and the captured images are transmitted to the host computer via a USB 3.0 data transmission interface. The host computer processes the incoming image signals in real time through an image processing module, implements stainless steel surface quality inspection, displays the inspection results on an interactive interface, and saves the data or sends control signals.

[0043] The image acquisition device, such as Figure 2 As shown, it includes a camera 1, a lens 2, a light source 3, and an assembly part, wherein the assembly part includes a camera clamp 4, a light source clamp 5, a robotic arm interface 6, a top plate 7, a left side plate 8, a right side plate 9, a front side plate 10, and a rear side plate 11. The camera is fixed on the top plate through the camera clamp 4, and the light source can be adjusted in height through the light source clamp 5.

[0044] Image acquisition devices are the prerequisite for solder joint surface quality inspection. The quality of the images they obtain largely determines the subsequent work and system accuracy. Therefore, it is particularly important to design a reasonable acquisition device based on the inspection tasks and requirements. The image acquisition device mainly includes three imaging components (camera, lens, and light source). The selection rules are as follows:

[0045] (1) Camera selection: Cameras can be divided into CCD cameras and CMOS cameras according to the chip type. CCD cameras have the advantages of high resolution, high signal-to-noise ratio, and good image quality, while CMOS cameras have the advantages of fast response speed, good economy, and high integration. The camera field of view must be larger than the target size. The scanning rate and camera resolution are determined based on the detection speed and detection accuracy. The appropriate interface is selected based on the communication distance and transmission rate requirements, such as GigE, Camera Link, HDMI, and USB interfaces.

[0046] (2) Lens selection: Determine the sensor size and pixel size, magnification, and lens transfer function based on the target size and measurement accuracy; calculate the lens focal length and field of view angle based on the system size and working distance, combined with the magnification; determine the aperture size and operating wavelength based on the on-site lighting conditions; and determine other requirements such as distortion, depth of field, and camera interface.

[0047] (3) Selection of light source: In a machine vision system, the role of the light source is to highlight the features of the object being measured as much as possible and weaken the background features. To this end, it is necessary to select the appropriate type of light source according to the actual use scenario and design a matching illumination method. The defect detection requirements of the workpiece surface are analyzed. In order to solve the problem of stainless steel reflection, the influence of the commonly used light source model on the shooting effect of the stainless steel workpiece surface is examined, and finally a coaxial light source with good performance indicators is selected. The outstanding feature of the coaxial light source is its high contrast, which is very effective when detecting mirror surfaces, glossy surfaces, or when it is desired to distinguish by gloss differences. The illumination principle is as follows: Figure 3 As shown in the figure, a coaxial light source emits light from the side onto a half-reflecting mirror, which then reflects the light onto the workpiece. The mirror-reflected light can return to the camera, while the diffusely reflected light from uneven parts of the workpiece surface, such as solder joints, cannot be received, thus creating a contrast between the foreground and background of the workpiece.

[0048] The camera uses the acA5472-17um black-and-white CMOS area array camera from Basler, Germany, with a resolution of 5472×3648 and a USB 3.0 interface; the lens uses the C11-3520-12M-P from Basler, Germany, with a focal length of 35mm; the light source uses the COH6011-W from OPT, with a light-emitting surface size of 52mm and bright field direct frontal illumination; the light source controller uses the DPA2024E-4H from OPT, a 256-level adjustable 4-channel constant current digital controller, which can not only adjust the brightness of the lighting source at any time, but also makes the output voltage more stable and reliable; the assembly parts are all made of high-strength-ratio, low-density aluminum alloy material.

[0049] The image acquisition device, because in actual scenarios, the surface of stainless steel products is easily disturbed by changes in the external environment, must ensure that the camera, lens and light source are fixed in a line through assembly parts, and at the same time ensure that they are consistent with the horizontal plane of the product.

[0050] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for detecting the surface quality of stainless steel welds based on machine vision, characterized in that: The method comprises: Step 1: Image acquisition; Step 2: image preprocessing; Step 3: Determine whether the solder joint surface is qualified: The image characteristics of a good solder joint surface are: the outer ring area of ​​the solder joint is basically intact, without cracks and excess metal, and the inner area is flat and has no holes; Step 4 Defect Detection: Including defect detection when the solder joint surface is unqualified and defect detection when the solder joint surface is qualified; If the solder joint surface is unqualified, it is judged as unqualified. If the solder joint surface is qualified, defect detection is carried out under the condition that the solder joint surface is qualified. If the defect detection fails when the solder joint surface is qualified, the solder joint is judged to be unqualified. If the defect detection passes when the solder joint surface is qualified, the solder joint is judged to be qualified. Step 3 in the method specifically includes: Adaptive-size template matching is used. First, a solder joint with an intact surface is selected as a template image. This image is looped and scaled by 10% each time. Template image matching is then performed. Finally, images of solder joints with intact surfaces at different scales, namely the ROI regions, are obtained. All ROI regions are compared with the template image for similarity, and the matching image with the highest similarity is selected. The best matching ratio is also obtained. This image is then used as a template for matching training with the image to be tested, resulting in the ROI region, which is then used to distinguish whether the solder joint surface is qualified. The defect detection in step 4 of the method when the solder joint surface is unqualified specifically includes: Firstly, the image of the unqualified solder joint surface is enhanced and the image is corrected by using the histogram equalization method. The grayscale value is adjusted by using the cumulative distribution function to achieve contrast enhancement. Crack detection: Based on the grayscale difference between the crack and the background, an appropriate threshold and number of cycles are selected, and region growing with automatic seed selection is used to extract the crack in the image. The extracted crack is a thin straight line with a background grayscale value of 255. In the method, the defect detection in step 4 when the solder joint surface is unqualified further includes: Spatter detection: The distance between solder joints is determined based on the thickness of the component to determine the length range of the burr line. Then, the Hough line transform is used to find a line that meets the preset length at the edge of the solder joint. In the method, the defect detection in step 4 when the solder joint surface is unqualified further includes: Solder spot area missing detection: The inner and outer circles of the solder spot are found based on circle detection using Hough transform. The circle detected when the solder spot area is missing is incomplete and has gaps. In the method, the defect detection in step 4 when the solder joint surface is unqualified further includes: Concave detection: The grayscale value of the concave part of the solder joint is different from that of other internal areas. Based on the difference in grayscale values, the appropriate threshold and number of cycles are determined, and the concave area is extracted using the region growing method with automatic seed selection; Solder penetration detection: Determine the appropriate threshold and number of cycles based on the grayscale value changes inside the solder joint, and distinguish solder penetration defects through the region growing method with automatic seed selection; In the method, the defect detection in step 4 when the solder joint surface is qualified specifically includes: Cold soldering detection: First, morphological processing is performed on the qualified image of the solder joint surface to eliminate noise. The background noise is first eroded and the foreground noise is expanded. The foreground noise is first removed using a closing operation and the background noise is removed using an opening operation. Then, the solder joint contour information is extracted. To complete the contour, a minimum circumscribed circle is made, the solder joint is filled, and its pixel area is calculated. The pixel area of ​​the qualified solder joint is determined and used as a threshold to determine whether the solder joint nugget diameter meets the standard.

2. The method for detecting the surface quality of stainless steel welds based on machine vision according to claim 1, wherein: Step 1 in the method is specifically: The host computer controls the image acquisition device to acquire images, which are then transmitted to the host computer through the data transmission interface.

3. The method for detecting the surface quality of stainless steel welds based on machine vision according to claim 1, wherein: Step 2 in the method is specifically: Image preprocessing includes grayscale processing, median filtering and solder joint target extraction based on Otsu algorithm.

4. A machine vision-based stainless steel weld surface quality detection system for implementing the machine vision-based stainless steel weld surface quality detection method according to any one of claims 1 to 3, comprising an image acquisition device, a host computer, and a control unit, characterized in that: The image acquisition device includes a camera, a lens, a light source, and an assembly. The assembly includes a camera fixture, a light source fixture, a robotic arm interface, a top plate, a left plate, a right plate, a front plate, and a rear plate. The camera is fixed to the top plate by the camera fixture, and the light source is adjusted in height by the light source fixture. The host is used to control the image acquisition device to acquire images and receive images transmitted by the image acquisition device through the data transmission interface. The control unit is used to perform image preprocessing, determine whether the solder joint surface is qualified, and perform defect detection operations.

Images