Steel structure weld defect detection method and system based on machine vision

Abstract

The present application belongs to the technical field of visual detection, and provides a steel structure weld defect detection method and system based on machine vision, which comprises: using a coaxial concentric multi-channel ring light source to perform time sequence alternating imaging on a moving weld; respectively turning on an inner ring light source and an outer ring light source, and collecting two weld images in a short time interval. By extracting the highlight feature regions of the two images, coordinate unification and shape comparison are completed, the area expansion and contraction variation coefficient is calculated, and the pseudo area generated by the slag microlens effect is accurately identified; then a dynamic mask is generated based on the pseudo area and interference is shielded, and a weld image without pseudo is obtained; finally, the pure image is input into a CNN defect recognition model to accurately identify real surface defects such as pores and undercut.

Description

Technical Field

[0001] This invention belongs to the field of visual inspection technology, specifically a method and system for detecting defects in steel structure welds based on machine vision. Background Technology

[0002] With the development of automation technology, non-contact inspection methods based on machine vision have gradually replaced traditional manual visual inspection. Existing machine vision inspection systems typically use industrial cameras with conventional single illumination sources to acquire two-dimensional surface images of welds, and then use traditional image processing algorithms (such as edge detection operators like Canny and Sobel) or deep learning-based convolutional neural networks (CNNs) to extract and classify defect features in the images.

[0003] However, in the complex working conditions of multi-pass welding between layers, the existing visual inspection technology faces an extremely severe and difficult-to-overcome technical bottleneck: the extremely high misjudgment rate caused by optical artifact interference.

[0004] Specifically, during interlayer inspection, tiny, irregularly shaped spatter or extremely thin glassy slag often remains on the edges and surfaces of the previous weld. This glassy slag is typically translucent and has a microscopic curved surface structure similar to a plano-convex lens. Under the illumination of high-brightness light sources (especially single-angle light sources) in existing machine vision systems, this translucent glassy slag produces a strong microlens effect (i.e., refraction and convergence of light) or strong specular reflection.

[0005] This complex optical phenomenon causes glassy slag to appear in two-dimensional images as extremely sharp dark spots, locally overexposed bright spots, or complex topological patterns with alternating light and dark areas. Traditional edge detection operators rely solely on the two-dimensional gray-level gradient of the image for calculation, making it easy to mistake these sharp boundaries generated by optical refraction for the edges of physical defects. Furthermore, conventional convolutional neural networks (CNNs), due to their input being only a two-dimensional pixel matrix from a single viewpoint, can only learn the texture and gray-level distribution features of a plane, completely lacking the ability to perceive and discriminate the physical optical properties (transmittance, refractive index) of the target object.

[0006] Therefore, existing visual algorithms are mathematically incapable of distinguishing between geometric shadows caused by real physical depressions (such as porosity and undercut) and optical artifacts caused by light refracted by translucent slag. This makes it extremely easy for the system to misclassify these harmless slag artifacts as serious porosity or undercut defects (i.e., over-detection or false positives in industry). This persistently high false alarm rate forces a significant amount of manual verification on-site, severely limiting the efficiency and reliability of automated welding inspection.

[0007] Therefore, there is an urgent need for a novel detection mechanism that can eliminate the interference of slag artifacts from the physical and optical root causes. Based on the aforementioned technical challenges, this invention provides a machine vision-based method and system for detecting weld defects in steel structures. Summary of the Invention

[0008] In order to overcome the shortcomings of the prior art, at least one technical problem raised in the background art is solved.

[0009] The technical solution adopted by this invention to solve its technical problem is:

[0010] On the one hand, the present invention provides a machine vision-based method for detecting defects in steel structure welds, including:

[0011] S1: A vision acquisition component is set above the interlayer weld to be inspected. The vision acquisition component includes an industrial camera and a concentric multi-channel ring light source set coaxially with the industrial camera. The concentric multi-channel ring light source includes an inner ring light source with a first radius and an outer ring light source with a second radius.

[0012] S2: During the movement of the interlayer weld, the vision acquisition component is controlled to perform time-sequential alternating imaging: at the first acquisition moment, the inner ring light source is turned on to acquire the first weld image, and at the second acquisition moment after a preset time interval, the outer ring light source is turned on to acquire the second weld image.

[0013] S3: Extract the first bright feature region of the first weld image and the second bright feature region of the second weld image, perform physical reference coordinate system-1 and morphological comparison on the first bright feature region and the second bright feature region, and obtain the area expansion and contraction variation coefficient.

[0014] S4: Determine whether there is a slag artifact region between the first and second bright feature regions at the same physical coordinate position based on the area expansion and contraction variation coefficient.

[0015] S5: Generate a dynamic mask based on the slag artifact region, and use the dynamic mask to shield the slag artifact region in the first weld image or the second weld image to obtain an artifact-free weld image.

[0016] S6: Construct a defect recognition model, input the artifact-free weld image into the defect recognition model, identify and output the real surface defects of the interlayer weld.

[0017] Preferably, the inner ring light source of the first radius and the outer ring light source of the second radius are specifically configured as follows:

[0018] The first radius of the inner ring light source is set between 15 mm and 85 mm, and the second radius of the outer ring light source is preferably set between 80 mm and 400 mm.

[0019] Preferably, the specific process of acquiring the first weld image is as follows:

[0020] At the first acquisition moment, the vision controller outputs the first trigger signal, which controls the inner ring light source to light up and the outer ring light source to turn off, triggering the industrial camera to perform global exposure, thereby acquiring the first weld seam image.

[0021] Preferably, the specific process of acquiring the second weld image is as follows:

[0022] At the second acquisition time after a preset time interval, the vision controller outputs a second trigger signal to turn off the inner ring light source and turn on the outer ring light source, triggering the industrial camera to perform a second global exposure, thereby acquiring the second weld seam image.

[0023] Preferably, the specific process for extracting the first highlighted feature region in the first weld image and the second highlighted feature region in the second weld image is as follows:

[0024] For the first weld seam image, the grayscale distribution information of all pixels inside the first weld seam image is traversed, a high-brightness grayscale threshold is set, and the set of continuous pixels in the first weld seam image with grayscale values ​​greater than the high-brightness grayscale threshold is extracted to form one or more independent connected components, which are defined as the first high-brightness feature region. In the second weld seam image, the set of continuous pixels with grayscale values ​​greater than the high-brightness grayscale threshold is extracted and defined as the second high-brightness feature region.

[0025] Preferably, the specific process for obtaining the area expansion / contraction variation coefficient includes:

[0026] Based on the known moving speed of the visual acquisition component and the preset time interval, the physical displacement is calculated as: preset time interval × known moving speed of the visual acquisition component. Combined with the pixel equivalent, the physical displacement is converted into a pixel translation compensation vector.

[0027] Using a pixel translation compensation vector, the coordinates of the second highlight feature region are reverse-translated and corrected so that the corrected second highlight feature region and the first highlight feature region are in a unified physical reference coordinate system. The total number of effective highlight pixels contained in the first highlight feature region is counted and recorded as the first area value, and the total number of effective highlight pixels contained in the second highlight feature region is recorded as the second area value. The ratio of the second area value to the first area value is calculated as the area expansion and contraction variation coefficient.

[0028] Preferably, the specific process of determining whether there is a slag artifact region between the first bright feature region and the second bright feature region at the same physical coordinate position based on the area expansion and contraction variation coefficient is as follows:

[0029] Calculate the area expansion and contraction variation coefficient between the second highlight feature region and the first highlight feature region, and compare the area expansion and contraction variation coefficient with the preset scaling ratio threshold.

[0030] If the coefficient of variation of area expansion and contraction between the first and second highlighted feature regions is greater than or equal to the scaling ratio threshold, it is determined that the highlighted region has undergone radial scaling and is marked as a slag artifact region.

[0031] Preferably, the specific process for obtaining the artifact-free weld image is as follows:

[0032] A dynamic mask is generated based on the slag artifact region. The generation rule of the dynamic mask is as follows: in the pixel matrix of the dynamic mask, all pixels belonging to the slag artifact region are assigned mask blocking values, and all remaining pixels in the pixel matrix of the dynamic mask except for the coordinates of the slag artifact region are assigned mask retention values. The first weld image or the second weld image is selected as the base image to be processed. The dynamic mask and the base image to be processed are directly overwritten and replaced with pixel values ​​to obtain the artifact-free weld image.

[0033] Preferably, the specific process for identifying and outputting the actual surface defects of the interlayer weld is as follows:

[0034] Construct an initial defect recognition model based on a convolutional neural network (CNN) and build a dedicated training sample set. Input the training set into the initial defect recognition model in batches, output the prediction results through forward propagation, and use the loss function to calculate the error between the prediction results and the true labels. Continuously update the weights and bias parameters in the network layers through the backpropagation algorithm until the loss function converges to the preset range to obtain the defect recognition model.

[0035] The artifact-free weld image is preprocessed with tensor quantization and used as a standard input tensor, which is then directly input into the actual surface defects identified in the defect recognition model.

[0036] On the other hand, the present invention provides a machine vision-based steel structure weld defect detection system, comprising:

[0037] Visual acquisition module: A visual acquisition component is set above the interlayer weld to be inspected. The visual acquisition component includes an industrial camera and a concentric multi-channel ring light source set coaxially with the industrial camera. The concentric multi-channel ring light source includes an inner ring light source with a first radius and an outer ring light source with a second radius.

[0038] Timing Alternating Imaging Control Module: During the movement of the interlayer weld, the vision acquisition component is controlled to perform timing alternating imaging: at the first acquisition moment, the inner ring light source is turned on to acquire the first weld image, and at the second acquisition moment after a preset time interval, the outer ring light source is turned on to acquire the second weld image.

[0039] Highlight feature extraction and comparison analysis module: Extract the first high-brightness feature region of the first weld image and the second high-brightness feature region of the second weld image, perform physical reference coordinate system-1 and morphological comparison on the first high-brightness feature region and the second high-brightness feature region, and obtain the area expansion and contraction variation coefficient.

[0040] Slag artifact detection module: Determines whether there is a slag artifact region between the first and second bright feature regions at the same physical coordinate position based on the area expansion and contraction variation coefficient;

[0041] Dynamic mask generation and artifact shielding module: Generates a dynamic mask based on the slag artifact region, and uses the dynamic mask to shield the slag artifact region in the first weld image or the second weld image to obtain an artifact-free weld image.

[0042] Weld Defect Recognition Module: Constructs a defect recognition model, inputs artifact-free weld images into the defect recognition model, identifies and outputs the true surface defects of interlayer welds.

[0043] The beneficial effects of this invention are as follows:

[0044] This invention employs a coaxial, concentric, multi-channel ring light source with alternating temporal imaging. Combined with high-brightness feature comparison and area expansion / contraction coefficient determination, it can accurately identify and eliminate slag artifacts from a physical optics perspective, significantly reducing the false detection rate of weld defects and avoiding misjudging harmless slag as real defects. By using a dynamic mask to shield artifact areas, a clean weld image is obtained, allowing the defect recognition model to focus more on extracting real defect features, thus improving defect detection accuracy and sensitivity. The overall solution is suitable for online inspection scenarios of interlayer welds, eliminating the need for manual secondary verification, effectively improving the efficiency and reliability of automated inspection of steel structure welds, and demonstrating strong engineering practicality. Attached Figure Description

[0045] The invention will now be further described with reference to the accompanying drawings.

[0046] Figure 1 This is a flowchart of the steps of the steel structure weld defect detection method based on machine vision of the present invention;

[0047] Figure 2 This is a system module diagram of the steel structure weld defect detection system based on machine vision according to the present invention. Detailed Implementation

[0048] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0049] Example 1: As Figure 1As shown in the embodiment of the present invention, the method and system for detecting defects in steel structure welds based on machine vision includes:

[0050] S1: A vision acquisition component is set above the interlayer weld to be inspected. The vision acquisition component includes an industrial camera and a concentric multi-channel ring light source coaxially arranged with the industrial camera. The concentric multi-channel ring light source includes at least an inner ring light source with a first radius and an outer ring light source with a second radius.

[0051] In some embodiments, the vision acquisition component is installed at the end of a welding machine or welding robot, so that the vision acquisition component moves synchronously above the interlayer weld seam along with the welding torch during the process of multi-layer and multi-pass welding of thick plates.

[0052] The overall structure of the vision acquisition component is based on the line connecting the center of the lens of the industrial camera as the main optical axis of the camera; in the detection state, the posture of the vision acquisition component is adjusted so that the main optical axis of the industrial camera is perpendicular to the surface of the interlayer weld to be detected.

[0053] The concentric multi-channel ring light source is horizontally fixed around the lens of the industrial camera, with the light-emitting surface of the concentric multi-channel ring light source facing the surface of the interlayer weld; the geometric center of the concentric multi-channel ring light source coincides with the main optical axis of the industrial camera (i.e., coaxially set).

[0054] To produce differentiated optical refraction effects, the inner ring light source and the outer ring light source have specific geometric constraints in their spatial arrangement:

[0055] The inner ring light source is arranged in a ring array and has a relatively small first radius; when the light emitted by the inner ring light source shines on the surface of the interlayer weld, a first incident angle is formed between the light and the main optical axis of the camera.

[0056] The outer ring light source is arranged in a ring array around the inner ring light source and has a relatively large second radius (i.e., the second radius is larger than the first radius); when the light emitted by the outer ring light source shines on the surface of the interlayer weld, a second incident angle is formed between the light and the main optical axis of the camera;

[0057] To ensure that the first and second incident angles can stably excite the microlens refraction variation of the glassy slag in practical engineering applications, the hardware spatial dimensions of the vision acquisition component must meet specific geometric proportions.

[0058] Under the premise that the preset detection object distance (i.e., the vertical distance from the optical center of the industrial camera lens to the surface of the interlayer weld) remains constant, the magnitudes of the first incident angle and the second incident angle are completely determined by the first radius and the second radius;

[0059] Since the second radius is larger than the first radius, at the same detection object distance, the second incident angle will inevitably be significantly larger than the first incident angle.

[0060] In practical steel structure welding inspection engineering configurations, preferably, the inspection object distance is set between 80 mm and 150 mm; based on this inspection object distance range:

[0061] In order to achieve near-vertical illumination at a small angle by the inner ring light source, the first incident angle is set between 10 degrees and 30 degrees; correspondingly, the first radius of the inner ring light source is preferably set between 15 mm and 85 mm.

[0062] In order to achieve large-angle oblique side illumination from the outer ring light source, the second incident angle is set between 45 degrees and 70 degrees; correspondingly, the second radius of the outer ring light source is preferably set between 80 mm and 400 mm.

[0063] The physical significance of this spatial layout and angle configuration lies in:

[0064] When facing the glassy slag that remains on the interlayer weld in the shape of a semi-transparent plano-convex lens, the first incident angle in the small angle range of 10 to 30 degrees can allow light to penetrate the central area of ​​the slag in a nearly perpendicular manner, and under the effect of refraction, it converges into a concentrated high-brightness spot on the target surface of the industrial camera.

[0065] The second incident angle, which is in the wide range of 45 to 70 degrees, causes the light to strike the curved surface of the glassy slag edge at a very high angle. When this incident light passes through the edge of the glassy slag, it will cause severe spherical aberration and refraction deviation, resulting in sharp divergence of refracted light rays or severe focus shift. This will form a caustic ring with a significantly enlarged area or a diffuse spot with a sudden drop in brightness on the target surface of the industrial camera. This drastic change in optical morphology caused by the clear difference in radius is the core physical basis for subsequent algorithms to distinguish slag artifacts from real physical depressions (true defects).

[0066] In other embodiments, when the object distance H = 100 mm:

[0067] If the first angle of incidence is required According to The first radius R1 is approximately at arrive The preferred range is 15 mm to 85 mm, which fully covers this theoretical value while allowing for engineering margins.

[0068] If the second angle of incidence is required The second radius R2 is approximately at arrive Between; the preferred range is 80 mm to 400 mm, also providing perfect coverage;

[0069] S2: During the relative movement of the interlayer weld seam, the vision acquisition component is controlled to perform time-sequential alternating imaging: at the first acquisition moment, the inner ring light source is turned on and the first weld seam image is acquired; at the second acquisition moment after a preset time interval, the outer ring light source is turned on and the second weld seam image is acquired.

[0070] In some embodiments, step S2 is specifically implemented to acquire two consecutive images of the same physical area under illumination by light sources at different incident angles, in order to capture transient optical variation features; specific implementation details are as follows:

[0071] During the continuous relative movement of the vision acquisition component along the extension direction of the interlayer weld, the vision controller outputs a high-frequency hardware trigger signal to control the strobe of the concentric multi-channel ring light source to be strictly synchronized with the exposure action of the industrial camera.

[0072] To ensure that the optical characteristics of different incident angles are strictly physically isolated in the time dimension and to avoid optical path aliasing, the first acquisition time and the second acquisition time together constitute a complete imaging cycle; within this imaging cycle, the inner ring light source and the outer ring light source adopt a non-overlapping, sequential lighting interaction logic;

[0073] Specifically, when the visual acquisition component moves above the area to be tested, the above-mentioned interaction logic is activated:

[0074] At the first acquisition moment, the vision controller outputs the first trigger signal to control the inner ring light source to light up instantly. At this time, the outer ring light source is forced to remain in the off state, and the industrial camera is simultaneously triggered to perform global exposure, thereby acquiring the first weld seam image.

[0075] The first weld image records the optical reflection and refraction patterns of the interlayer weld surface under single illumination at a small first incident angle;

[0076] At the second acquisition moment after a preset time interval, the vision controller outputs a second trigger signal to control the inner ring light source to turn off instantly and simultaneously turn on the outer ring light source instantly, while triggering the industrial camera to perform a second global exposure, thereby acquiring the second weld seam image;

[0077] The second weld image records the optical reflection and refraction patterns of the same interlayer weld surface under separate illumination at a large second incident angle;

[0078] It should be noted that the first acquisition time and the second acquisition time mentioned in this embodiment are only used to refer to two adjacent imaging actions in different lighting states, and do not constitute a strict limitation on the absolute time sequence. In the actual continuous scanning detection process, the alternating imaging cycle is in a high-frequency cyclical state (i.e., inner ring - outer ring - inner ring - outer ring...). Therefore, any two adjacent images extracted for comparison can be either the first weld image acquired first and the second weld image acquired later, or the second weld image acquired first and the first weld image acquired later. Regardless of the time sequence, as long as the two images correspond to the first incident angle and the second incident angle respectively, and the time interval satisfies the constraint of the preset time interval, the relative morphological variation of the microlens effect can be excited and captured, and all fall within the protection scope of this invention.

[0079] The value of the preset time interval is not an arbitrarily set fixed value, but is dynamically determined based on the relative motion speed between the vision acquisition component and the interlayer weld and the physical resolution of the industrial camera pixels.

[0080] Since the visual acquisition component continues to move during the preset time interval (i.e., the very short transition period between two adjacent exposures), this will inevitably lead to a physical displacement difference in the field of view between the first weld image and the second weld image. In order to ensure that the same tiny slag or defect can be accurately compared in subsequent steps, this physical displacement difference must be strictly limited.

[0081] Therefore, the principle for setting the preset time interval is: the product of the time interval and the relative motion speed (i.e., the actual physical displacement distance generated during this period) must be strictly less than the actual physical size of a single pixel of the industrial camera mapped onto the surface of the interlayer weld, or at most not exceed the minimum defect detection size tolerance allowed by the system.

[0082] In practical engineering applications, microsecond- to millisecond-level stroboscopic controllers are typically used to perform the aforementioned time-sequential alternating imaging. This preset time interval constraint based on motion speed and pixel accuracy ensures that the physical coordinates of the same glassy slag do not undergo any macroscopic translation in the first and second weld seam images, thus providing a reliable data source for the subsequent direct extraction of its optical morphology variation (rather than positional movement).

[0083] S3: Extract the first bright feature region in the first weld image and the second bright feature region in the second weld image, perform physical reference coordinate system-1 and morphological comparison on the first bright feature region and the second bright feature region, and obtain the area expansion and contraction variation coefficient.

[0084] In some embodiments, firstly, for the first weld image, the grayscale distribution information of all pixels inside the first weld image is traversed; by setting a preset high-brightness grayscale threshold, the set of continuous pixels in the first weld image with grayscale values ​​greater than the high-brightness grayscale threshold is extracted to form one or more independent connected components, which are defined as the first high-brightness feature region; similarly, using the same image processing logic, the set of continuous pixels with grayscale values ​​greater than the high-brightness grayscale threshold is extracted in the second weld image and defined as the second high-brightness feature region;

[0085] Subsequently, because the visual acquisition component undergoes a small, continuous movement relative to the interlayer weld seam within the preset time interval described in step S2, the second weld seam image has a translational deviation in the field of view relative to the first weld seam image; therefore, the extracted features must be spatially aligned.

[0086] The specific alignment method is as follows: based on the known moving speed of the visual acquisition component and the preset time interval, the physical displacement is calculated (physical displacement = preset time interval × known moving speed of the visual acquisition component), and combined with the pixel equivalent of the camera, the physical displacement is converted into a pixel translation compensation vector in the image coordinate system.

[0087] For example, assuming the visual acquisition component moves at a constant speed of 100 mm / s along the positive Y-axis of the image coordinate system, and the preset time interval is set to 1 millisecond (i.e., 0.001 seconds), the calculated physical displacement is 0.1 mm. If the industrial camera, under the current detection distance, has a pixel equivalent of 0.05 mm / pixel after calibration (i.e., 1 pixel on the image represents 0.05 mm in actual physical space), then dividing the 0.1 mm physical displacement by the pixel equivalent yields a relative offset of 2 pixels along the Y-axis of the image. Thus, the generated pixel translation compensation vector is 2 pixel units along the Y-axis.

[0088] Using the pixel translation compensation vector, the coordinates of the second bright feature region are reversed and corrected, thereby eliminating the misalignment caused by the motion, so that the corrected second bright feature region and the first bright feature region can be in a unified physical reference coordinate system.

[0089] For example, following the aforementioned motion parameters, suppose that in the first weld image, the center pixel coordinates of a certain first bright feature region extracted are located at (500, Y: 600). Since the camera has moved an equivalent distance of 2 pixels along the positive Y-axis, in the subsequently acquired second weld image, the same physical slag will recede in the opposite direction in the field of view, and the center pixel coordinates of the corresponding second bright feature region will shift to (500, Y: 598). At this time, the previously generated pixel translation compensation vector is called to perform a reverse addition correction on all pixel coordinates of the second bright feature region (i.e., the Y-axis coordinates are uniformly increased by 2), so that its center coordinates are remapped back to (500, Y: 600). Through this correction of pure coordinate coefficient values ​​based on prior motion parameters, without the need for time-consuming and easily failed global feature point matching, the feature regions in the two frames of images can be perfectly overlapped in the logical coordinate system.

[0090] After completing the spatial alignment, morphological comparison is performed on a pair of features that are in the same or adjacent positions under the same physical coordinates (i.e., two images of the same suspected slag under different light sources).

[0091] The specific implementation methods of the morphological comparison include:

[0092] The total number of effective bright pixels contained in the first bright feature region (denoted as the first area value) and the total number of effective bright pixels contained in the corresponding second bright feature region (denoted as the second area value) are counted respectively. Then, the ratio of the second area value to the first area value is calculated and used as the area expansion and contraction variation coefficient. This coefficient directly reflects the absolute size change of the bright spot of the suspected region on the two-dimensional image plane when the inner ring light source is switched to the outer ring light source.

[0093] The basis for setting the high-brightness grayscale threshold is as follows: Under normal machine vision lighting, the surface of a normal metal interlayer weld mainly exhibits diffuse reflection, with its overall background grayscale value at a medium level. True physical defects (such as pores and undercut) cannot reflect light, resulting in extremely low grayscale values ​​and appearing as dark spots. Only translucent glassy slag or extremely smooth spatter will produce strong specular reflection or microlens refraction and convergence under direct light, forming locally overexposed bright spots. Therefore, the high-brightness grayscale threshold is set to a critical value significantly higher than the average grayscale of a normal metal weld background (e.g., 1.5 to 2 times the average background grayscale, or close to the pixel saturation value of 255). This purely physical setting logic based on the difference in material optical reflectivity ensures that the system only extracts artifact candidates caused by slag for subsequent comparison, while true dark defects are safely preserved in the background and will not be misprocessed.

[0094] S4: Determine whether there is a slag artifact region between the first bright feature region and the second bright feature region at the same physical coordinate position based on the area expansion and contraction variation coefficient;

[0095] After completing the physical reference coordinate system one, for the first and second highlighted feature regions, morphological variation measurement is performed on a pair of first and second highlighted feature regions located at the same physical coordinate position.

[0096] Specifically, the first bright feature area is formed by illumination from a small first incident angle (inner ring light source) at a first moment; while the second bright feature area is formed by illumination from a large second incident angle (outer ring light source) at a second moment.

[0097] If there is a translucent, plano-convex lens-shaped glassy residual slag at the same physical coordinate position, then according to the law of optical refraction, when the incident angle of the light source changes abruptly from a smaller first incident angle to a larger second incident angle, the refracted light rays passing through the glassy residual slag will be severely deflected and diverged, causing the convergence pattern of the glassy residual slag on the target surface of the industrial camera to change fundamentally.

[0098] This optical variation caused by the microlens effect manifests in the image as one of the following two situations, or both:

[0099] Firstly, radial scaling: the geometric area of ​​the second highlighted feature region is significantly enlarged or reduced compared to the first highlighted feature region;

[0100] Secondly, caustic morphology variation: The first bright feature area originally appears as a solid bright spot with extremely high central brightness; however, under the illumination of the second incident angle, due to the divergence of refracted light, the gray value of the central area of ​​the second bright feature area drops sharply, while the edge area still maintains a high gray value, thus morphologically mutating into a hollow bright ring (i.e., a caustic ring), or breaking into multiple discrete diffuse spots.

[0101] By calculating the area expansion and contraction variation coefficient between the second bright feature region and the first bright feature region, and comparing the area expansion and contraction variation coefficient with a preset scaling ratio threshold, it is determined whether there is a slag artifact region between the first bright feature region and the second bright feature region at the same physical coordinate position.

[0102] If the coefficient of variation of area expansion and contraction between the first and second bright feature regions is greater than or equal to the scaling ratio threshold, it indicates that when the incident angle of the light source abruptly changes from a smaller first incident angle to a larger second incident angle, the refracted light passing through the bright region is drastically deflected, causing its convergence area on the industrial camera target surface to significantly expand or shrink. At this time, it is determined that the bright region has undergone radial scaling caused by the microlens effect and is marked as a slag artifact region.

[0103] If the coefficient of variation of area expansion and contraction between the first and second bright feature regions is less than the scaling ratio threshold, it indicates that the geometric dimensions of the bright region are relatively stable under different incident angle illumination, which does not conform to the physical characteristics of severe refraction of microlenses.

[0104] The basis for setting the scaling ratio threshold and the caustic morphology variation determination logic is as follows:

[0105] In conventional machine vision inspection, if there are real physical depressions (such as porosity or undercut defects) on the surface of the interlayer weld, these depressions will not produce strong reflection or refraction under any angle of light source illumination. Therefore, they will always appear as dark spots with extremely low grayscale in the image. Even if different angles of light source cause the shadows at the edges of the dark spots to shift or stretch slightly, the morphological changes caused by this pure geometric occlusion are extremely small and will never produce drastic expansion or contraction of bright areas or caustic phenomena such as abrupt change from solid bright spots to hollow bright rings.

[0106] Therefore, the scaling threshold is usually set to a value significantly larger than the tolerance of pure geometric shadow deformation (e.g., set to an area magnification or reduction of more than 30% to 50%). This judgment logic, based on the essential difference between the geometric shadow stability of true defects (physical pits) and the optical refractive drastic changes of pseudo-defects (glassy residual slag), enables this method to reliably identify and peel off slag artifact regions in extremely complex interlayer weld backgrounds, fundamentally eliminating the misjudgments that are easily generated by traditional two-dimensional image processing algorithms.

[0107] S5: Generate a dynamic mask based on the slag artifact region, and use the dynamic mask to shield the slag artifact region in the first weld image or the second weld image to obtain an artifact-free weld image.

[0108] In some embodiments, after determining all slag artifact regions, a binary dynamic mask with a resolution completely consistent with the original acquired image (i.e., the first weld image or the second weld image) is generated based on the set of pixel coordinates of these slag artifact regions.

[0109] Specifically, the generation rule of the dynamic mask is as follows: in the pixel matrix of the dynamic mask, all pixels belonging to the slag artifact region are assigned a mask blocking value (e.g., logic "0" or pure black); at the same time, all remaining pixels in the pixel matrix of the dynamic mask, except for the coordinates of the slag artifact region (i.e., pixels corresponding to the normal weld background and potential real defect regions in the original acquired image), are assigned a mask retention value (e.g., logic "1" or pure white).

[0110] Since the size, shape, and distribution of the splashed glassy residual slag are completely random and unpredictable during the actual welding process, the dynamic mask is calculated and dynamically generated in real time for each frame of the image, rather than a pre-set fixed masking area.

[0111] Subsequently, an image masking operation is performed; preferably, a first weld seam image (or a second weld seam image) with more uniform lighting and more obvious shadow features is selected as the base image to be processed; the dynamic mask is directly overwritten and replaced with the base image to be processed to obtain an artifact-free weld seam image;

[0112] For example, the average grayscale value of the region corresponding to the mask retention value in the dynamic mask is calculated on the base image to be processed (assuming that the average grayscale value of the normal weld background is calculated to be 115). Then, the dynamic mask is traversed, and when the value at the coordinate (based on machine vision steel structure weld defect detection method and system: 150, Y:150) is found to be the mask blocking value (i.e., the location is a slag artifact), the original extremely high grayscale value (e.g., 250) of the base image to be processed at that coordinate is directly and forcibly replaced with the aforementioned calculated average background grayscale value of 115. Through this smooth replacement, the originally glaring microlens bright spots are filled in and perfectly integrated into the surrounding normal weld background.

[0113] Through this spatial domain masking, the slag artifact areas in the base image to be processed, which are originally bright and easily misjudged as defect edges by edge detection operators, are forcibly erased (for example, by forcibly replacing their pixel gray values ​​with the average gray value of the surrounding normal weld background, or by directly setting them to an invalid background color). The image output after the above masking process is the artifact-free weld image that eliminates the interference of microlens refraction.

[0114] In other embodiments, when generating the dynamic mask, preferably, a morphological dilation operation is performed on the marked slag artifact region; that is, based on the determined boundary of the slag artifact region, the width of the mask is uniformly expanded outward by several pixels (e.g., 3 to 5 pixels) to form a mask blocking area slightly larger than the actual bright spot area; this adaptive outward expansion processing based on optical overflow characteristics ensures that the optical features of residual slag in the artifact-removed weld image are thoroughly and cleanly stripped away, maximizing the purification of the background of the image to be detected;

[0115] S6: Construct a defect recognition model, input the artifact-free weld image into the defect recognition model, identify and output the real surface defects of the interlayer weld.

[0116] Construct an end-to-end initial defect recognition model based on a convolutional neural network (CNN) (e.g., an object detection network based on the YOLO architecture or a semantic segmentation network based on the U-Net architecture).

[0117] In order for the defect recognition model to accurately identify real surface defects in interlayer welds, a dedicated training sample set needs to be constructed. Specifically, a large number of weld images containing real physical defects (such as porosity, undercut, lack of fusion, cracks, etc.) are collected, and the boundaries and categories of these real defects are manually labeled to generate a training set with ground truth labels.

[0118] The training set is input into the initial defect recognition model in batches. The model outputs the prediction results through forward propagation and uses a loss function (such as a weighted sum of cross-entropy loss and bounding box regression loss) to calculate the error between the prediction results and the true labels.

[0119] Subsequently, the weights and bias parameters in the network layers are continuously updated through the backpropagation algorithm. After multiple rounds of iterative training, the loss function converges to the preset range. At this point, the model has completed deep learning of the real physical concave / shadow features, solidified the network parameters, and obtained the defect recognition model.

[0120] Since the bright glassy slag and its accompanying halo in the artifact-removed weld image have been completely removed by the dynamic mask (smoothly replaced by background grayscale), only the normal weld fish scale pattern background and potential real physical defects (manifested as pure physical dark spots) are retained in the image.

[0121] The artifact-free weld image undergoes necessary tensor preprocessing (e.g., normalization, scaling to the input resolution specified by the model), and is then used as a standard input tensor, directly input into the aforementioned deployed defect recognition model. The convolutional and pooling layers within the model sequentially perform forward propagation calculations on the clean image, extracting image features from edge contours to deep semantics layer by layer.

[0122] After feature decoding of the deep network of the model and calculation by the fully connected layer (or detection head), the defect identification model directly outputs the final detection result for the interlayer weld.

[0123] The specific output includes: the category label of the identified real surface defects (such as "porosity" or "bite"), the precise location information of the defect in the image coordinate system (such as the center coordinates and length and width of the bounding box, or the pixel-level segmentation mask), and the model's confidence score for the judgment result.

[0124] In some embodiments, a confidence threshold can be further set to filter out minor noise points with scores below the threshold, and finally map the high-confidence real defect coordinates back to the actual physical space coordinate system to guide subsequent grinding robots or welding equipment to perform precise rework.

[0125] This invention employs hardware multi-angle lighting and dynamic masking algorithms in preliminary steps (S1 to S5) to precisely remove slag artifact regions in the physical and spatial domains in advance. This means that the data input into the defect identification model in step S6 is already a thoroughly cleaned, high-quality base map. Therefore, in step S6, the defect identification model no longer needs to consume enormous computing power to learn how to eliminate slag interference; instead, it can focus all its network weights on extremely sensitive capture of real physical concave features. This pre-emptive physical artifact removal mechanism allows the AI ​​model in step S6 to confidently and boldly increase its detection sensitivity, pushing the detection capability of real, minute defects to its limit while completely eliminating false slag detections.

[0126] Example 2: As Figure 2 As shown, based on the specific implementation process of Embodiment 1, the present invention provides a steel structure weld defect detection system based on machine vision, including:

[0127] Visual acquisition module: A visual acquisition component is set above the interlayer weld to be inspected. The visual acquisition component includes an industrial camera and a concentric multi-channel ring light source set coaxially with the industrial camera. The concentric multi-channel ring light source includes an inner ring light source with a first radius and an outer ring light source with a second radius.

[0128] Timing Alternating Imaging Control Module: During the movement of the interlayer weld, the vision acquisition component is controlled to perform timing alternating imaging: at the first acquisition moment, the inner ring light source is turned on to acquire the first weld image, and at the second acquisition moment after a preset time interval, the outer ring light source is turned on to acquire the second weld image.

[0129] Highlight feature extraction and comparison analysis module: Extract the first high-brightness feature region of the first weld image and the second high-brightness feature region of the second weld image, perform physical reference coordinate system-1 and morphological comparison on the first high-brightness feature region and the second high-brightness feature region, and obtain the area expansion and contraction variation coefficient.

[0130] Slag artifact detection module: Determines whether there is a slag artifact region between the first and second bright feature regions at the same physical coordinate position based on the area expansion and contraction variation coefficient;

[0131] Dynamic mask generation and artifact shielding module: Generates a dynamic mask based on the slag artifact region, and uses the dynamic mask to shield the slag artifact region in the first weld image or the second weld image to obtain an artifact-free weld image.

[0132] Weld Defect Recognition Module: Constructs a defect recognition model, inputs artifact-free weld images into the defect recognition model, identifies and outputs the true surface defects of interlayer welds.

[0133] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A machine vision-based method for detecting weld defects in steel structures, characterized in that: include: S1: A vision acquisition component is set above the interlayer weld to be inspected. The vision acquisition component includes an industrial camera and a concentric multi-channel ring light source set coaxially with the industrial camera. The concentric multi-channel ring light source includes an inner ring light source with a first radius and an outer ring light source with a second radius. S2: During the movement of the interlayer weld, the vision acquisition component is controlled to perform time-sequential alternating imaging: at the first acquisition moment, the inner ring light source is turned on to acquire the first weld image, and at the second acquisition moment after a preset time interval, the outer ring light source is turned on to acquire the second weld image. S3: Extract the first bright feature region of the first weld image and the second bright feature region of the second weld image, perform physical reference coordinate system-1 and morphological comparison on the first bright feature region and the second bright feature region, and obtain the area expansion and contraction variation coefficient. S4: Determine whether there is a slag artifact region between the first and second bright feature regions at the same physical coordinate position based on the area expansion and contraction variation coefficient. S5: Generate a dynamic mask based on the slag artifact region, and use the dynamic mask to shield the slag artifact region in the first weld image or the second weld image to obtain an artifact-free weld image. S6: Construct a defect recognition model, input the artifact-free weld image into the defect recognition model, identify and output the real surface defects of the interlayer weld.

2. The method for detecting weld defects in steel structures based on machine vision according to claim 1, characterized in that, The inner ring light source of the first radius and the outer ring light source of the second radius are specifically configured as follows: The first radius of the inner ring light source is set between 15 mm and 85 mm, and the second radius of the outer ring light source is set between 80 mm and 400 mm.

3. The method for detecting weld defects in steel structures based on machine vision according to claim 1, characterized in that, The specific process for acquiring the image of the first weld seam is as follows: At the first acquisition moment, the vision controller outputs the first trigger signal, which controls the inner ring light source to light up and the outer ring light source to turn off, triggering the industrial camera to perform global exposure, thereby acquiring the first weld seam image.

4. The method for detecting defects in steel structure welds based on machine vision according to claim 1, characterized in that, The specific process for acquiring the image of the second weld seam is as follows: At the second acquisition time after a preset time interval, the vision controller outputs a second trigger signal to turn off the inner ring light source and turn on the outer ring light source, triggering the industrial camera to perform a second global exposure, thereby acquiring the second weld seam image.

5. The method for detecting defects in steel structure welds based on machine vision according to claim 1, characterized in that, The specific process for extracting the first highlighted feature region from the first weld image and the second highlighted feature region from the second weld image is as follows: For the first weld seam image, the grayscale distribution information of all pixels inside the first weld seam image is traversed, a high-brightness grayscale threshold is set, and the set of continuous pixels in the first weld seam image with grayscale values ​​greater than the high-brightness grayscale threshold is extracted to form one or more independent connected components, which are defined as the first high-brightness feature region. In the second weld seam image, the set of continuous pixels with grayscale values ​​greater than the high-brightness grayscale threshold is extracted and defined as the second high-brightness feature region.

6. The method for detecting defects in steel structure welds based on machine vision according to claim 1, characterized in that, The specific process for obtaining the area expansion / contraction variation coefficient includes: Based on the known moving speed of the visual acquisition component and the preset time interval, the physical displacement is calculated as: preset time interval × known moving speed of the visual acquisition component. Combined with the pixel equivalent, the physical displacement is converted into a pixel translation compensation vector. Using a pixel translation compensation vector, the coordinates of the second highlight feature region are reverse-translated and corrected so that the corrected second highlight feature region and the first highlight feature region are in a unified physical reference coordinate system. The total number of effective highlight pixels contained in the first highlight feature region is counted and recorded as the first area value, and the total number of effective highlight pixels contained in the second highlight feature region is recorded as the second area value. The ratio of the second area value to the first area value is calculated as the area expansion and contraction variation coefficient.

7. The method for detecting defects in steel structure welds based on machine vision according to claim 1, characterized in that, The specific process for determining whether there is a slag artifact region between the first bright feature region and the second bright feature region at the same physical coordinate position based on the area expansion and contraction variation coefficient is as follows: Calculate the area expansion and contraction variation coefficient between the second highlight feature region and the first highlight feature region, and compare the area expansion and contraction variation coefficient with the preset scaling ratio threshold. If the coefficient of variation of area expansion and contraction between the first and second highlighted feature regions is greater than or equal to the scaling ratio threshold, it is determined that the highlighted region has undergone radial scaling and is marked as a slag artifact region.

8. The method for detecting defects in steel structure welds based on machine vision according to claim 1, characterized in that, The specific process for obtaining the artifact-free weld image is as follows: A dynamic mask is generated based on the slag artifact region. The generation rule of the dynamic mask is as follows: in the pixel matrix of the dynamic mask, all pixels belonging to the slag artifact region are assigned mask blocking values, and all remaining pixels in the pixel matrix of the dynamic mask except for the coordinates of the slag artifact region are assigned mask retention values. The first weld image or the second weld image is selected as the base image to be processed. The dynamic mask and the base image to be processed are directly overwritten and replaced with pixel values ​​to obtain the artifact-free weld image.

9. The method for detecting weld defects in steel structures based on machine vision according to claim 1, characterized in that, The specific process for identifying and outputting the actual surface defects of the interlayer weld is as follows: Construct an initial defect recognition model based on a convolutional neural network (CNN) and build a dedicated training sample set. Input the training set into the initial defect recognition model in batches, output the prediction results through forward propagation, and use the loss function to calculate the error between the prediction results and the true labels. Continuously update the weights and bias parameters in the network layers through the backpropagation algorithm until the loss function converges to the preset range to obtain the defect recognition model. The artifact-free weld image is preprocessed with tensor quantization and used as a standard input tensor, which is then directly input into the actual surface defects identified in the defect recognition model.

10. A machine vision-based steel structure weld defect detection system, used to perform the method described in any one of claims 1-9, characterized in that: include: Visual acquisition module: A visual acquisition component is set above the interlayer weld to be inspected. The visual acquisition component includes an industrial camera and a concentric multi-channel ring light source set coaxially with the industrial camera. The concentric multi-channel ring light source includes an inner ring light source with a first radius and an outer ring light source with a second radius. Timing Alternating Imaging Control Module: During the movement of the interlayer weld, the vision acquisition component is controlled to perform timing alternating imaging: at the first acquisition moment, the inner ring light source is turned on to acquire the first weld image, and at the second acquisition moment after a preset time interval, the outer ring light source is turned on to acquire the second weld image. Highlight feature extraction and comparison analysis module: Extract the first high-brightness feature region of the first weld image and the second high-brightness feature region of the second weld image, perform physical reference coordinate system-1 and morphological comparison on the first high-brightness feature region and the second high-brightness feature region, and obtain the area expansion and contraction variation coefficient. Slag artifact detection module: Determines whether there is a slag artifact region between the first and second bright feature regions at the same physical coordinate position based on the area expansion and contraction variation coefficient; Dynamic mask generation and artifact shielding module: Generates a dynamic mask based on the slag artifact region, and uses the dynamic mask to shield the slag artifact region in the first weld image or the second weld image to obtain an artifact-free weld image. Weld Defect Recognition Module: Constructs a defect recognition model, inputs artifact-free weld images into the defect recognition model, identifies and outputs the true surface defects of interlayer welds.

Images