A building steel structure weld quality robot vision detection system and method

Abstract

This invention discloses a robotic vision inspection system and method for the quality of welds in building steel structures, relating to the field of non-destructive testing technology. It aims to solve the technical problem that existing infrared detection methods cannot effectively detect closed microcracks inside welds in building steel structures. The system includes: simultaneously acquiring a first infrared band sequence of thermal images and a second infrared band sequence of thermal images of the weld area under inspection during the natural cooling stage of the weld; registering the first and second infrared band sequence thermal images and extracting a first and second temperature sequence pixel-by-pixel from the registered image sequence; and performing numerical differentiation based on the first and second temperature sequences. This invention has the advantages of converting the thermal resistance effect of cracks into a quantifiable signal and realizing non-contact internal defect detection and depth estimation.

Description

Technical Field

[0001] This invention relates to the field of nondestructive testing technology, and more specifically, to a robotic vision inspection system and method for the quality of welds in building steel structures. Background Technology

[0002] For steel structures, especially large stadiums, bridges, and offshore platform modules used in high-altitude and cold regions, the quality of welds directly affects structural safety. During multi-layer, multi-pass welding, factors such as excessive post-weld cooling rates due to extremely low temperatures, interpass temperature gradients, and strong restraint stresses can easily lead to cold cracks, interpass tearing, and microcracks with extremely narrow or compressed openings within the weld. These internally closed microcracks are completely invisible in post-weld visual inspections, and conventional non-destructive testing methods also face significant limitations: ultrasonic testing has insufficient resolution for near-surface closed cracks, radiographic testing is insensitive to tightly closed cracks parallel to the beam direction, and magnetic particle and penetrant testing can only detect surface opening defects.

[0003] Infrared thermography, as a non-contact inspection method, has been attempted to detect defects based on post-weld cooling curve anomalies. However, existing methods generally suffer from two shortcomings. First, most methods only use a single wavelength to acquire the surface temperature sequence of the weld, and the acquired cooling information only reflects the thermal behavior within a few micrometers of the outermost layer of the weld. They cannot detect temperature field disturbances below the surface caused by the thermal resistance effect of internal cracks, thus lacking effective detection capability for internal closed microcracks. Second, even when using multi-band infrared methods, existing methods mainly focus on the temperature difference between the surface and subsurface layers at a fixed moment, failing to systematically utilize the dynamic differences in cooling behavior between different depths represented by different wavelengths during the entire post-weld natural cooling process. This results in a lack of clear physical mechanism support and robust quantitative criteria for extracting internal thermal resistance anomaly signals, making them highly susceptible to interference from environmental noise and post-weld airflow disturbances. Therefore, we propose a robotic vision inspection system and method for weld quality in building steel structures. Summary of the Invention

[0004] The purpose of this invention is to provide a robotic vision inspection system and method for the quality of welds in building steel structures, so as to solve the technical problem that existing infrared detection methods cannot effectively detect closed microcracks inside welds in building steel structures.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a robot visual inspection method for the quality of welds in building steel structures, comprising: During the natural cooling stage of the weld, the first infrared band sequence thermal image and the second infrared band sequence thermal image of the inspected weld area are acquired simultaneously. The penetration depth of the first infrared band is less than that of the second infrared band. The first infrared band sequence thermal image and the second infrared band sequence thermal image are registered, and the first temperature sequence and the second temperature sequence are extracted pixel by pixel from the registered image sequence; Numerical differentiation is performed based on the first temperature sequence and the second temperature sequence to obtain the surface cooling rate curve and the shallow body cooling rate curve for each pixel. Calculate the difference between the shallow body cooling rate and the surface cooling rate at the same pixel point at the same time, and construct a spatiotemporal distribution map of the cooling rate difference in the weld area; In the spatiotemporal distribution map of the cooling rate difference, transient positive peak values ​​that meet the conditions of spatial aggregation and short duration are identified, and the existence of internal closed microcracks is determined and their locations are marked based on the transient positive peak values.

[0006] Preferably, in the spatiotemporal distribution map of the cooling rate difference, transient positive peak values ​​that satisfy the conditions of spatial clustering and short duration are identified, including: The curve of the cooling rate difference of each pixel over time is scanned, and the waveform segment on the curve whose amplitude exceeds the upper limit of the noise statistical level is identified as a candidate positive peak. The upper limit of the noise statistics level is dynamically determined by multiplying the root mean square of the cooling rate difference value of multiple defect-free reference pixels in the weld area by a preset coefficient.

[0007] Preferably, the identification of transient positive peak values ​​that satisfy the conditions of spatial aggregation and short duration further includes: By analyzing spatial connectivity, adjacent pixels that simultaneously exhibit the candidate positive peak are grouped into a region to eliminate isolated noise points and obtain the transient positive peak value. Extract the time delay of the occurrence of the transient positive peak relative to the cooling start point, as well as the amplitude of the transient positive peak.

[0008] Preferably, after determining the presence and location of the internal closed-type microcrack based on the transient positive peak value, the method further includes: Based on the delay and the amplitude, estimate the relative burial depth of the internal closed microcrack from the surface layer; The estimation relationship for the relative burial depth is as follows: the shorter the delay and the larger the amplitude, the shallower the estimated relative burial depth; the longer the delay and the smaller the amplitude, the deeper the estimated relative burial depth.

[0009] Preferably, before performing numerical differentiation based on the first temperature sequence and the second temperature sequence, the method further includes: The first temperature sequence and the second temperature sequence are preprocessed using a zero-phase time-domain low-pass filter. The cutoff characteristics of the zero-phase time-domain low-pass filter are set according to the highest frequency occupied by the temperature change signal in the normal region of the weld.

[0010] Preferably, registering the first infrared band sequence thermal image and the second infrared band sequence thermal image includes: Significant feature points are detected in the first infrared band sequence thermal image and the second infrared band sequence thermal image. Based on the feature point matching results, the affine transformation matrix between frames is estimated, and the affine transformation matrix is ​​used to perform spatial coordinate transformation on the second infrared band sequence thermal image. The image after spatial coordinate transformation is locally adjusted using B-spline-based free deformation registration to ensure that the first infrared band sequence thermal image and the second infrared band sequence thermal image maintain pixel-by-pixel spatial correspondence in the cooling sequence.

[0011] Preferably, the first infrared band is a long-wave infrared band, and the second infrared band is a mid-wave infrared band; the simultaneous acquisition of the first infrared band sequence thermal images and the second infrared band sequence thermal images of the inspected weld area includes: Infrared radiation from the inspected area is separated into bands by a beam splitter and guided to a first imaging channel sensitive to the first infrared band and a second imaging channel sensitive to the second infrared band, respectively. The first imaging channel and the second imaging channel are driven by a synchronization signal controller to record images with the same frame period and the same exposure start time.

[0012] A robotic vision inspection system for weld quality in building steel structures includes: The image acquisition module is used to simultaneously acquire a first infrared band sequence thermal image and a second infrared band sequence thermal image of the inspected weld area during the natural cooling stage of the weld. The penetration depth of the first infrared band is less than that of the second infrared band. The registration and extraction module is used to register the first infrared band sequence thermal image and the second infrared band sequence thermal image, and extract the first temperature sequence and the second temperature sequence pixel by pixel from the registered image sequence; The cooling rate calculation module is used to perform numerical differentiation operations based on the first temperature sequence and the second temperature sequence to obtain the surface cooling rate curve and the shallow body cooling rate curve of each pixel. The difference distribution map generation module is used to calculate the difference between the shallow body cooling rate and the surface cooling rate pixel by pixel, obtain the curve of the cooling rate difference over time, and construct the spatiotemporal distribution map of the cooling rate difference in the weld area. The transient positive peak value identification and determination module is used to identify transient positive peak values ​​that meet the conditions of spatial aggregation and short duration in the spatiotemporal distribution map of the cooling rate difference, and to determine the existence of internal closed microcracks and mark their locations based on the transient positive peak values.

[0013] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement a robot visual inspection method for weld quality of building steel structures.

[0014] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a robotic visual inspection method for the quality of welds in building steel structures.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention simultaneously acquires thermal images of the first and second infrared band sequences. Utilizing the inherent physical difference that long-wave infrared waves can only penetrate the surface of steel while short-wave or mid-wave infrared waves can penetrate to the shallow layer, two independent cooling curves reflecting the surface temperature dynamics and the overall temperature dynamics of the shallow layer can be simultaneously obtained at the same pixel point. Furthermore, based on the difference between the cooling rate of the shallow layer and the surface layer at the same time, a spatiotemporal distribution map of the cooling rate difference is constructed. For the first time, the thermal resistance effect of internal closed cracks on the heat transfer path from the inside of the weld to the surface is transformed into a quantifiable and detectable positive difference signal in the cooling rate, achieving non-contact detection of internal closed microcracks such as cold cracks, interlaminar tears, and extrusion-induced closed cracks in high-altitude and cold regions.

[0016] 2. Based on the above solution, this invention further implements dynamic threshold detection based on the statistical characteristics of reference pixels on the cooling rate difference curve. The upper limit of the noise statistical level that changes with cooling time is constructed by multiplying the root mean square of the cooling rate difference value in the defect-free area by a preset coefficient. Combined with spatial connectivity analysis, transient positive peak values ​​that are spatially concentrated and short in time are extracted. This effectively eliminates the interference of isolated noise points and random environmental fluctuations, so that the thermal resistance anomaly response of internal closed microcracks can be accurately and reliably identified from the complex thermal dynamic background.

[0017] 3. Based on dynamic threshold detection and positive peak extraction, this invention constructs an estimation relationship of relative burial depth according to the delay and amplitude of transient positive peak value. It establishes a quantitative mapping between the time response characteristics and intensity characteristics of thermal resistance signal and the depth of crack from the surface layer. This makes the detection results not only include the two-dimensional location information of crack, but also the relative burial depth information characterizing the crack burial depth, providing a more sufficient basis for the quantitative assessment of welding quality and repair welding decision of steel structure in high-altitude and cold regions. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is a schematic diagram of the transient positive peak value identification process of the present invention; Figure 3 This is a schematic diagram of the image registration and temperature sequence preprocessing process of the present invention; Figure 4 This is a schematic diagram illustrating the process of relative burial depth estimation and result visualization in this invention. Detailed Implementation

[0019] To facilitate understanding of the technical solution of the present invention by those skilled in the art, the technical solution of the present invention will now be further described in conjunction with the accompanying drawings.

[0020] Example 1, such as Figures 1 to 4 As shown, this invention provides a robot vision inspection method for the quality of welds in building steel structures, the method comprising: Step S101: In the initial stage when the welding arc is extinguished and the weld metal begins to cool naturally, the first infrared band sequence thermal image and the second infrared band sequence thermal image of the weld area under inspection are acquired simultaneously. Step S102: Register the first infrared band sequence thermal image and the second infrared band sequence thermal image; Step S103: Extract the first temperature sequence and the second temperature sequence pixel by pixel from the registered image sequence; Step S104: Calculate the surface cooling rate curve and shallow volume cooling rate curve for each pixel using numerical differentiation. Step S105: Calculate the difference between the shallow body cooling rate and the surface cooling rate at the same pixel point at the same time, and construct a spatiotemporal distribution map of the cooling rate difference in the weld area. Step S106: Identify spatially clustered and short-lived transient positive peaks in the spatiotemporal distribution map of cooling rate differences; Step S107: Based on the identified transient positive peak value, determine the existence of internal closed microcracks and mark their location.

[0021] In this embodiment of the invention, the above steps work together to construct a complete technical link from the acquisition of underlying physical quantities to the final defect determination. Step S101, by selecting two infrared bands with different penetration depths, achieves completely synchronous temporal and spatial separation measurement of the surface thermal state and shallow body thermal state of the weld from the physical source. This transforms the hindering effect of internal defects on heat conduction from a slight difference in the temperature field of a single surface into a difference in the temporal sequence of temperature response between the two layers, greatly enhancing the signal-to-noise ratio. Step S102 ensures that the coordinates and timing of the two band data are strictly aligned during subsequent pixel-by-pixel differential operations, which is a prerequisite for constructing accurate cooling rate differences. Steps S103 to S105 transform the original spatiotemporal thermal image data stream into a spatiotemporal distribution map of cooling rate differences that clearly highlights abnormal areas through point-by-point and frame-by-frame numerical differentiation and algebraic operations, mapping the physical phenomenon into a directly processable mathematical image. Steps S106 and S107 then accurately detect and locate cracks based on the spatiotemporal continuity and physical laws of the difference distribution map.

[0022] Taking the post-weld inspection scenario of a butt weld on a thick steel plate in a high-altitude, cold region as an example, the image acquisition module of the robot vision inspection system moves above the weld that has just been welded. Within seconds of the arc being extinguished, it begins to simultaneously acquire sequential thermal images in the long-wave infrared band (first infrared band, corresponding to the surface penetration depth) and the mid-wave infrared band (second infrared band, corresponding to the shallow body penetration depth). After the registration module completes the pixel-level alignment in step S102, the temperature sequence extraction module extracts the temperature sequence from the image data for each spatial location. Extracting time The first temperature sequence of the changing S103 step Second temperature sequence When a cold crack exists somewhere inside the weld, caused by an excessively rapid cooling rate due to a high-temperature environment, this crack constitutes a planar thermal resistance. During natural cooling, surface heat is continuously lost through radiation and convection, and the temperature... The temperature drops rapidly; however, the shallow layer below the crack has its heat flow to the surface blocked by the crack, causing heat to accumulate and leading to a decrease in temperature. The descent is relatively delayed. This physical mechanism causes the difference in cooling rate calculated in step S105. A significant positive increase will occur within a specific time window, namely the transient positive peak value of S106. However, in defect-free regions, the cooling rates of the surface and shallow layers exhibit a stable relative relationship, and such abnormal protrusions do not occur.

[0023] It should be noted that during most of the natural cooling period of the weld, the surface layer typically cools faster than the shallow layer due to direct heat dissipation; therefore, the baseline value of this cooling rate difference is usually negative. When the thermal resistance effect of an internal closed crack blocks the heat flow from the shallow layer to the surface, the cooling rate of the shallow layer decreases relative to the surface. This difference is represented on the time curve as a bulge from negative to zero or even positive, i.e., a positive increase. This positive increase is a localized abnormal rise relative to the overall trend of the difference curve and is unrelated to the absolute sign of the difference.

[0024] As a specific implementation method, the formation and capture of the difference signal are closely coupled with the multi-band hierarchical infrared detection and point-by-point deconstruction analysis of the dynamics of heat conduction in the scheme, realizing the leap from "seeing the surface of the weld" to "perceiving the abnormal heat flow inside the weld skin".

[0025] In one embodiment, before performing numerical differentiation on the first temperature sequence and the second temperature sequence in step S104, the method further includes: preprocessing the first temperature sequence and the second temperature sequence to reduce the influence of infrared detector noise and high-frequency random fluctuations caused by post-weld environmental airflow disturbance on the differentiation result; the preprocessing uses a zero-phase time-domain low-pass filter, the cutoff characteristic of which is set according to the highest frequency occupied by the temperature change signal in the normal area of ​​the weld; then, differential operation based on the temperature difference between adjacent frames is performed on the smoothed sequence to obtain the instantaneous cooling rate at each moment, thereby constructing the surface cooling rate curve and the shallow body cooling rate curve.

[0026] For pixel coordinates The first temperature sequence at point is filtered and denoted as . The second temperature sequence, after filtering, is denoted as... The time interval between adjacent frames is Surface cooling rate and shallow body cooling rate Defined by the following formulas respectively: ; ; in, These are pixel coordinates used to identify the specific spatial location of the weld area in the image. The cooling time is defined as the instant the welding arc extinguishes. For pixels The surface temperature value after zero-phase time-domain low-pass filtering. For pixels The shallow body temperature value after zero-phase time-domain low-pass filtering. This is the time interval between adjacent frames, i.e., the sampling period of the infrared thermal image sequence. For pixels At any moment The surface cooling rate indicates how quickly the surface temperature decreases. For pixels At any moment The shallow body cooling rate indicates how quickly the temperature of the shallow body decreases.

[0027] In this embodiment, the preprocessing step serves as a prerequisite for the differential operation, effectively solving the technical problem that directly performing numerical difference on noisy temperature signals would severely amplify high-frequency noise, causing violent oscillations in the cooling rate curve and completely drowning out minute difference signals. The zero-phase filter, while filtering out high-frequency interference such as detector 1 / f noise, shot noise, and temperature fluctuations caused by random ambient airflow, does not introduce any phase delay, ensuring that the time position of the temperature inflection point remains unchanged after filtering. This is crucial for subsequent estimation of crack depth based on the occurrence time of the cooling rate difference peak.

[0028] As a specific implementation method, the design of this zero-phase filter can be based on Fourier transform. After transforming the signal to the frequency domain, it is multiplied by a real symmetric rectangular window or Hanning window function. The cutoff frequency is determined by analyzing the spectrum of the temperature drop curves of multiple defect-free reference areas. Usually, 3 to 5 times the signal's main frequency is selected as the cutoff point to completely preserve the smooth downward trend of the cooling curve while removing high-frequency glitches.

[0029] In one embodiment, for steps S105 and S106 above, the cooling rate difference is obtained by subtracting the surface cooling rate from the shallow layer cooling rate. The method for detecting transient positive peaks is as follows: the cooling rate difference curve of each pixel over time is scanned, and the waveform segment on the curve that bulges from the noise floor towards the positive value, exceeds the upper limit of the noise statistical level, and then falls back to the floor is identified as a candidate positive peak; the upper limit of the noise statistical level is dynamically determined by multiplying the root mean square of the cooling rate difference values ​​of multiple reference pixels confirmed to be defect-free in the weld area by a preset coefficient; then, through spatial connectivity analysis, adjacent pixels that simultaneously exhibit positive peaks are clustered into a region to exclude isolated noise points.

[0030] Among them, the difference in cooling rate Defined as: ; Upper limit of noise statistical level From the selected set of defect-free reference pixels The root mean square of the cooling rate difference value of all pixels is calculated and multiplied by a preset coefficient. get: ; in, For the reference pixel set at time The average difference value, For reference pixel count; when pixels The difference curve at time satisfy Furthermore, if the difference value rises from the noise floor before and after that moment and then falls back to the floor, then... A candidate positive peak is formed at this point; the time of appearance of this positive peak is relative to the cooling start point. The delay is The amplitude of the positive peak .

[0031] In this embodiment, a data-driven dynamic threshold strategy is used instead of a fixed threshold, effectively avoiding false detections caused by uneven overall thermal field in different weld areas and cooling stages. The reference pixel set can typically be manually calibrated from a few defect-free areas or automatically selected through iterative analysis. Multiplied by a preset coefficient... (For example =5) A detection confidence level much higher than the random noise level was set. Furthermore, spatial connectivity analysis utilizes the prior knowledge that defects themselves have a certain spatial extension to ensure that only those positive peak sets that are spatially aggregated and conform to the crack morphology characteristics are identified as defect regions, while abnormal peaks that occasionally appear in individual pixels due to extreme noise are effectively filtered out, significantly enhancing the detection system's accuracy and robustness in identifying internal closed cracks.

[0032] In one embodiment, regarding the specific selection of the first infrared band and the second infrared band, the first infrared band is selected from the spectral band covering the long-wave infrared atmospheric window in the thermal imager, and the second infrared band is selected from the spectral band covering the mid-wave infrared atmospheric window in the thermal imager. The difference in photon energy between the first infrared band and the second infrared band results in different absorption coefficients in the steel, thereby forming the required distinction between the depth of surface penetration and shallow volume penetration.

[0033] This embodiment clarifies a mature and high-performance hardware selection basis for implementing a dual-band scheme. Long-wave infrared (typically the 8-14μm band) photons have relatively low energy levels, strongly couple with the lattice vibrations of steel, and have a very high absorption coefficient. After incident, the energy is completely absorbed at a distance of only tens of micrometers from the surface, thus its detection signal accurately originates from the surface layer. Mid-wave infrared (typically the 3-5μm band), in comparison, has higher photon energy and stronger penetrating power in steel. Its effective detection depth can reach hundreds of micrometers, precisely covering the depth range of interlayer tears and thin-layer extrusion closure cracks typically distributed near the surface of welds, i.e., the shallow layer.

[0034] In one embodiment, the synchronous acquisition of the first infrared band sequence thermal image and the second infrared band sequence thermal image is achieved as follows: the infrared radiation from the area under inspection is separated into bands by a beam splitter and guided to the first imaging channel sensitive to the first infrared band and the second imaging channel sensitive to the second infrared band, respectively. The first imaging channel and the second imaging channel are driven by a synchronization signal controller to record images with the same frame period and the same exposure start time, ensuring the time alignment of the first infrared band sequence thermal image and the second infrared band sequence thermal image, and the initial alignment of spatial registration is achieved through the previous optical path calibration.

[0035] This embodiment constructs a common-path beam splitting imaging system to ensure "simultaneous" acquisition of dual-band data from the same source. Infrared radiation from the same point is first collected by the same front-facing lens, and then split according to wavelength by devices such as dichroic beam splitters or gratings. Trigger pulses generated by a precision synchronization signal controller simultaneously ignite the integration process of both detectors, controlling the inter-frame time skew to the nanosecond level. This is the fundamental hardware guarantee for differential analysis of transient physical processes in this invention. The initial optical path calibration uses a standard calibration target (such as a checkerboard thermal target) to establish a precise geometric transformation model between the two channels, providing a near-coincident initial value for the pixel-level fine registration in the subsequent S102 step, reducing the difficulty and computational load of the fine registration algorithm.

[0036] In one embodiment, after determining the existence of an internal closed microcrack in step S107, the method further includes: using the acquired visible light image of the weld surface as a background, superimposing and marking the location and outline of the crack; and estimating the relative depth of the crack from the surface based on the delay of the time of the transient positive peak relative to the cooling start point, and the amplitude of the transient positive peak; the shorter the delay and the larger the amplitude, the shallower the relative depth; the longer the delay and the smaller the amplitude, the deeper the relative depth, and encoding the relative depth information as a color mark in the detection result image.

[0037] relative burial depth Based on delay and amplitude Estimate using the following formula: ; In the formula, This constant, determined through calibration experiments, reflects that the closer an internal closed crack is to the surface, the earlier and more significant the characteristic of heat transfer obstruction occurs, thus delaying the amount of heat transfer. The smaller the amplitude The larger the value, the higher the calculated relative burial depth. The smaller the value, the shallower the crack.

[0038] This embodiment, based on the qualitative determination of the presence or absence of defects, further achieves quantitative inference of defect depth, providing crucial three-dimensional information for weld repair processes. The underlying physical logic is that the closer the crack is to the surface, the earlier its heat-blocking effect manifests (delay amount). Smaller (and more drastic) effects on the surface temperature (magnitude) (Larger); Conversely, the deeper the crack, the longer it takes for the heat wave to reach the crack and be reflected or diffracted back to be reflected on the surface temperature difference, and the signal becomes weaker due to diffusion.

[0039] As a specific implementation method, constant It can be determined under laboratory conditions by calibration experiments on standard specimens with pre-existing artificial defects at a series of different depths (such as horizontal cracks pre-induced by electrical discharge machining). In output, the HSV color space is used to linearly map the depth range onto a color wheel (e.g., shallow cracks are highlighted in red, and deep cracks are marked in blue), generating a semi-transparent pseudo-color map of crack depth on a visible light background, forming an intuitive "X-ray" visualization report of the weld's internal quality.

[0040] In one embodiment, this method is applied to multi-layer, multi-pass welding scenarios for large-scale steel structures in high-altitude and cold regions. Inspection is initiated immediately after the welding arc of each weld pass extinguishes and the weld enters the natural cooling stage. Internal crack detection is completed before the next weld pass, preventing the accumulation of defective welds and thus achieving closed-loop quality control of the welding process. In high-altitude and cold regions, extremely low ambient temperatures and strong gusts of wind accelerate post-weld cooling, making "interlayer tears," root cold cracks, and other "internal closed-type microcracks" high-frequency defects. Traditional inspection methods are post-weld non-destructive testing, lagging behind the welding process. Once a defect is detected, subsequent multiple weld passes may have already covered it, making rework extremely difficult. This method utilizes the inevitable natural cooling process after welding as a "thermal signal source" carrying internal defect information. The inspection system, mounted on a robot, operates online in real time, completing the inspection of the current weld pass within the interlayer temperature control waiting time between the completion of one weld pass and the start of the next. Once a crack is discovered, the grinding and removal process can be started immediately, realizing a closed-loop process of "welding-inspection-feedback", which greatly reduces the risk of overall structural scrap due to the accumulation of defects.

[0041] In one embodiment, before extracting the first temperature sequence and the second temperature sequence in step S103, the method further includes performing non-uniformity correction on the original infrared sequence thermal image based on radiometric calibration, using multi-point correction coefficients to eliminate fixed pattern noise caused by inconsistent responses of each detector element, and replacing pixels with abnormal responses with bilinear interpolation results of gray values ​​of neighboring normal pixels to ensure the radiometric consistency of image data in the spatiotemporal dimension.

[0042] This embodiment operates at the source of the data stream, ensuring image constancy at the pixel level. Infrared focal plane array detectors inevitably suffer from inconsistent (non-uniform) response rates among pixels, manifesting as fixed pattern noise in the image. In conventional temperature monitoring, this noise may only affect visual appearance, but for the pixel-by-pixel fine differential calculation process of this invention, it is drastically amplified by differentiation and subtraction operations, directly generating false "cooling rate anomalies," whose spatial morphology is easily confused with crack signals. By obtaining correction coefficients through multi-point blackbody calibration and implementing real-time correction, while simultaneously processing bad pixels, the response of all effective pixels in the entire image can be equivalent to that of an ideal uniform detector, thus ensuring the physical authenticity of the data generated in subsequent steps S104 and S105.

[0043] In one embodiment, the pixel-level spatiotemporal registration of the first infrared band sequence thermal image and the second infrared band sequence thermal image in step S102 includes: automatically detecting significant feature points in the first infrared band sequence thermal image and the second infrared band sequence thermal image, estimating the affine transformation matrix between frames based on the feature point matching results; using the affine transformation matrix to perform spatial coordinate transformation and grayscale resampling on the second infrared band sequence thermal image to register the second infrared band sequence thermal image onto the pixel grid of the first infrared band sequence thermal image; for the small shrinkage deformation of the weld caused by the welding thermal cycle, further local adjustments are made using free deformation registration based on B-splines to ensure that the first infrared band sequence thermal image and the second infrared band sequence thermal image maintain pixel-by-pixel spatial correspondence throughout the entire cooling sequence, where B-spline is a mathematical tool for elastic image registration, and its full name is BasisSpline.

[0044] This embodiment details the algorithmic strategy for achieving pixel-level registration. Global affine registration corrects the linear geometric differences between the two channels caused by installation, lens distortion, etc., forming the main body of the registration. However, weld metal undergoes thermal expansion and contraction within a cooling range of several hundred degrees Celsius, which is a nonlinear, non-rigid microscopic deformation process that cannot be described by a single global transformation. Therefore, a B-spline-based free deformation (FFD) model is introduced as a supplementary step. By manipulating the control point grid overlaid on the image, local plastic deformation is simulated. This model can accurately "follow" the spatial displacement of each tiny weld protrusion or depression throughout the cooling process, ensuring that the label of the "same physical point" is strictly locked to the "same pixel coordinate" over time.

[0045] As a specific implementation method, salient feature points can be extracted using Scale Invariant Feature Transform (SIFT) or Speed-Up Robust Feature Transform (SURF) algorithms. The grid spacing of the B-spline can be set to 32×32 or 64×64 pixels according to the expected deformation amplitude. The optimization of deformation is solved iteratively using normalized mutual information as the similarity measure.

[0046] It should be noted that the reference pixel set The selection of reference pixels relies on the prior knowledge of the pre-existing crack location, but in actual inspection applications, the defect location is unknown. Therefore, this invention provides the following automatic reference pixel selection strategy that does not depend on prior defect knowledge, ensuring reliable implementation even in scenarios where the surface is not visible.

[0047] This strategy is based on the following physical facts: During the natural cooling process after welding, the only temperature difference between the surface layer and the shallow bulk of the defect-free region is caused by uniform thermal diffusion of the material. The difference in cooling rate exhibits a low-amplitude, slowly changing, and stable random process throughout the cooling sequence. However, regions with internal closed cracks produce significant positive transient peaks due to thermal resistance effects, which are statistically outliers deviating from the overall distribution. Therefore, the essential difference in statistical behavior between normal and outlier regions can be utilized to automatically select reference pixels through an iterative self-confirmation algorithm. The specific steps are as follows: The first step is initial coarse selection. Within the weld area, an initial set of pixels is selected using a uniform grid or random sampling method, with the set numbering no less than 10% of the total number of pixels in the weld area. This step does not require the initial set to be completely defect-free; it only needs to ensure that defective pixels account for a low proportion of the set, so that subsequent statistical parameters are not dominated by outliers.

[0048] The second step is statistical modeling. The root mean square (RMS) value of the cooling rate difference among all pixels in the current set over the entire cooling sequence is calculated as a feature quantity characterizing the overall fluctuation level of each pixel. Distribution analysis is then performed on this feature quantity. It is assumed that the feature quantity of normal pixels follows a normal distribution centered at a relatively small mean, while the feature quantity of defective pixels will significantly deviate from this distribution due to the presence of transient positive peaks.

[0049] The third step is outlier removal. The mean and standard deviation of the current set of features are calculated. The mean plus three times the standard deviation is used as a threshold. Pixels with features exceeding the threshold are identified as suspected defective pixels and removed. The remaining pixels are retained to form a new set of reference pixels.

[0050] The fourth step is iterative convergence. Steps two and three are repeated until the number of pixels removed between two consecutive iterations is zero or less than a preset proportion. This indicates that there are no abnormal pixels with abnormal fluctuations in the set. The pixels that remain at this point constitute a statistically consistent set of defect-free reference pixels. .

[0051] The physical basis of this iterative self-confirmation algorithm lies in the fact that the transient positive peak caused by the internal closed crack is a local phenomenon, occupying only a very small proportion of the weld area in space; while the thermal behavior of the normal area during the post-weld cooling process has a high degree of spatial consistency. By iteratively eliminating statistical outliers, the algorithm can robustly converge to a group of reference pixels that truly represent normal cooling behavior, without relying on any surface-visible information or prior knowledge of defects.

[0052] As an optional initial coarse selection optimization, visible light images can be used before iteration to help eliminate pixel areas with obvious spatter, undercut, or poor forming on the weld surface, further improving the purity of the initial set. However, it should be emphasized that this auxiliary step is not necessary. Even if the initial set contains a certain proportion of internal defect pixels, as long as their proportion is not enough to distort the statistical characteristics of the initial distribution, the above iterative elimination mechanism can still effectively filter them out.

[0053] The following explains some of the terms used in this invention: Surface layer: refers to the extremely thin layer of weld metal that is most exposed and directly participates in external radiative heat dissipation and air convection heat transfer. In this invention, it specifically refers to the deepest surface region of the material that can be detected in the long-wave infrared band.

[0054] Shallow volume: refers to the near-surface volume region of a material located at a certain depth below the surface, which, although not in direct contact with the atmosphere, can still be detected by the mid-wave infrared band through internal heat conduction and infrared radiation.

[0055] Internally closed microcracks: These are planar defects that exist inside the weld, with the two sides of the crack in close contact and no macroscopic opening. These defects are invisible under conventional visual inspection or surface optical testing, but they pose a significant threat to structural integrity.

[0056] Cooling rate difference: refers to the algebraic difference between the shallow volume cooling rate and the surface cooling rate calculated by the algorithm at the same pixel point and at the same time.

[0057] Transient positive peak value: refers to the characteristic waveform peak value on the curve of the difference in cooling rate over time, which is caused by the thermal resistance effect of the internal closed crack, and then drops rapidly in a short period of time.

[0058] In this embodiment, the "short duration" condition is quantified by the relative ratio of the duration of the transient positive peak value to the total cooling time. Its derivation is based on the following physical law: the thermal resistance effect of an internal closed crack is only significant in a specific stage of post-weld cooling. In the initial cooling phase, the temperature difference between the weld interior and surface is large, the heat flux density is high, and the crack's blocking effect on heat conduction is most severe, manifested as a rapid increase in the cooling rate difference to its peak value. As cooling continues, the temperature difference between the interior and surface gradually decreases, the heat flux density decreases, and the influence of the crack's thermal resistance on the surface cooling behavior weakens accordingly, naturally attenuating the difference signal. The duration of this process is determined by the crack depth and the material's thermal diffusivity, but it is always a stage-specific phenomenon in the entire post-weld cooling process, essentially characterized by its "short duration relative to the entire cooling process."

[0059] Based on the above analysis, the quantitative criterion for the transient positive peak value to satisfy "short duration" is set as: the duration from the peak value's initial value to its return to the upper limit of the noise statistical level. It should not exceed the full cooling time. The 20% proportion is determined as follows: the post-weld cooling process can be divided into a rapid cooling stage and a slow cooling stage based on the temperature drop. During the rapid cooling stage, the temperature gradient between the surface and shallow layers is the largest, and the change in heat flux density is also the most drastic. The generation and fading of crack thermal resistance signals both occur during this stage. For typical medium-thick plate steel structure welds, under extremely cold conditions of -25℃, the duration of the rapid cooling stage, from the extinguishing of the welding arc to the weld temperature dropping to near ambient temperature, typically accounts for 15% to 25% of the entire cooling process. Taking a conservative value of 20% as the upper limit of the time window ensures that all valid crack signals are covered while excluding long-term, slowly changing spurious signals caused by environmental disturbances in the later stages of cooling.

[0060] When a candidate positive peak simultaneously meets the following conditions: its amplitude exceeds the upper limit of the noise statistical level, it is spatially clustered, and its duration is... A transient positive peak value is only considered valid if it does not exceed 20% of the total cooling time.

[0061] Example 2: The present invention also provides a robotic vision inspection system for the quality of welds in building steel structures, comprising: The image acquisition module is used to simultaneously acquire a first infrared band sequence thermal image and a second infrared band sequence thermal image after the welding arc is extinguished. The penetration depth of the first infrared band is less than that of the second infrared band, so that the first infrared band sequence thermal image reflects the surface temperature dynamics and the second infrared band sequence thermal image reflects the overall temperature dynamics of the shallow body. The registration module is used to eliminate the spatial misalignment and temporal skew of the first infrared band sequence thermal images and the second infrared band sequence thermal images, ensuring frame-by-frame and pixel-by-pixel correspondence. The temperature sequence extraction module is used to extract the first temperature sequence and the second temperature sequence pixel by pixel from the registered image sequence. The cooling rate calculation module is used to generate surface cooling rate curves and shallow body cooling rate curves by performing numerical differentiation on the first temperature sequence and the second temperature sequence. The differential distribution map generation module is used to calculate the difference between the shallow body cooling rate and the surface cooling rate pixel by pixel, obtain the curve of the cooling rate difference over time, and construct a spatiotemporal distribution map of the cooling rate difference. The transient positive peak value identification and determination module is used to detect the phenomenon of spatially concentrated and short-term positive increase in cooling rate difference in the spatiotemporal distribution map of cooling rate difference, identify the transient positive peak value region where the cooling rate of the shallow layer is reduced compared with the surface layer, and determine the existence and location of closed microcracks in the interior below the surface layer accordingly. The results output module is used to overlay the detected crack location and crack outline onto the weld background image and output the relative burial depth information.

[0062] The various modules in this system work together to form a complete detection system. Preferably, the image acquisition module is deployed at the end effector of the mobile robot, and its core is a dual-band infrared thermal imager containing a dichroic beam splitter, a long-wave infrared focal plane detector, and a mid-wave infrared focal plane detector. The registration module, temperature sequence extraction module, cooling rate calculation module, difference distribution map generation module, and transient positive peak recognition and judgment module are located in the robot's onboard edge computing device or a remote high-performance computer. These modules are implemented based on software instructions, loaded and executed by the processor, to complete the registration of dual-band infrared sequence images, the generation of cooling rate difference distribution maps, and the analysis and judgment of transient crack signals, and transmit the judgment results to the result output module. The result output module may include a display and a data interface, displaying a visible light weld image with pseudo-color annotations of crack burial depth on the display in real time, and sending information such as the crack location coordinates and estimated burial depth to the programmable logic controller (PLC) in the factory automation network through the data interface to trigger subsequent defect grinding or repair welding processes.

[0063] Example 3: The present invention also provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the program to implement a robot visual inspection method for the quality of welds in building steel structures.

[0064] Example 4: The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a robot visual inspection method for weld quality of building steel structures.

[0065] Example 5: To verify the actual detection efficiency of the method of the present invention for internal closed microcracks, a set of Q345qE bridge steel standard test blocks with pre-existing known defects were used for post-weld cooling detection experiments. The test blocks were 300mm × 200mm × 40mm in size, and a 200mm long weld was deposited on the surface of the test blocks using submerged arc welding. Before welding, three horizontal artificial closed cracks with different embedment depths were pre-fabricated inside the test blocks using electrical discharge machining. The crack dimensions were all 10mm long, 5mm wide, and with a gap of less than 0.01mm, to simulate cold cracks caused by excessive post-weld cooling rates in cold regions and interlayer tearing in multi-layer, multi-pass welding. The calibration parameters of the three pre-fabricated cracks are shown in the table below: Table 1: Calibration parameters for prefabricated internal closed cracks ; The experimental environment temperature was controlled at -25℃ to simulate working conditions in extremely cold regions. After the welding arc was extinguished, the image acquisition module of the robot vision inspection system immediately and simultaneously activated the long-wave infrared thermal imager (first infrared band, 8-14μm) and the mid-wave infrared thermal imager (second infrared band, 3-5μm), continuously acquiring a sequence of thermal images of the weld area at a frame rate of 50Hz. The acquisition time was 60 seconds after the welding arc was extinguished, for a total of 3000 frames.

[0066] After non-uniformity correction and bad pixel replacement, the acquired raw infrared thermal images were subjected to coarse registration using affine transformation based on feature point matching and fine registration using free deformation based on B-splines, ensuring pixel-by-pixel spatial correspondence between the two image bands across all 3000 frames. Subsequently, the first and second temperature sequences were extracted from each pixel in the registered image sequence, smoothed using a zero-phase time-domain low-pass filter (cutoff frequency set to 2.5Hz based on the temperature spectrum of the defect-free region), and the instantaneous cooling rate was calculated based on the temperature difference between adjacent frames.

[0067] Select a healthy area within the weld region that is far from the location of the pre-existing crack as the reference pixel set. (A total of 20×20=400 pixels were selected), the root mean square of the cooling rate difference at each time point was calculated and a preset coefficient was used. Construct an upper limit for the statistical level of noise. The table below shows five representative moments during the cooling process. Calculated values ​​and corresponding differences in average cooling rate in normal regions: Table 2: Upper Limit of Noise Statistical Level Dynamically calculated value ; The cooling rate difference curves of all pixels in the weld area are scanned frame by frame, exceeding the corresponding time points. The waveform segments were used as candidate positive peaks. Obvious transient positive peak signals were successfully detected in the coordinate regions corresponding to the three pre-cracks, while no such signal was found in the reference defect-free region. The table below records the key parameters of the transient positive peaks detected at each pre-crack location: Table 3: Transient positive peak value detection results at each pre-existing crack location ; Analysis of the data in Table 3 shows that as the depth of the pre-crack increases, the delay of the transient positive peak value increases. The duration was extended from 3.2s to 22.4s, with a significant increase. The temperature decreased from 1.67℃ / s to 0.29℃ / s, showing a significant negative correlation, which perfectly matches the physical law revealed in this invention that "the shallower the crack, the earlier and stronger the heat flow interruption appears; the deeper the crack, the later and weaker the appearance." The candidate positive peaks at the three detection locations each form a continuous pixel cluster region in space. After spatial connectivity analysis and elimination of isolated noise points, the circumscribed rectangle size of the cluster regions is close to the actual size of the pre-existing crack, 10mm × 5mm, with a two-dimensional positioning deviation of less than 2mm.

[0068] delay and amplitude Substitute into the relative burial depth estimation formula , where constant The pre-calibration experiment determined the depth to be 0.80℃·mm / s. The table below shows a comparison between the estimated and actual burial depths: Table 4: Comparison of Relative Burial Depth Estimation Results with Actual Burial Depth ; As shown in Table 4, the estimated burial depth ranking of the three sets of data (C1 being the shallowest at 0.15 mm, C2 at 0.28 mm, and C3 at 0.45 mm) is completely consistent with the actual burial depth ranking, verifying that the method of this invention can effectively distinguish internal closed microcracks at different depth levels. The estimated burial depth of crack C1 (0.15 mm) is highly consistent with the actual burial depth; the estimated burial depths of cracks C2 and C3 are slightly smaller in absolute value than the actual burial depth. This reflects the physical law that the thermal resistance signal attenuates due to diffusion during long-distance propagation to the surface, causing the sensitivity of delay and amplitude to depth changes to gradually decrease. Within a detection range of 0.15 mm to 0.60 mm, this method can provide relative burial depth information with clear depth-to-shallow distinction. The upper limit of the aforementioned 0.60 mm range is determined by the following physical mechanism: as the crack burial depth increases, the thermal resistance signal undergoes a longer thermal diffusion path during propagation to the surface, causing the signal to continuously attenuate and diffuse during transmission, resulting in a gradual decrease in the sensitivity of delay and amplitude to depth changes. When the burial depth exceeds 0.60 mm, the changes in delay and amplitude tend to saturate, and their product is approximately constant. At this point, although the estimation relationship can still indicate the relative depth trend of the crack, the quantitative resolution has significantly decreased. Therefore, 0.60 mm is determined as the upper limit of the quantitative application of this estimation relationship. For internal closed cracks with a burial depth exceeding 0.60 mm, the method can still detect and locate the crack in two dimensions by the presence or absence of transient positive peaks. However, the estimation result of the relative burial depth should be understood as a qualitative reference for the depth order, rather than a precise depth value.

[0069] Finally, the output module overlays the three detected crack locations with red outlines onto the visible light background image of the weld, and encodes the colors according to the estimated burial depth: C1 is marked with dark red (lightest), C2 with orange, and C3 with yellow (relatively darkest), generating an intuitive visual inspection report of the weld's internal quality.

[0070] Through the complete data verification process described above, this embodiment clearly demonstrates the end-to-end technical effectiveness of the method of the present invention, from dual-band infrared sequence image acquisition, construction of cooling rate difference distribution map, detection of dynamic threshold transient positive peak value, to the location of internal closed microcracks and estimation of relative burial depth.

[0071] The embodiments disclosed in this invention are preferred embodiments, but are not limited thereto. Those skilled in the art can easily understand the spirit of this invention based on the above embodiments and make different extensions and variations, but as long as they do not depart from the spirit of this invention, they are all within the protection scope of this invention.

Claims

1. A robotic vision inspection method for the quality of welds in building steel structures, characterized in that, include: During the natural cooling stage of the weld, the first infrared band sequence thermal image and the second infrared band sequence thermal image of the inspected weld area are acquired simultaneously. The penetration depth of the first infrared band is less than that of the second infrared band. The first infrared band sequence thermal image and the second infrared band sequence thermal image are registered, and the first temperature sequence and the second temperature sequence are extracted pixel by pixel from the registered image sequence; Numerical differentiation is performed based on the first temperature sequence and the second temperature sequence to obtain the surface cooling rate curve and the shallow body cooling rate curve for each pixel. Calculate the difference between the shallow body cooling rate and the surface cooling rate at the same pixel point at the same time, and construct a spatiotemporal distribution map of the cooling rate difference in the weld area; In the spatiotemporal distribution map of the cooling rate difference, transient positive peak values ​​that meet the conditions of spatial aggregation and short duration are identified, and the existence of internal closed microcracks is determined and their locations are marked based on the transient positive peak values.

2. The robotic vision inspection method for weld quality of building steel structures according to claim 1, characterized in that, In the spatiotemporal distribution map of the cooling rate difference, transient positive peak values ​​that satisfy the conditions of spatial clustering and short duration are identified, including: The curve of the cooling rate difference of each pixel over time is scanned, and the waveform segment on the curve whose amplitude exceeds the upper limit of the noise statistical level is identified as a candidate positive peak. The upper limit of the noise statistics level is dynamically determined by multiplying the root mean square of the cooling rate difference value of multiple defect-free reference pixels in the weld area by a preset coefficient.

3. The robotic vision inspection method for weld quality of building steel structures according to claim 2, characterized in that, The identification of transient positive peak values ​​that satisfy the conditions of spatial aggregation and short duration also includes: By analyzing spatial connectivity, adjacent pixels that simultaneously exhibit the candidate positive peak are grouped into a region to eliminate isolated noise points and obtain the transient positive peak value. Extract the time delay of the occurrence of the transient positive peak relative to the cooling start point, as well as the amplitude of the transient positive peak.

4. The robot vision inspection method for weld quality of building steel structures according to claim 3, characterized in that, After determining the existence and location of the internal closed-type microcrack based on the transient positive peak value, the process further includes: Based on the delay and the amplitude, estimate the relative burial depth of the internal closed microcrack from the surface layer; The estimation relationship for the relative burial depth is as follows: the shorter the delay and the larger the amplitude, the shallower the estimated relative burial depth; the longer the delay and the smaller the amplitude, the deeper the estimated relative burial depth.

5. The robot vision inspection method for weld quality of building steel structures according to claim 1, characterized in that, Before performing numerical differentiation based on the first temperature sequence and the second temperature sequence, the method further includes: The first temperature sequence and the second temperature sequence are preprocessed using a zero-phase time-domain low-pass filter. The cutoff characteristics of the zero-phase time-domain low-pass filter are set according to the highest frequency occupied by the temperature change signal in the normal region of the weld.

6. The robotic vision inspection method for weld quality of building steel structures according to claim 1, characterized in that, Registration of the first infrared band sequence thermal image and the second infrared band sequence thermal image includes: Significant feature points are detected in the first infrared band sequence thermal image and the second infrared band sequence thermal image. Based on the feature point matching results, the affine transformation matrix between frames is estimated, and the affine transformation matrix is ​​used to perform spatial coordinate transformation on the second infrared band sequence thermal image. The image after spatial coordinate transformation is locally adjusted using B-spline-based free deformation registration to ensure that the first infrared band sequence thermal image and the second infrared band sequence thermal image maintain pixel-by-pixel spatial correspondence in the cooling sequence.

7. The robot vision inspection method for weld quality of building steel structures according to claim 1, characterized in that, The first infrared band is a long-wave infrared band, and the second infrared band is a mid-wave infrared band; the synchronous acquisition of the first infrared band sequence thermal images and the second infrared band sequence thermal images of the inspected weld area includes: Infrared radiation from the inspected area is separated into bands by a beam splitter and guided to a first imaging channel sensitive to the first infrared band and a second imaging channel sensitive to the second infrared band, respectively. The first imaging channel and the second imaging channel are driven by a synchronization signal controller to record images with the same frame period and the same exposure start time.

8. A robotic vision inspection system for the quality of welds in building steel structures, characterized in that, include: The image acquisition module is used to simultaneously acquire a first infrared band sequence thermal image and a second infrared band sequence thermal image of the inspected weld area during the natural cooling stage of the weld. The penetration depth of the first infrared band is less than that of the second infrared band. The registration and extraction module is used to register the first infrared band sequence thermal image and the second infrared band sequence thermal image, and extract the first temperature sequence and the second temperature sequence pixel by pixel from the registered image sequence; The cooling rate calculation module is used to perform numerical differentiation operations based on the first temperature sequence and the second temperature sequence to obtain the surface cooling rate curve and the shallow body cooling rate curve of each pixel. The difference distribution map generation module is used to calculate the difference between the shallow body cooling rate and the surface cooling rate pixel by pixel, obtain the curve of the cooling rate difference over time, and construct the spatiotemporal distribution map of the cooling rate difference in the weld area. The transient positive peak value identification and determination module is used to identify transient positive peak values ​​that meet the conditions of spatial aggregation and short duration in the spatiotemporal distribution map of the cooling rate difference, and to determine the existence of internal closed microcracks and mark their locations based on the transient positive peak values.

9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images