Steel structure welding quality detection method and system based on machine vision

Abstract

This application relates to the field of welding quality inspection and discloses a machine vision-based method and system for inspecting the welding quality of steel structures. The method includes: acquiring a sequence of original weld images via high-frequency sampling and preprocessing to generate a clear sequence; extracting features from the sequence to obtain static defect indices of the weld; constructing time-series data by arranging features according to time, and calculating the weld symmetry fluctuation amplitude and the trend of molten pool shape change to obtain a dynamic time-series dataset; inputting the dataset into a neural network model to analyze the time-series evolution pattern of the surface profile curve and identify potential defect points; extracting the crack period amplitude sequence from the potential defect points, performing a Fourier transform to obtain a defect frequency distribution map; extracting the frequency peak position from the map and determining the process parameter deviation, generating an alarm and updating the database; determining the sampling frequency deviation value based on the alarm, and adjusting the high-frequency sampling frequency and optimizing dynamic detection through feedback loops. This application improves the real-time performance, accuracy, and parameter adjustment accuracy of the inspection.

Description

Technical Field

[0001] This application relates to the field of welding quality inspection, and in particular to a method and system for inspecting the welding quality of steel structures based on machine vision. Background Technology

[0002] The welding quality of steel structures directly determines the load-bearing capacity and service safety of components. As the requirements for welding precision in large-scale steel structure projects increase, traditional methods relying on manual or offline inspection are no longer sufficient to meet the needs of efficient production. Real-time inspection technology based on machine vision has become the core direction of technological upgrading in the industry and is of great significance to ensuring the reliability of steel structure projects.

[0003] Existing steel structure welding quality inspection technologies suffer from several shortcomings, making it difficult to meet the demands of actual production for real-time performance, accuracy, and adaptability. Firstly, traditional inspection methods primarily rely on static image analysis or offline sampling inspection, such as manual visual inspection of weld surface defects and ultrasonic testing for internal voids. These methods cannot capture the dynamic changes in the molten pool shape and weld contour during welding in real time. When minute fluctuations in process parameters such as welding current and speed cause instantaneous defects like porosity and cracks, missed or delayed detections are likely, leading to defective components entering subsequent processes. Secondly, existing machine vision inspection technologies often focus on pixel feature extraction at a single point in time, failing to construct a time-dimensional feature evolution model. This makes it impossible to analyze the temporal patterns of parameters such as weld symmetry and molten pool fluctuations, hindering the accurate tracing of the periodic causes of defects. For example, it cannot predict the risk of heat-affected zone expansion based on the trend of molten pool shape changes. Third, the detection process lacks an adaptive optimization mechanism. The sampling frequency and image preprocessing parameters are fixed. When dealing with welding scenarios involving steel structures of varying thicknesses, either the sampling frequency is too low, resulting in the loss of key dynamic features, or the poor parameter adaptability leads to incomplete image noise filtering, affecting the accuracy of defect identification. Furthermore, existing technologies rely on empirical thresholds to judge process parameter deviations, failing to incorporate frequency domain analysis to quantify defect occurrence patterns. This results in a lack of data support for parameter adjustments and a high misjudgment rate. These defects are particularly prominent in large-scale steel structure projects. For example, in the welding of bridge steel box girders, even minor defects in insufficient penetration depth, if not detected in time, can cause structural fatigue damage during service, seriously threatening project safety.

[0004] To address the aforementioned issues, this invention proposes a technical approach that integrates machine vision with temporal analysis and feedback optimization. This approach solves the problems of traditional inspection methods, such as difficulty in capturing dynamic defects in real time, inability to trace the periodic causes of defects, and lack of adaptability in the inspection process. It improves the real-time performance, accuracy, and precision of steel structure welding quality inspection and process parameter adjustment. Summary of the Invention

[0005] This application provides a machine vision-based method and system for inspecting the welding quality of steel structures, which solves the problems of traditional inspection methods in steel structure welding process, such as difficulty in capturing dynamic defects in real time, inability to trace the periodic causes of defects, and lack of adaptability in the inspection process. It improves the real-time performance, accuracy, and process parameter adjustment accuracy of steel structure welding quality inspection.

[0006] In a first aspect, this application provides a machine vision-based method for inspecting the welding quality of steel structures, the method comprising:

[0007] Step S101: Obtain the original weld image sequence of the steel structure welding process through high-frequency sampling, and perform image preprocessing on the original weld image sequence to generate a clear weld image sequence;

[0008] Step S102: Extract pixel-level features from each frame of the weld seam image sequence to obtain the static defect index of the weld seam;

[0009] Step S103: Arrange the pixel-level features in chronological order to construct the first time series data, calculate the weld symmetry fluctuation amplitude and the molten pool shape change trend, and obtain a time series dataset reflecting dynamic changes;

[0010] Step S104: Input the time series dataset into the neural network model, analyze the temporal evolution pattern of the surface profile curve, and determine the potential defect occurrence point;

[0011] Step S105: Extract the crack period amplitude sequence from the potential defect occurrence point, perform Fourier transform on the crack period amplitude sequence to obtain frequency characteristic parameters, and generate a defect frequency distribution map.

[0012] Step S106: Extract the frequency peak position from the defect frequency distribution map, determine the deviation of welding process parameters, generate an alarm signal and update the real-time monitoring database;

[0013] Step S107: Determine the sampling frequency deviation value based on the alarm signal, adjust the high-frequency sampling frequency through a feedback loop mechanism, reconstruct the time series dataset and update the neural network model input to optimize the dynamic detection process.

[0014] Secondly, this application provides a machine vision-based steel structure welding quality inspection system, the system comprising:

[0015] The image acquisition module is used to continuously acquire the original weld seam image sequence of the steel structure welding process in a high-frequency sampling manner, and to perform filtering, noise reduction, contrast enhancement, sharpness screening and standardization processing on the sequence frame by frame to output a clear weld seam image sequence.

[0016] The feature extraction module is used to extract the surface contour curve and edge straightness from each frame of the clear weld image sequence using a convolutional neural network, calculate the weld width, penetration depth, internal void volume and heat-affected zone range, and integrate them to generate static defect indicators of the weld.

[0017] The time series construction module is used to arrange the weld symmetry and molten pool shape parameters in the pixel-level features into the first time series data in chronological order, calculate the fluctuation amplitude of weld symmetry and the change trend of molten pool shape, and form a time series dataset that reflects dynamic changes.

[0018] The defect identification module is used to input the time series dataset into the neural network model, analyze the temporal evolution pattern of the surface contour curve, calculate the edge straightness fluctuation value, and determine the potential defect occurrence point.

[0019] The frequency analysis module is used to extract the crack cycle amplitude sequence from the potential defect occurrence point, perform Fourier transform on it to obtain frequency characteristic parameters, and generate and standardize the defect frequency distribution map.

[0020] The deviation alarm module is used to extract the frequency peak position from the defect frequency distribution map, match the welding process parameter association mapping table to determine the deviation, generate an alarm signal and update the real-time monitoring database.

[0021] The feedback optimization module is used to determine the sampling frequency deviation value based on the alarm signal, adjust the high-frequency sampling frequency through a feedback loop mechanism, reconstruct the time series dataset and update the neural network model input, and optimize the dynamic detection process.

[0022] This application proposes a machine vision-based method and system for inspecting the welding quality of steel structures. It is suitable for real-time quality monitoring and process optimization during the welding process of steel structures, and can solve the problems of traditional inspection methods, such as difficulty in capturing dynamic defects in real time, inability to trace the periodic causes of defects, and lack of adaptability in the inspection process. Compared with the prior art, the beneficial effects of the technical solution of this application are at least as follows:

[0023] First, by combining high-frequency sampling and image preprocessing with time-series analysis and neural network modeling, the dynamic features such as the shape of the molten pool and the contour of the weld seam during the welding process are captured and analyzed in real time, effectively identifying instantaneous defects and improving the real-time performance and accuracy of the detection.

[0024] Secondly, by constructing a time series dataset and integrating frequency domain analysis, it is possible to trace the periodic patterns of defect occurrence, identify the correlation between welding process parameter deviations and defect frequency characteristics, and improve the accuracy of defect cause analysis.

[0025] Third, a feedback optimization mechanism is introduced to dynamically adjust the sampling frequency and model input based on alarm signals, which enhances the system's adaptability to different welding scenarios and the self-optimization capability of the detection process, and reduces the false judgment rate and the risk of missed detection. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart illustrating the machine vision-based steel structure welding quality inspection method described in this application.

[0028] Figure 2 This is a flowchart illustrating the process of obtaining static defect indicators of welds during the machine vision-based steel structure welding quality inspection in this application.

[0029] Figure 3 This is a schematic diagram of the process for generating alarm information and update logs during the machine vision-based steel structure welding quality inspection in this application;

[0030] Figure 4 This is a schematic diagram of the steel structure welding quality inspection system based on machine vision in this application. Detailed Implementation

[0031] This application provides a machine vision-based method and system for inspecting the welding quality of steel structures. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the machine vision-based steel structure welding quality inspection method in this application includes:

[0033] Step S101: Obtain the original weld image sequence of the steel structure welding process through high-frequency sampling, and perform image preprocessing on the original weld image sequence to generate a clear weld image sequence.

[0034] In one specific embodiment, step S101 may specifically include the following steps:

[0035] Original weld seam image sequences were obtained from the steel structure welding process using high-frequency sampling.

[0036] For each frame of the original weld seam image sequence, a filtering algorithm is used to remove noise to generate the first image sequence;

[0037] Perform contrast enhancement processing on each frame of the first image sequence, adjust the grayscale values ​​of the image pixels, and generate a second image sequence.

[0038] Based on the second image sequence, a preliminary weld seam image sequence is determined, wherein the pixel values ​​of the preliminary weld seam image sequence meet a preset sharpness threshold;

[0039] The preliminary weld image sequence is standardized to unify image size and format, generating a clear weld image sequence.

[0040] Specifically, in the steel structure welding scenario, a high-speed industrial camera installed next to the welding equipment acquires a sequence of original weld images during the welding process at a high sampling rate of 500-1000 frames per second. This sequence contains continuous image data of changes in the molten pool morphology, arc intensity fluctuations, and weld surface formation during the welding process. Each frame corresponds to weld area information at a specific time point during the welding process, providing complete raw data support for subsequent image preprocessing. To address noise in the original weld image sequence caused by welding arc radiation, welding fumes, and equipment vibration, a filtering algorithm is applied to each frame. This algorithm smooths the image pixel matrix, adjusting the grayscale value of noisy pixels to the average of the grayscale values ​​of effective pixels within a certain surrounding range, eliminating abrupt changes in pixel grayscale values, and making the pixel distribution in the weld area more continuous, thus generating the first image sequence. The filtering algorithm removes noise, initially separating the weld area from the background area, laying the foundation for subsequent weld feature extraction.

[0041] Contrast enhancement is performed on each frame of the first image sequence. First, the number of pixels corresponding to different gray levels in each frame is counted to determine the distribution range of gray values. Then, by adjusting the mapping relationship of pixel gray values, the gray values ​​of pixels originally concentrated in a narrow gray range are stretched to a wider gray range, increasing the gray value difference between the weld area and the substrate area. For example, the gray value difference in the weld edge area, which was originally only 10-20, is increased to 40-60, making the surface contour and edge details of the weld clearer, thus generating the second image sequence. Enhancing contrast strengthens the visual distinguishability of the weld area, ensuring accurate capture of the geometric features of the weld in subsequent processing.

[0042] Based on the second image sequence, the average pixel gradient value of each frame is calculated. The average pixel gradient value reflects the image sharpness by measuring the drastic change in grayscale between adjacent pixels in the image, using the formula... Where M and N are the number of rows and columns of the image, respectively. Each pixel is in x and y The grayscale change rate in the direction. A preset sharpness threshold of 30-50 is set. Images with an average pixel gradient value greater than this threshold are selected to form a preliminary weld seam image sequence. Blurry frames with an average pixel gradient value lower than the threshold due to momentary dense obstruction by welding fumes or brief camera defocusing are removed. This sharpness threshold selection ensures that each frame in the preliminary weld seam image sequence has sufficient detail sharpness to meet the image quality requirements for subsequent feature extraction.

[0043] The initial weld image sequence was standardized by scaling all images to a uniform size of 512×512 pixels and converting them to 8-bit grayscale format. This eliminated differences in size and format between images acquired at different times, resulting in a clear weld image sequence. By standardizing image size and format, the feature extraction logic for different frames is ensured to remain consistent when the convolutional neural network extracts pixel-level features. This avoids additional errors introduced by differences in image specifications and provides a standardized data foundation for the accurate extraction of key features such as weld width and penetration depth.

[0044] Step S102: Extract pixel-level features from each frame of the weld seam image sequence to obtain the static defect index of the weld seam.

[0045] In one specific embodiment, step S102 may specifically include the following steps:

[0046] For each frame in the sequence of clear weld images, a convolutional neural network is used to extract the surface contour curve and edge straightness.

[0047] Calculate the weld width and penetration depth based on the surface profile curve;

[0048] The continuity of the weld edge is identified based on the stated edge straightness;

[0049] A three-dimensional model of the steel structure weld is constructed by combining the surface contour curve and edge straightness to determine the internal void volume;

[0050] Calculate the extent of the heat-affected zone based on the weld width, penetration depth, and internal cavity volume.

[0051] The static defect index of the weld is obtained by weighted summing of the internal cavity volume and the heat-affected zone range.

[0052] Specifically, please refer to Figure 2 For each frame in a clear weld seam image sequence, it is input into a convolutional neural network. The network performs feature extraction through the collaborative operation of multiple convolutional and pooling layers. The convolutional layers use 3×3 kernels to capture local pixel features, exploring the differences in pixel distribution between the weld seam region and the surrounding substrate. The pooling layers, through max pooling, reduce feature dimensionality while retaining crucial geometric information for subsequent analysis. Ultimately, the surface contour curve reflecting the weld seam's surface geometry and the edge straightness characterizing the weld seam's edge regularity are extracted from the image. Leveraging the neural network's automatic feature extraction capabilities, image pixel information is transformed into quantifiable contour and edge parameters, providing standardized foundational data for subsequent defect analysis and avoiding errors caused by subjective human judgment.

[0053] The weld width and penetration depth are calculated based on the extracted surface profile curve. Pixels corresponding to the two edges of the weld are identified on the transverse section of the curve, and the weld width is obtained by calculating the straight-line distance between the two points. Simultaneously, the lowest point of the surface profile curve is located, and the vertical distance from this point to the steel structure substrate surface is calculated to obtain the penetration depth. Weld edge continuity is identified based on edge straightness by calculating the deviation distance between each pixel on the edge line and the fitted straight line. If the deviation distance of 20 consecutive pixels is less than 0.4 mm, the corresponding weld edge segment is considered continuous. If the deviation distance of pixels in a certain area exceeds 0.4 mm, and the deviation distance of 5 adjacent pixels is greater than 0.3 mm, this area is marked as a weld edge interruption point. This transformation of edge continuity judgment into quantifiable deviation calculation solves the problem of traditional inspection methods, which struggle to detect subtle edge interruptions and are prone to missed detections.

[0054] When constructing a 3D model of a steel structure weld by combining surface contour curves and edge straightness, the geometry of the weld's outer surface is built based on the trend of the surface contour curves. Areas where edge straightness is interrupted are used as markers of potential voids within the model. By analyzing the morphology of these interrupted areas in 3D space, the volume of internal voids is determined. Contour and edge features in a 2D image are transformed into 3D volume parameters, solving the technical problem of traditional inspection methods failing to quantify internal voids. When calculating the heat-affected zone (HAZ) range based on weld width, penetration depth, and internal void volume, the product of weld width and penetration depth is used as a basic reference value. This reference value is then corrected by incorporating the proportion of internal void volume to the total weld volume, yielding the HAZ range. A weighted sum of the internal void volume and HAZ range is performed. By pre-setting the weights of these two types of parameters, previously dispersed defect parameters are integrated into a single quantitative index. This weighting process integrates two key defect parameters—internal defect volume and HAZ range—into a single quantitative index, solving the problems of dispersed defect assessment dimensions and the difficulty in comprehensively judging the overall weld quality in traditional inspection methods.

[0055] Step S103: Arrange the pixel-level features in chronological order to construct the first time series data, calculate the weld symmetry fluctuation amplitude and the molten pool shape change trend, and obtain a time series dataset reflecting dynamic changes.

[0056] In one specific embodiment, step S103 may specifically include the following steps:

[0057] Pixel-level feature sequences are extracted from a sequence of clear weld seam images. These pixel-level feature sequences contain previously extracted surface contour curves and edge straightness-related feature data.

[0058] Weld symmetry is obtained by calculating the mirror similarity of pixel distributions on both sides of the weld region in the pixel-level feature sequence.

[0059] The weld pool shape parameters are obtained by extracting the weld pool boundary using an edge detection algorithm and calculating its elliptic fitting degree.

[0060] The weld symmetry values ​​and molten pool shape parameter values ​​obtained at different sampling times are arranged sequentially according to the timestamps to form two sets of one-dimensional sequences, which constitute the first time series data.

[0061] The fluctuation amplitude is obtained by calculating the standard deviation of the weld symmetry sequence in the first time series data;

[0062] Linear regression is used to fit the trend line of the molten pool shape parameter sequence in the first time series data to determine the trend of molten pool shape change over time.

[0063] By weighting the symmetry of the fusion weld and the changing trend of the molten pool shape parameters over time, a time series dataset reflecting the frequency of porosity is obtained.

[0064] Specifically, the pixel-level feature sequence extracted from the clear weld seam image sequence includes surface contour curves and edge straightness-related feature data, which form the basis for subsequent analysis of weld seam dynamic changes. When calculating weld seam symmetry, for the weld seam region in each frame of the pixel-level feature sequence, the weld seam centerline is selected as the axis of symmetry, and the mirror similarity of pixels on both sides in terms of grayscale value and pixel distribution density is calculated. Specifically, the difference in grayscale values ​​of corresponding pixels on both sides is calculated using the Euclidean distance formula, which is: ,in and These are the coordinates of pixels at symmetrical positions on both sides of the weld centerline. The smaller the difference in grayscale values, the higher the mirror similarity and the better the weld symmetry.

[0065] To obtain the molten pool shape parameters, the Canny edge detection algorithm is first used to extract the molten pool boundary. This algorithm determines the edge pixels by calculating the image gradient, and then performs non-maximum suppression and double thresholding on the edge pixels to obtain a clear molten pool boundary contour. Subsequently, an ellipse fitting is performed on the molten pool boundary contour, and the degree of coincidence between the fitted ellipse and the actual boundary contour is calculated, i.e., the ellipse fitting degree. The ratio of the major axis to the minor axis of the ellipse is used as the molten pool shape parameter. The closer the ratio is to 1, the closer the molten pool shape is to a circle; the larger the ratio, the narrower and more elongated the molten pool shape.

[0066] The weld symmetry values ​​and molten pool shape parameters obtained at different sampling times are arranged sequentially according to their corresponding timestamps, forming two independent one-dimensional sequences. These two sequences together constitute the first time series data. The timestamps precisely correspond to each sampling moment in the welding process, ensuring the continuity and accuracy of the data in the time dimension, and providing time dimension support for subsequent analysis of dynamic changes.

[0067] When calculating the amplitude of weld symmetry fluctuation, based on the weld symmetry sequence in the first time series data, the average value of the sequence is first calculated. ,in For the first in the sequence There are several symmetrical values, where n is the sequence length. The variance is then calculated by summing the squared deviations of each value from the mean. The square root of the variance is the standard deviation. The standard deviation is the amplitude of weld symmetry fluctuation. The larger the fluctuation amplitude, the more drastic the change in weld symmetry during the welding process.

[0068] To determine the trend of the molten pool shape over time, a trend line is fitted using linear regression on the molten pool shape parameter sequence in the first time series data. Let the molten pool shape parameter sequence be... The corresponding timestamp sequence is The trend line equation was obtained by fitting using the least squares method. Where b is the bias and slope Reflecting the trend of molten pool shape change, When the time is positive, the shape parameters of the molten pool increase with time, and the molten pool gradually expands; When the value is negative, the shape parameter of the molten pool decreases with time, and the molten pool gradually shrinks.

[0069] The weld symmetry fluctuation amplitude and the molten pool shape change trend were weighted and fused. Based on the relative importance of their influence on the frequency of porosity, the weight of the weld symmetry fluctuation amplitude was set to 0.6, and the weight of the molten pool shape change trend was set to 0.4. The fusion formula is as follows: F represents the integrated index after fusion. The integrated indexes at different time points are arranged by timestamp to form a time series dataset reflecting the frequency of porosity. This dataset can intuitively show the dynamic changes in the frequency of porosity with welding time.

[0070] By extracting pixel-level feature sequences and calculating weld symmetry and molten pool shape parameters, image features are transformed into quantifiable data, solving the problem that existing technologies often focus on single-time-point pixel feature extraction and cannot comprehensively capture the dynamic features of the weld. First-time-series data is constructed, enabling continuous tracking of weld features over time during the welding process, solving the problem that traditional inspection methods cannot capture dynamic defects in real time. The amplitude of weld symmetry fluctuations and the trend of molten pool shape changes are calculated, quantifying the dynamic changes in weld features and solving the problem that existing technologies struggle to analyze the temporal patterns of weld parameters and accurately trace the periodic causes of defects. Weighted fusion forms a time-series dataset reflecting the frequency of porosity occurrence, providing precise data support for subsequent defect identification and process parameter adjustment, solving the problems of insufficient adaptability in the inspection process and lack of data basis for parameter adjustment.

[0071] Step S104: Input the time series dataset into the neural network model, analyze the temporal evolution pattern of the surface profile curve, and determine the potential defect occurrence point.

[0072] In one specific embodiment, step S104 may specifically include the following steps:

[0073] The time series dataset is input into a neural network model, wherein the neural network model is a long short-term memory network.

[0074] The temporal evolution pattern of the surface profile curve is analyzed using the long short-term memory network.

[0075] Calculate the fluctuation value of edge straightness based on the temporal evolution pattern;

[0076] If the fluctuation value of the edge straightness exceeds the preset fluctuation threshold, the potential defect occurrence point is determined;

[0077] Select the surface profile curve segment within the continuous time window corresponding to the potential defect occurrence point, and use Fourier transform to extract the crack period amplitude;

[0078] The crack period amplitude is associated with the timestamp of the corresponding potential defect occurrence point, and the data points are arranged in chronological order according to the timestamps. The data points are then connected by a linear interpolation method to generate a continuous temporal distribution of potential defect occurrence points.

[0079] Specifically, the time-series dataset is input into a Long Short-Term Memory (LSTM) network, which consists of an input layer, hidden layers, and an output layer. The input layer receives temporal feature data from the time-series dataset, such as the amplitude of weld symmetry fluctuations and the trend of molten pool shape changes. The hidden layer captures long-term dependencies in the time-series data through a gating mechanism (input gate, forget gate, and output gate). In the steel structure welding scenario, the time step of the time-series dataset is set to 0.001–0.002 seconds per step, matching the high-frequency sampling frequency. Each time step corresponds to a set of dynamic feature data of the weld. The network learns the changing patterns of the surface profile curve under different welding states through iterative training, and then analyzes the temporal evolution pattern of the surface profile curve. Among them, the forget gate determines the proportion of surface profile curve features retained from previous moments, the input gate controls the weight of new features included at the current moment, and the output gate adjusts the influence of the hidden layer state on the output. Through the synergistic effect of the three gates, the network can accurately identify the continuous changing trend of the surface profile curve during the welding process, such as the gradual rise of the profile curve during the molten pool expansion stage and the stable fluctuation of the profile curve during the weld formation stage.

[0080] When calculating the fluctuation value of edge straightness based on the temporal evolution model, the original edge straightness data corresponding to each time step is first extracted from the temporal evolution model. Then, the sliding window method is used to locally process the original data, with the window size set to 5-10 time steps. The variance of the edge straightness data within each window is calculated, and this variance is the fluctuation value of edge straightness. The variance is calculated using the formula... Where m is the number of data items in the window. For the first in the window Edge flatness values ​​at each time step This represents the average straightness of the edge within the window. The preset fluctuation threshold needs to be determined based on the welding process requirements of the steel structure and historical defect-free welding data. For example, in bridge steel structure welding, the fluctuation threshold is set to 0.02–0.05 mm². If the fluctuation value of the edge straightness within a certain window exceeds this threshold, a potential defect is identified within the corresponding time step. This determination method transforms the dynamic changes in edge straightness into quantifiable fluctuation values ​​for comparison, avoiding the missed detection of instantaneous defects due to static threshold judgments.

[0081] A surface profile curve segment within a continuous time window corresponding to the potential defect occurrence point is selected, with a window duration of 0.01–0.02 seconds, covering key time-series data before and after the potential defect occurrence. Then, a Fourier transform is performed on this curve segment to convert the time-domain surface profile curve data into frequency-domain data. The Fourier transform uses the formula... ,in Let be the amplitude function of the surface profile curve in the time domain. For time, Angular frequency, For the frequency characteristic function in the frequency domain, from The amplitude of the frequency component related to the crack characteristics is extracted. This amplitude is the crack period amplitude. Different types of cracks (such as hot cracks and cold cracks) correspond to different frequency components, and their crack period amplitudes also differ. For example, the frequency component corresponding to hot cracks is 50~100Hz, and the crack period amplitude is 0.1~0.3mm.

[0082] The crack cycle amplitude is correlated with the timestamp of the corresponding potential defect occurrence point to establish a one-to-one correspondence between "timestamp - crack cycle amplitude". The correlated data are arranged in chronological order of timestamps to form discrete data points, and then a linear interpolation method is used to connect these data points. The interpolation formula is as follows: Where t is the interpolation point timestamp, and Timestamps of two adjacent data points , and The crack cycle amplitude corresponds to the timestamp. for The crack cycle amplitude is obtained by interpolation at different times. By filling the gaps between discrete data points through linear interpolation, a continuous temporal distribution of potential defect occurrence points is generated. This distribution can intuitively present the frequency and intensity changes of potential defects in the time dimension, such as the dense distribution of potential defect occurrence points in a certain welding stage and the gradual increase of crack cycle amplitude.

[0083] This study employs a Long Short-Term Memory (LSTM) network to analyze the temporal evolution pattern of surface profile curves. By utilizing a network gating mechanism to capture long-term dependencies in temporal data, it addresses the shortcomings of existing technologies that lack a temporal dimension feature evolution model and cannot analyze the dynamic changes in weld seams, enabling precise tracking of dynamic features during welding. The study calculates edge straightness fluctuations using a sliding window method and combines this with a preset threshold to determine potential defect occurrence points, transforming dynamic fluctuations into quantitative judgment indicators. This solves the problem of traditional static detection easily missing instantaneous defects, improving the timeliness of defect identification. Fourier transform is applied to surface profile curve segments to extract crack period amplitudes, achieving the conversion between time-domain and frequency-domain data. This solves the problem of difficulty in quantifying crack features and provides data support for defect type differentiation. Linear interpolation generates a continuous temporal distribution of potential defect occurrence points, intuitively presenting the temporal distribution characteristics of defects. This solves the problem of being unable to trace the temporal patterns of defect occurrence and provides a clear temporal dimension reference for subsequent process parameter adjustments.

[0084] Step S105: Extract the crack period amplitude sequence from the potential defect occurrence point, perform Fourier transform on the crack period amplitude sequence to obtain frequency characteristic parameters, and generate a defect frequency distribution map.

[0085] In one specific embodiment, step S105 may specifically include the following steps:

[0086] For each potential defect occurrence point, the crack cycle amplitude of the surface profile curve segment within the corresponding continuous time window is retrieved and arranged by timestamp to form a crack cycle amplitude sequence.

[0087] Outliers in the crack cycle amplitude sequence are removed, and the gaps left after removing outliers are filled by linear interpolation to obtain the preprocessed crack cycle amplitude sequence.

[0088] Perform a Fourier transform on the preprocessed crack period amplitude sequence to obtain frequency characteristic parameters;

[0089] Using the dominant frequency in the frequency characteristic parameters as a benchmark, the ratio of the number of defects occurring to the time interval corresponding to the dominant frequency is calculated to determine the defect density. The variation law of the defect density with time is analyzed to obtain the periodicity of the defect density distribution.

[0090] Based on the periodicity of the defect density distribution, identify the frequency components corresponding to the repeated occurrence of defect density, and determine the frequency peak position based on the frequency components.

[0091] Construct a coordinate system with frequency as the horizontal axis and amplitude as the vertical axis, and use the frequency and crack period amplitude corresponding to each frequency peak position as data points, and connect each data point to form an initial defect frequency distribution map;

[0092] The initial defect frequency distribution map is standardized to generate a standardized defect frequency distribution map.

[0093] Specifically, for each potential defect occurrence point, its corresponding continuous time window is first determined, with the window duration set based on the high-frequency sampling frequency. If the sampling frequency is 500 frames / second and the window duration is set to 0.02 seconds, corresponding to 10 frames of image data, the crack cycle amplitude of the surface contour curve segment within this window is retrieved. The crack cycle amplitude of each frame is determined by the frequency features extracted previously using Fourier transform. For example, if the timestamp corresponding to a potential defect occurrence point is 10.00s, and the window covers the period from 10.00s to 10.02s, 10 crack cycle amplitude values ​​within this period are extracted and arranged in chronological order according to their timestamps to form a crack cycle amplitude sequence. Each data point in the sequence is bound to a specific timestamp to ensure temporal integrity.

[0094] When removing outliers from the crack cycle amplitude sequence, the 3σ criterion is used, and the average value of the sequence is calculated first. μ and standard deviation σ In a steel structure welding scenario, if the sequence data is [0.14, 0.15, 0.14, 0.4, 0.15, 0.14, 0.13, 0.14, 0.16, 0.15], the average value is calculated. Standard deviation Exceeding Values ​​within the range of 0.17 ± 0.22926 (i.e., 0.4) are considered outliers and removed. The resulting gaps are filled using linear interpolation. If the data on either side of the gap are... (corresponding timestamp) )and The timestamp for the vacant position is Then fill in the value This method allows for the acquisition of preprocessed crack cycle amplitude sequences, ensuring the continuity and effectiveness of the sequence data.

[0095] The preprocessed crack period amplitude sequence was subjected to a Fourier transform using the Discrete Fourier Transform formula. Where N is the sequence length, For the nth data in the sequence, K For frequency index, These are the frequency characteristic parameters after Fourier transform. The frequency characteristic parameters obtained after transformation contain the amplitude and phase information of different frequency components. For example, in the defect detection caused by welding current fluctuations, the amplitudes corresponding to frequency components such as 50Hz and 100Hz can be extracted. These components reflect the frequency characteristics of the crack period amplitude changing with time, providing frequency domain data support for subsequent defect analysis.

[0096] Using the dominant frequency in the frequency characteristic parameters as a benchmark, the dominant frequency refers to the frequency component with the largest amplitude. For example, after a Fourier transform of a sequence, the 50Hz component has the largest amplitude, and this frequency is the dominant frequency. The number of defects corresponding to this dominant frequency is counted. If, within a 10-second monitoring period, the defect corresponding to the 50Hz frequency component occurs 8 times with a time interval of 10 seconds, then the defect density = 8 / 10 = 0.8 times / second. The defect density at different time periods is calculated using a sliding time window (window duration set to 2 seconds), obtaining a curve showing the change in defect density over time. The fluctuation pattern of the curve is analyzed. If the curve shows a peak every 5 seconds, then the periodicity of the defect density distribution is determined to be 5 seconds. This periodicity reflects the temporal regularity of defect occurrence during welding and is related to the periodic fluctuations of welding process parameters.

[0097] Based on the periodicity of the defect density distribution, identify the frequency components corresponding to the repeated occurrences of defect density. For example, if the period is 5 seconds, the corresponding frequency is 1 / 5 = 0.2Hz. This frequency component is the frequency component corresponding to the repeated occurrence of defect density, and its corresponding amplitude peak position is the frequency peak position. Construct a coordinate system with frequency as the horizontal axis (unit: Hz) and amplitude as the vertical axis (unit: mm). Mark the frequencies corresponding to each frequency peak position (e.g., 0.2Hz, 0.4Hz) and crack period amplitudes (e.g., 0.15mm, 0.12mm) as data points in the coordinate system. Connect the data points with a broken line to form an initial defect frequency distribution map. The map allows for a visual observation of the defect amplitude corresponding to different frequencies, reflecting the frequency distribution characteristics of the defects.

[0098] The initial defect frequency distribution map is standardized by first calculating the maximum amplitude of all data points. Then the amplitude of each data point Convert to normalized amplitude This ensures that the standardized amplitude range is between [0,1]. For example, the initial amplitude data in the graph is [0.15, 0.12, 0.09]. After standardization, the amplitude is [1.0, 0.8, 0.6], while the frequency coordinate remains unchanged, generating a standardized defect frequency distribution map. This map eliminates the influence of amplitude differences between different inspection batches, facilitating a unified comparative analysis of defect frequency distribution under different welding scenarios.

[0099] Constructing and preprocessing a crack cycle amplitude sequence based on timestamps solves the problems of disordered time series and outliers in the original data that lead to analytical errors, providing high-quality data for subsequent frequency domain analysis. Performing Fourier transform on the sequence to obtain frequency characteristic parameters transforms the time-domain data into frequency-domain data, solving the problem that traditional time-domain analysis struggles to capture the periodic characteristics of defects and enabling accurate extraction of defect frequency characteristics. Calculating defect density and analyzing periodicity using the dominant frequency solves the problem of not being able to quantify the temporal regularity of defect occurrence, providing a temporal dimension for tracing defect causes. Constructing and standardizing a defect frequency distribution map solves the problem of difficulty in comparing data under different detection scenarios, facilitating unified analysis of defect frequency distribution characteristics. The overall process, through a combination of time-series and frequency-domain analysis, addresses the problem in the background technology that existing technologies cannot trace the periodic causes of defects, improving the accuracy and comprehensiveness of defect analysis.

[0100] Step S106: Extract the frequency peak position from the defect frequency distribution map, determine the deviation of welding process parameters, generate an alarm signal and update the real-time monitoring database.

[0101] In one specific embodiment, step S106 may specifically include the following steps:

[0102] Extract the frequency peak positions from the defect frequency distribution map, and record the specific frequency value and the corresponding standardized amplitude for each frequency peak position;

[0103] Based on historical defect data of steel structure welding, the peak range of characteristic frequencies corresponding to different welding process parameter deviations is statistically analyzed, and an association mapping table is constructed.

[0104] The specific frequency values ​​of each extracted frequency peak position are matched with the characteristic frequency peak range in the association mapping table to determine the deviation of welding process parameters;

[0105] Based on the determined type of welding process parameter deviation, the alarm level is determined, an alarm signal is generated and output in a synchronous manner using audible and visual signals and digital signals. The alarm signal includes the welding process parameter deviation, the corresponding frequency peak position, the alarm level, and the timestamp of the deviation occurrence.

[0106] The generated alarm signal is associated with the historical record entry of the current welding batch, the real-time monitoring database is updated, and a database update log is generated.

[0107] Specifically, please refer to Figure 3When extracting frequency peak locations from a defect frequency distribution map, the corresponding data of frequency and standardized amplitude in the map are first traversed to identify the turning point where the amplitude changes from rising to falling; this turning point is the frequency peak location. In the steel structure welding scenario, the horizontal axis (frequency) of the defect frequency distribution map is set to a range of 0~200Hz, and the vertical axis (standardized amplitude) is set to a range of 0~1. If, during the traversal, it is found that the standardized amplitude at frequency 18Hz rises from 0.75 to 0.85 and then falls back to 0.7, and the standardized amplitude at frequency 52Hz rises from 0.62 to 0.78 and then falls back to 0.58, these two locations are the frequency peak locations. The standardized amplitude corresponding to 18Hz (0.85) and 52Hz (0.78) is recorded simultaneously to ensure a one-to-one correspondence between the specific frequency value and the standardized amplitude at each frequency peak location, providing basic data support for subsequent welding process parameter deviation judgment.

[0108] When constructing a correlation mapping table based on historical defect data of steel structure welding, historical defect data from past steel structure welding projects (covering scenarios such as factory steel structures and bridge steel structures) are retrieved. This data includes deviation records of process parameters such as welding current, welding voltage, and welding speed, as well as frequency peak data obtained from corresponding defect detection. Statistical analysis revealed that when the welding current deviation is ±6A, the corresponding characteristic frequency peak range is 15–20Hz; when the welding voltage deviation is ±0.6V, the characteristic frequency peak range is 48–55Hz; and when the welding speed deviation is ±0.25m / min, the characteristic frequency peak range is 82–88Hz. The correspondence between "welding process parameter deviation type - characteristic frequency peak range" is organized into a correlation mapping table, where each process parameter deviation type corresponds to a unique characteristic frequency peak range, establishing a quantitative correlation logic between process parameter deviation and frequency characteristics.

[0109] When matching the extracted frequency peak location with the characteristic frequency peak range in the associated mapping table, an interval comparison method is used. If the extracted 18Hz falls within the 15-20Hz range, the corresponding welding process parameter deviation is determined to be welding current deviation ±6A; if 52Hz falls within the 48-55Hz range, the corresponding welding process parameter deviation is determined to be welding voltage deviation ±0.6V. If there is a special case where a frequency value falls within two or more characteristic frequency peak ranges simultaneously (e.g., a frequency of 85Hz falls within both the 82-88Hz range corresponding to welding speed deviation and the 83-87Hz range corresponding to another parameter deviation), then the process parameter deviation records when that frequency appeared in the historical data are retrieved. Priority is given to matching process parameter deviation types with a historical occurrence probability exceeding 75% to ensure the accuracy of welding process parameter deviation determination.

[0110] The alarm level is determined based on the identified type of welding process parameter deviation, and a three-level alarm standard is set: A Level 1 alarm is defined when the absolute value of the welding current deviation exceeds 10%, the absolute value of the welding voltage deviation exceeds 8%, or the absolute value of the welding speed deviation exceeds 0.4 m / min; a Level 2 alarm is defined when the absolute value of the welding current deviation is between 5% and 10%, the absolute value of the welding voltage deviation is between 4% and 8%, or the absolute value of the welding speed deviation is between 0.2 and 0.4 m / min; and a Level 3 alarm is defined when the absolute value of the welding current deviation is within 5%, the absolute value of the welding voltage deviation is within 4%, or the absolute value of the welding speed deviation is within 0.2 m / min. For example, if the rated welding current is 100A and the measured welding current is 106A, the deviation is 6%, which falls under the Level 2 alarm category. An alarm signal is generated containing "welding current deviation ±6A, corresponding peak frequency position 18Hz, alarm level 2, deviation occurrence timestamp 2024-05-20 10:15:30". The welding equipment's built-in audible and visual alarm emits alarms of corresponding levels (level 2 alarm is an orange light and a buzzer at 1-second interval, level 3 alarm is a yellow light and a buzzer at 2-second interval), and transmits them to the welding quality monitoring and control system in digital signal form, achieving synchronous output of audible and visual signals and digital signals.

[0111] When associating the generated alarm signal with the historical record entry of the current welding batch, first determine the unique identifier number of the current welding batch (e.g., GJG20240520-003). Locate the corresponding historical record entry for this batch in the real-time monitoring database, and write information such as the welding process parameter deviation type, corresponding frequency peak position, alarm level, and deviation occurrence timestamp from the alarm signal into the "Process Deviation Record" field under that entry. After the information is written, the real-time monitoring database is automatically updated, and a database update log is generated. The log content includes the update time (e.g., 2024-05-20 10:15:35), the update operation content (adding one secondary record for batch GJG20240520-003), and the operation execution account, ensuring that every database update has a traceable record.

[0112] By extracting the peak frequency locations from the defect frequency distribution map and recording the corresponding data, the frequency domain characteristics are transformed into quantifiable criteria for judging process deviations. This solves the problem that existing technologies rely on manual experience to set thresholds for judging process deviations and lack objective data support. A correlation mapping table is constructed based on historical defect data to establish a direct correlation between process parameter deviations and frequency characteristics, solving the problem that existing technologies cannot accurately trace the cause of defects (i.e., the type of process parameter deviation) through detection data. Frequency values ​​and process deviation types are matched by interval comparison combined with the historical probability priority principle, improving the accuracy of deviation judgment and solving the problem of misjudgment caused by overlapping frequency ranges under a single matching logic. Alarm levels are determined according to deviation types, and audible, visual, and digital signals are output simultaneously to ensure timely alarm information acquisition in different scenarios (on-site operation, remote monitoring), solving the problem of traditional alarm methods being singular and prone to information omissions. Alarm signals are associated with batch records and the database is updated to generate logs, achieving complete retention and traceability of detection data. This solves the problem of scattered detection data and difficulty in welding quality review and process optimization in existing technologies, comprehensively improving the accuracy of welding process parameter deviation judgment and the standardization of detection data management.

[0113] Step S107: Determine the sampling frequency deviation value based on the alarm signal, adjust the high-frequency sampling frequency through a feedback loop mechanism, reconstruct the time series dataset and update the neural network model input to optimize the dynamic detection process.

[0114] In one specific embodiment, step S107 may specifically include the following steps:

[0115] Based on the alarm signal, retrieve the original high-frequency sampling frequency during the period when the deviation occurred, and calculate the difference between it and the optimal sampling frequency threshold determined by the historical defect-free welding scenario to obtain the sampling frequency deviation value.

[0116] Based on the sampling frequency deviation value, the high-frequency sampling frequency is adjusted through a feedback loop mechanism to obtain the adjusted high-frequency sampling frequency.

[0117] The weld seam image sequence of the welding process is acquired at the adjusted high-frequency sampling frequency, and after filtering, noise reduction, contrast enhancement and standardization, a new clear weld seam image sequence is generated.

[0118] Calculate the amplitude of weld symmetry fluctuations and the trend of molten pool shape changes in the new clear weld image sequence, and integrate them to form a new time series dataset;

[0119] The new time-series dataset is input into the Long Short-Term Memory network to optimize its training parameters and complete the optimization of the dynamic detection process.

[0120] Specifically, when determining the sampling frequency deviation value based on the alarm signal, the timestamp of the time period during which the deviation occurred is first extracted from the alarm signal (e.g., 2024-05-20 10:15:30-10:15:40). Based on this timestamp, the original high-frequency sampling frequency for the corresponding time period is retrieved from the sampling records stored in the system. Assuming the retrieval result is 800 frames / second, the optimal sampling frequency threshold determined from historical defect-free welding scenarios (e.g., defect-free welding projects with the same steel structure material, thickness, and welding process) is retrieved from the database. This threshold is obtained by statistically analyzing the correlation data between sampling frequency and defect identification accuracy in historical defect-free scenarios. If the statistics show that the defect identification accuracy reaches 98% with no redundant data when the sampling frequency is 900 frames / second, then the optimal sampling frequency threshold is set to 900 frames / second. The sampling frequency deviation value is calculated using the absolute difference formula, which is: ,in The original high-frequency sampling frequency, To find the optimal sampling frequency threshold, substitute the data to obtain... This value is the sampling frequency deviation.

[0121] When adjusting the high-frequency sampling frequency based on the sampling frequency deviation value through a feedback loop mechanism, the feedback loop mechanism includes a deviation analysis module and a frequency adjustment module. The deviation analysis module determines the type of the sampling frequency deviation value; if the sampling frequency deviation value... If the value is positive, a "increase sampling frequency" command is sent to the frequency adjustment module; if the value is negative, a "decrease sampling frequency" command is sent. When the frame / second is negative, the frequency adjustment module increases the sampling frequency in preset steps (e.g., 50 frames / second). After the first adjustment, the sampling frequency is 800 + 50 = 850 frames / second. At the same time, the adjusted frequency is temporarily stored in the buffer area, and the quality of welding image acquisition is monitored for 10 consecutive seconds at this frequency. If there is still loss of dynamic features in the image due to insufficient sampling frequency (e.g., incomplete details of molten pool shape changes), the frequency is increased to 900 frames / second in steps of 50 frames / second until the acquired image can fully present the dynamic changes of the molten pool and weld contour. Finally, the adjusted high-frequency sampling frequency is 900 frames / second.

[0122] After acquiring a sequence of weld seam images during the welding process at the adjusted high-frequency sampling rate, the sequence is subjected to filtering and noise reduction processing. A Gaussian filtering algorithm is used to smooth each frame of the image, removing noise caused by welding arc light and fumes. Subsequently, contrast enhancement processing is performed, and histogram equalization is used to adjust the distribution of image pixel grayscale values, expanding the grayscale difference between the weld seam area and the substrate area, making the weld seam edge and molten pool boundary clearer. Finally, standardization processing is performed, scaling all images to a size of 512×512 pixels and uniformly converting them to 8-bit grayscale format to generate a new clear weld seam image sequence. This sequence matches the adjusted sampling frequency and can more accurately reflect the dynamic characteristics of the welding process.

[0123] When calculating the weld symmetry fluctuation amplitude and molten pool shape change trend in a new sequence of clear weld images, pixel-level features (surface contour curve, edge straightness) of each frame are first extracted from the sequence and arranged in chronological order to form a feature sequence. To calculate the weld symmetry fluctuation amplitude, the weld symmetry values ​​at each time point are obtained using a mirror similarity algorithm, and then the standard deviation of this numerical sequence is calculated, which is the fluctuation amplitude. To calculate the molten pool shape change trend, linear regression is performed on the molten pool shape parameters (ellipse fit, major axis to minor axis ratio) sequence at each time point to fit the trend line equation. The fluctuation amplitude data and trend line data are integrated by timestamp to form a new time series dataset. The time step of this dataset corresponds to the adjusted sampling frequency (900 frames / second), that is, every 1 / 900th of a second is a time step, and each time step contains a set of fluctuation amplitude and trend data.

[0124] When a new time-series dataset is input into a Long Short-Term Memory (LSTM) network, the network first uses this dataset as new training samples and merges it with the existing training set. The existing training set contains time-series feature data from historical defect-free welding scenarios and historical defective welding scenarios. During the fusion process, the proportion of the two types of data must be maintained consistent with the original training set to avoid data imbalance caused by the addition of new samples. After fusion, the network uses gradient descent to optimize the weight parameters, focusing on adjusting the weight matrices of the input gate, forget gate, and output gate. The input gate weight matrix controls the influence of new feature data at the current time step on the network's hidden state, the forget gate weight matrix determines the retention ratio of hidden states from previous time steps, and the output gate weight matrix adjusts the contribution coefficient of the hidden state to the network output. During optimization, gradient descent calculates the partial derivatives of the objective function with respect to each weight parameter to determine the direction of parameter adjustment, iteratively updating the weight matrix values ​​to adapt the network's feature capture capability for time-series data to the welding dynamic features reflected in the new time-series dataset. The optimization process uses defect recognition accuracy as the objective function, which is derived by comparing the network's judgment results for defect samples in the training set with the actual defect labeling results. When a new time-series dataset is input, the network judges defect samples in the training set based on the initial weight parameters, and calculates the deviation between the judgment results and the actual annotations to obtain the initial defect recognition accuracy. As the weight parameters are iteratively updated, the defect recognition accuracy is continuously calculated after each parameter adjustment, and the trend of accuracy changes is observed. When the accuracy improves to the preset ideal range, and the accuracy fluctuation is less than a set threshold in multiple consecutive iterations, it indicates that the network parameters have fully adapted to the new time-series dataset. At this point, optimization stops, and the training parameters are updated. After the parameter update, the input gate, forget gate, and output gate of the Long Short-Term Memory network can more accurately process the dynamic feature data collected at the adjusted sampling frequency. When analyzing the temporal evolution pattern of surface contour curves, the input gate can more accurately filter key features related to defects, the forget gate can reasonably retain contour evolution information from historical moments, and the output gate can accurately output analysis results reflecting the contour change patterns. This makes the network's judgment of potential defect occurrence points more consistent with the dynamic defect characteristics in actual welding scenarios, thereby optimizing the dynamic detection process and ensuring that the detection process can self-adapt with the sampling frequency adjustment, improving the accuracy of steel structure welding quality inspection.

[0125] By calculating the sampling frequency deviation value using alarm signals and historical best thresholds, the sampling frequency adjustment is correlated with the actual defect situation, solving the problem of fixed sampling frequencies in existing technologies that cannot adapt to dynamic welding scenarios. The feedback loop mechanism adjusts and verifies the sampling frequency step by step according to the deviation value, ensuring that the adjusted frequency can accurately capture dynamic features, solving the problems of lack of feedback verification in sampling frequency adjustment and easy over-adjustment or under-adjustment. Standardized preprocessing is performed on newly acquired images to ensure the consistency of subsequent feature extraction, solving the problem of feature extraction error caused by differences in image specifications under different sampling frequencies. New fluctuation amplitude and trend data are integrated to construct a dataset and optimize network parameters, making the detection model adapt to the adjusted sampling frequency, solving the problem of fixed model parameters in existing technologies that cannot be optimized with changes in sampling conditions. The overall process achieves dynamic adaptation of sampling frequency and detection model through feedback optimization, solving the problems of lack of adaptability and high defect false negative rate in the detection process in the background technology, and improving the accuracy and scene adaptability of dynamic detection.

[0126] Please see Figure 4 The following describes the machine vision-based steel structure welding quality inspection system 400 in the embodiments of this application, including:

[0127] The image acquisition module 401 is used to continuously acquire the original weld seam image sequence of the steel structure welding process in a high-frequency sampling manner, and to perform filtering and noise reduction, contrast enhancement, sharpness screening and standardization processing on the sequence frame by frame to output a clear weld seam image sequence.

[0128] Feature extraction module 402 is used to extract surface contour curves and edge straightness from each frame of a clear weld image sequence using a convolutional neural network, calculate weld width, penetration depth, internal void volume and heat-affected zone range, and integrate to generate weld static defect indexes.

[0129] The time series construction module 403 is used to arrange the weld symmetry and molten pool shape parameters in the pixel-level features into a first time series data in chronological order, calculate the fluctuation amplitude of weld symmetry and the change trend of molten pool shape, and form a time series dataset that reflects dynamic changes.

[0130] The defect identification module 404 is used to input the time series dataset into the neural network model, analyze the temporal evolution mode of the surface contour curve, calculate the edge straightness fluctuation value, and determine the potential defect occurrence point.

[0131] The frequency analysis module 405 is used to extract the crack period amplitude sequence from the potential defect occurrence point, perform Fourier transform on it to obtain frequency characteristic parameters, and generate and standardize the defect frequency distribution map.

[0132] The deviation alarm module 406 is used to extract the frequency peak position from the defect frequency distribution map, match the welding process parameter association mapping table to determine the deviation, generate an alarm signal and update the real-time monitoring database.

[0133] The feedback optimization module 407 is used to determine the sampling frequency deviation value based on the alarm signal, adjust the high-frequency sampling frequency through a feedback loop mechanism, reconstruct the time series dataset and update the neural network model input, and optimize the dynamic detection process.

[0134] Through the collaborative efforts of the aforementioned components, the system constructs a fully closed-loop dynamic inspection system for steel structure welding quality, encompassing "data acquisition, feature extraction, time series analysis, defect identification, frequency analysis, deviation alarm, and feedback optimization." This system achieves fully automated processing from the acquisition of original welding process images to the adaptive optimization of the dynamic inspection process.

[0135] The image acquisition module 401 acquires raw weld images using high-frequency sampling and performs preprocessing, providing a high-quality image data foundation for subsequent modules. The feature extraction module 402 extracts static defect indicators from single-frame images, providing pixel-level features (weld symmetry, molten pool shape parameters) for the time-series construction module 403 and basic data for edge straightness analysis for the defect identification module 404. The time-series construction module 403 integrates the static features into a dynamic time-series dataset, which serves as the core input to the defect identification module 404, supporting the neural network model's analysis of the temporal evolution pattern of the surface contour curve, thereby determining potential defect occurrence points. The potential defect occurrence points output by the defect identification module 404 are directly sent to the frequency analysis module 405 for further analysis. The target data is obtained by taking the crack cycle amplitude sequence, and the standardized defect frequency distribution map generated by the frequency analysis module 405 becomes the key analysis basis for the deviation alarm module 406. The deviation alarm module 406 determines the deviation of process parameters based on the frequency peak position and generates an alarm signal. On the one hand, it updates the real-time monitoring database to achieve data retention, and on the other hand, it transmits the alarm signal to the feedback optimization module 407. After adjusting the sampling frequency according to the alarm signal, the feedback optimization module 407 re-triggers the image acquisition module 401 to acquire data at the new frequency, drives each module to re-execute the data processing flow, and updates the input data of the neural network model in the defect identification module 404 to form a closed-loop optimization, ensuring that the entire detection system can continuously adapt to changes in the welding scenario and improve detection accuracy and dynamic adaptability.

[0136] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for detecting welding quality of a steel structure based on machine vision, characterized in that, The method includes: Step S101: Obtain the original weld image sequence of the steel structure welding process through high-frequency sampling, and perform image preprocessing on the original weld image sequence to generate a clear weld image sequence; Step S102: Extract pixel-level features from each frame of the weld image sequence to obtain the static defect index of the weld; Step S103: Arrange the pixel-level features in chronological order to construct the first time series data, calculate the weld symmetry fluctuation amplitude and the molten pool shape change trend, and obtain a time series dataset reflecting dynamic changes; Step S104: Input the time series dataset into the neural network model, analyze the temporal evolution pattern of the surface profile curve, and determine the potential defect occurrence point; Step S105: Extract the crack period amplitude sequence from the potential defect occurrence point, perform Fourier transform on the crack period amplitude sequence to obtain frequency characteristic parameters, and generate a defect frequency distribution map. Step S106: Extract the frequency peak position from the defect frequency distribution map, determine the deviation of welding process parameters, generate an alarm signal and update the real-time monitoring database; Step S107: Determine the sampling frequency deviation value based on the alarm signal, adjust the high-frequency sampling frequency through a feedback loop mechanism, reconstruct the time series dataset and update the neural network model input to optimize the dynamic detection process; Step S104 includes: The time series dataset is input into a neural network model, wherein the neural network model is a long short-term memory network. The temporal evolution pattern of the surface profile curve is analyzed using the long short-term memory network. Calculate the fluctuation value of edge straightness based on the temporal evolution pattern; If the fluctuation value of the edge straightness exceeds the preset fluctuation threshold, the potential defect occurrence point is determined; Select the surface profile curve segment within the continuous time window corresponding to the potential defect occurrence point, and use Fourier transform to extract the crack period amplitude; The crack period amplitude is associated with the timestamp of the corresponding potential defect occurrence point, and the data points are arranged in chronological order of the timestamps. The data points are then connected by a linear interpolation method to generate a continuous temporal distribution of potential defect occurrence points. Step S105 includes: For each potential defect occurrence point, the crack cycle amplitude of the surface profile curve segment within the corresponding continuous time window is retrieved and arranged by timestamp to form a crack cycle amplitude sequence. Outliers in the crack cycle amplitude sequence are removed, and the gaps left after removing outliers are filled by linear interpolation to obtain the preprocessed crack cycle amplitude sequence. Perform a Fourier transform on the preprocessed crack period amplitude sequence to obtain frequency characteristic parameters; Using the dominant frequency in the frequency characteristic parameters as a benchmark, the ratio of the number of defects occurring to the time interval corresponding to the dominant frequency is calculated to determine the defect density. The variation law of the defect density with time is analyzed to obtain the periodicity of the defect density distribution. Based on the periodicity of the defect density distribution, identify the frequency components corresponding to the repeated occurrence of defect density, and determine the frequency peak position based on the frequency components. Construct a coordinate system with frequency as the horizontal axis and amplitude as the vertical axis, and use the frequency and crack period amplitude corresponding to each frequency peak position as data points, and connect each data point to form an initial defect frequency distribution map; The initial defect frequency distribution map is standardized to generate a standardized defect frequency distribution map.

2. The method of claim 1, wherein, Step S101 includes: Original weld seam image sequences were obtained from the steel structure welding process using high-frequency sampling. For each frame of the original weld seam image sequence, a filtering algorithm is used to remove noise to generate the first image sequence; Perform contrast enhancement processing on each frame of the first image sequence, adjust the grayscale values ​​of the image pixels, and generate a second image sequence. Based on the second image sequence, a preliminary weld seam image sequence is determined, wherein the pixel values ​​of the preliminary weld seam image sequence meet a preset sharpness threshold; The preliminary weld image sequence is standardized to unify image size and format, generating a clear weld image sequence.

3. The method of claim 1, wherein, Step S102 includes: For each frame in the sequence of clear weld images, a convolutional neural network is used to extract the surface contour curve and edge straightness. Calculate the weld width and penetration depth based on the surface profile curve; The continuity of the weld edge is identified based on the stated edge straightness; A three-dimensional model of the steel structure weld is constructed by combining the surface contour curve and edge straightness to determine the internal void volume; Calculate the extent of the heat-affected zone based on the weld width, penetration depth, and internal cavity volume. The static defect index of the weld is obtained by weighted summing of the internal cavity volume and the heat-affected zone range.

4. The method of claim 1, wherein, Step S103 includes: Pixel-level feature sequences are extracted from a sequence of clear weld seam images. These pixel-level feature sequences contain previously extracted surface contour curves and edge straightness-related feature data. Weld symmetry is obtained by calculating the mirror similarity of pixel distributions on both sides of the weld region in the pixel-level feature sequence. The weld pool shape parameters are obtained by extracting the weld pool boundary using an edge detection algorithm and calculating its elliptic fitting degree. The weld symmetry values ​​and molten pool shape parameter values ​​obtained at different sampling times are arranged sequentially according to the timestamps to form two sets of one-dimensional sequences, which constitute the first time series data. The fluctuation amplitude is obtained by calculating the standard deviation of the weld symmetry sequence in the first time series data; Linear regression is used to fit the trend line of the molten pool shape parameter sequence in the first time series data to determine the trend of molten pool shape change over time. By weighting the fluctuation amplitude of the weld symmetry sequence and the changing trend of the molten pool shape parameters over time, a time series dataset reflecting the frequency of porosity is obtained.

5. The method of claim 1, wherein, Step S106 includes: Extract the frequency peak positions from the defect frequency distribution map, and record the specific frequency value and the corresponding standardized amplitude for each frequency peak position; Based on historical defect data of steel structure welding, the peak range of characteristic frequencies corresponding to different welding process parameter deviations is statistically analyzed, and an association mapping table is constructed. The specific frequency values ​​of each extracted frequency peak position are matched with the characteristic frequency peak range in the association mapping table to determine the deviation of welding process parameters; Based on the determined type of welding process parameter deviation, the alarm level is determined, an alarm signal is generated and output in a synchronous manner using audible and visual signals and digital signals. The alarm signal includes the welding process parameter deviation, the corresponding frequency peak position, the alarm level, and the timestamp of the deviation occurrence. The generated alarm signal is associated with the historical record entry of the current welding batch, the real-time monitoring database is updated, and a database update log is generated.

6. The method of claim 1, wherein, Step S107 includes: Based on the alarm signal, retrieve the original high-frequency sampling frequency during the period when the deviation occurred, and calculate the difference between it and the optimal sampling frequency threshold determined by the historical defect-free welding scenario to obtain the sampling frequency deviation value. Based on the sampling frequency deviation value, the high-frequency sampling frequency is adjusted through a feedback loop mechanism to obtain the adjusted high-frequency sampling frequency. The weld seam image sequence of the welding process is acquired at the adjusted high-frequency sampling frequency, and after filtering, noise reduction, contrast enhancement and standardization, a new clear weld seam image sequence is generated. Calculate the amplitude of weld symmetry fluctuations and the trend of molten pool shape changes in the new clear weld image sequence, and integrate them to form a new time series dataset; The new time-series dataset is input into the Long Short-Term Memory network, and its training parameters are optimized to complete the optimization of the dynamic detection process.

7. A machine vision-based steel structure welding quality detection system, characterized in that, For implementing the method as described in any one of claims 1 to 6, the machine vision-based steel structure welding quality inspection system comprises: The image acquisition module is used to continuously acquire the original weld seam image sequence of the steel structure welding process in a high-frequency sampling manner, and to perform filtering, noise reduction, contrast enhancement, sharpness screening and standardization processing on the sequence frame by frame to output a clear weld seam image sequence. The feature extraction module is used to extract the surface contour curve and edge straightness from each frame of the clear weld image sequence using a convolutional neural network, calculate the weld width, penetration depth, internal void volume and heat-affected zone range, and integrate them to generate static defect indicators of the weld. The time series construction module is used to arrange the weld symmetry and molten pool shape parameters in the pixel-level features into the first time series data in chronological order, calculate the fluctuation amplitude of weld symmetry and the change trend of molten pool shape, and form a time series dataset that reflects dynamic changes. The defect identification module is used to input the time series dataset into the neural network model, analyze the temporal evolution pattern of the surface contour curve, calculate the edge straightness fluctuation value, and determine the potential defect occurrence point. The frequency analysis module is used to extract the crack cycle amplitude sequence from the potential defect occurrence point, perform Fourier transform on it to obtain frequency characteristic parameters, and generate and standardize the defect frequency distribution map. The deviation alarm module is used to extract the frequency peak position from the defect frequency distribution map, match the welding process parameter association mapping table to determine the deviation, generate an alarm signal and update the real-time monitoring database. The feedback optimization module is used to determine the sampling frequency deviation value based on the alarm signal, adjust the high-frequency sampling frequency through a feedback loop mechanism, reconstruct the time series dataset and update the neural network model input, and optimize the dynamic detection process.

Images