Method for Detecting Welding Quality of Steel Structures in Construction Projects Based on Deep Neural Network

Abstract

The present invention discloses a method for detecting the welding quality of steel structures in construction projects based on a deep neural network, including: obtaining the original steel structure welding image; preprocessing the original steel structure welding image; extracting the steel structure welding defect features from the preprocessed steel structure welding image through a deep neural network model; detecting the steel structure welding defects through a deep neural network model; and outputting the steel structure welding quality detection result through a deep neural network model. The present invention can intelligently detect the welding quality of steel structures in construction projects, improve the detection efficiency and accuracy, ensure the reliability and safety of steel structures in construction projects, and reduce the labor cost and material cost of detecting the welding quality of steel structures in construction projects.

Description

Technical Field

[0001] The present invention relates to the technical field of welding quality inspection for steel structures in construction engineering, and particularly relates to a method for inspecting the welding quality of steel structures in construction engineering based on a deep neural network. Background Art

[0002] The welded connection area of a building is a potential weak area of the steel structure in construction engineering, and it is prone to fatigue failure and fracture during the service of the steel structure. The quality of the welding area directly affects the safety and reliability of the entire building structure. Therefore, it is very necessary to inspect the welding quality of steel structures in construction engineering. Traditional inspection of the welding quality of steel structures mainly relies on manual experience judgment or simple image processing recognition, which has deficiencies such as low inspection efficiency and limited evaluation accuracy. To improve the inspection efficiency, in this field, ultrasonic, X-ray and other inspection equipment are used to inspect the welding quality of steel structures, but operators still need to be involved throughout the inspection process, resulting in high labor and equipment material costs. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for inspecting the welding quality of steel structures in construction engineering based on a deep neural network, so as to solve the problems of low inspection efficiency, poor accuracy, and high labor and material costs in inspecting the welding quality of steel structures in construction engineering.

[0004] To solve the above technical problems, the present invention provides a method for inspecting the welding quality of steel structures in construction engineering based on a deep neural network, including:

[0005] Step S100, obtaining the original steel structure welding image;

[0006] Step S200, preprocessing the original steel structure welding image;

[0007] Step S300, extracting the steel structure welding defect features from the preprocessed steel structure welding image through a deep neural network model; in step S300, the method for extracting the steel structure welding defect features from the preprocessed image through the deep neural network model includes:

[0008] Step S310, constructing a multi-branch feature encoder based on a deep learning architecture, and decomposing the preprocessed steel structure welding image into different high, middle and low branches of the deep learning architecture;

[0009] Step S320, performing fusion processing on the feature maps obtained from different high, middle and low branches to obtain the multi-dimensional fusion features for defect detection in formula (20):

[0010] (20);

[0011] In formula (20), is the multi-dimensional fusion feature for defect detection, is a certain scale branch, is the set of all scales, is the scale branch weight, is the scale branch output feature;

[0012] Step S400: Detect the steel structure welding defects through a deep neural network model;

[0013] Step S500, output the steel structure welding quality detection result through a deep neural network model.

[0014] Furthermore, for the steel structure welding quality detection method based on a deep neural network provided by the present invention, in step S310, it further includes:

[0015] In the high-scale branch, add the enhanced convolution algorithm formula (15) considering the steel characteristics to calculate the welding defect feature of the high-scale branch:

[0016] (15);

[0017] In formula (15), is the output feature of this scale branch, is the processing function considering local features and global features, is the input feature of the high-scale branch, is the texture direction coefficient, is the texture scale coefficient;

[0018] In the middle-scale branch, calculate the welding defect feature of the middle-scale branch through formula (17):

[0019] (17);

[0020] In formula (17), is the enhanced output feature in the middle-scale branch, is the output feature of this scale branch, is the attention weight map, is the weight adjustment coefficient;

[0021] Among them, the attention weight map in formula (17) is calculated through formula (18):

[0022] (18);

[0023] In formula (18), is the attention weight map, is the S-shaped non-linear activation function, is the convolution function, is the output feature of the high-scale branch, is the mask The mask obtained after Gaussian blur processing, represents element-wise multiplication;

[0024] Among them, the weight adjustment coefficient in formula (17) can be calculated according to formula (19):

[0025] (19);

[0026] In formula (19), is the weight adjustment coefficient, is the mask is the sum of all elements in, is the total number of pixels in the image;

[0027] In the low-scale branch, dilated convolution is added to output the welding defect features of the low-scale branch.

[0028] Furthermore, the method for detecting the welding quality of steel structures in building engineering based on a deep neural network provided by the present invention further includes, in step S320:

[0029] According to the importance of each scale feature for the classification task, the scale branch in formula (21) The weight is adaptively adjusted by formula (29):

[0030] (29);

[0031] In formula (29), is the weight of the scale branch , is the normalization exponential function, T is the gradient temperature coefficient, is the defect classification loss function, is the gradient of the defect classification loss function with respect to the feature .

[0032] Furthermore, the method for detecting steel structure welding defects by the deep neural network model in step S400 of the method for detecting the welding quality of steel structures in building engineering based on a deep neural network provided by the present invention includes:

[0033] Construct a new multi-task decision function module based on the deep learning architecture, and there are three branches: defect classification, defect localization, and confidence evaluation. The three branches share a multi-dimensional fusion feature layer Make collaborative decisions; calculate the overall multi-task decision loss function through formula (21) :[[]]

[0034] (21);

[0035] In formula (21), is the overall multi-task decision loss function, , and are weight coefficients respectively, is the defect classification loss function, is the defect localization loss function; is the confidence loss function;

[0036] Simultaneously backpropagate the losses of the three branches of defect classification, defect localization, and confidence evaluation, continuously adjust the parameters of the deep neural network, achieve collaborative optimization of branch parameters, and improve the interpretability of the detection results;

[0037] After completing the steel structure welding defect detection, the defect classification branch outputs a multi-dimensional probability vector, and each dimension corresponds to a type of steel structure welding defect; the defect localization branch outputs a detection image with the corresponding defect position marked on the original welding image ; the confidence evaluation branch outputs the detection confidence evaluation result.

[0038] Furthermore, in the method for detecting the welding quality of steel structures in construction engineering provided by the present invention, in step S400, it further includes:

[0039] In the defect classification branch, construct an independent fully connected network, and calculate the defect classification loss function using formula (25) , which is used to ensure accurate classification of defect types;

[0040] (25);

[0041] In formula (25), is the defect classification loss function, is the defect category in the preset set of steel structure welding defect types, is the steel defect sensitivity coefficient, is the predicted probability of the model for the positive category, is used to suppress the gradient of easily classified samples;

[0042] In the defect localization branch, construct an independent deconvolution network, and calculate the defect localization loss function using formula (26) , which is used to accurately locate the position of the steel structure welding defect;

[0043] (26);

[0044] In formula (26), is the defect location loss function, used to measure the overlap degree of the location box boundary, is the Euclidean distance, is the diagonal length of the minimum closed region, is the steel structure defect adjustment coefficient;

[0045] In the confidence evaluation branch, an independent network composed of 2 layers of long short-term memory networks and 1 layer of Attention mechanism is constructed, and the confidence loss function is calculated using formula (28) , used to improve the credibility and generalization ability of the detection results;

[0046] (28);

[0047] In formula (28), is the confidence loss function, is the local region feature of the detected welding defect, is the local feature of the qualified weld region detected.

[0048] Furthermore, in the building engineering steel structure welding quality detection method based on a deep neural network provided by the present invention, in step S400, it further includes:

[0049] In the defect location branch, the input feature is locally weighted through formula (27) to reduce the false detection rate of the weld boundary region:

[0050] (27);

[0051] In formula (27), is the weighted local feature, is the original local feature of the recognized defect, is the weld type influence coefficient, exp is the natural exponential function, is the distance from the current pixel point to the weld center line.

[0052] Furthermore, in the building engineering steel structure welding quality detection method based on a deep neural network provided by the present invention, in step S400, the method for the confidence evaluation branch to output the detection confidence evaluation result includes:

[0053] The confidence of the steel structure welding quality detection output through formula (22):

[0054] (22);

[0055] In formula (22), is the confidence level for steel structure welding quality inspection; is a function used to normalize the output to between 0 and 1; is the predicted probability of the model for the positive class; is the bounding box boundary overlap coefficient; is the branch gradient credibility compensation parameter.

[0056] Furthermore, for the steel structure welding quality inspection method based on a deep neural network provided by the present invention, the branch gradient credibility compensation parameter in formula (22) can be calculated using formula (23):

[0057] (23);

[0058] In formula (23), is the branch gradient credibility compensation parameter, is the overall multi-task decision loss function, is the weight coefficient, is a constant, represents the indicator function, which returns 1 when is greater than the threshold and returns 0 otherwise; is the gradient of the total loss function with respect to the weight coefficient .

[0059] Furthermore, for the steel structure welding quality inspection method based on a deep neural network provided by the present invention, in step S500, the method for outputting the steel structure welding quality inspection result through the deep neural network model includes: integrating the output results of the defect classification, defect localization, and confidence evaluation branches through the deep neural network model to generate a steel structure welding quality inspection report.

[0060] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0061] The steel structure welding quality inspection method based on a deep neural network provided by the present invention extracts features from the preprocessed original steel structure welding image through the deep neural network, detects welding defects and outputs the inspection result. By integrating the deep neural network, the intelligent level of steel structure welding quality inspection in construction engineering is improved, the inspection efficiency and accuracy of steel structure welding quality in construction engineering can be significantly enhanced, the reliability and safety of steel structures in construction engineering are ensured, and the labor cost and material cost in the process of steel structure welding quality inspection in construction engineering are reduced, having good economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1Flowchart of a method for detecting the welding quality of steel structures in construction projects based on a deep neural network;

[0063] Figure 2 Example diagram of the preprocessing process of the original steel structure welding image;

[0064] Figure 3 Example diagram of the feature extraction process of the preprocessed steel structure welding image;

[0065] Figure 4 Example diagram of the steel structure welding defect detection process. Specific implementation mode

[0066] The present invention will be described in detail below with reference to the accompanying drawings: According to the following description, the advantages and features of the present invention will be clearer. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise scales, only for the purpose of facilitating and clearly assisting in explaining the purpose of the embodiments of the present invention.

[0067] Please refer to Figures 1 to 4 , an embodiment of the present invention provides a method for detecting the welding quality of steel structures in construction projects based on a deep neural network, which may include the following steps:

[0068] Step S100, obtaining the original steel structure welding image Specifically, an image acquisition device such as a mobile phone, a camera, a drone equipped with a camera, etc. can be used to take an image of the steel structure welding area.

[0069] Step S200, preprocessing the original steel structure welding image including:

[0070] Step S210, preprocessing the original steel structure welding image for illumination correction. Specifically, it may include:

[0071] Step S211, extracting the luminance component of the welding image in the YUV color space, while retaining the chrominance information ( U and V channels) to avoid image distortion. Among them, YUV is a color encoding method that decomposes image information into luminance (Y) and chrominance (U, V).

[0072] For the original welding image obtained under general on-site environmental conditions , the luminance component of the original steel structure welding image is convolved with the Gaussian filter Perform convolution to obtain the estimated value of the illumination component of the original steel structure welding image in formula (1): Perform convolution to obtain the estimated value of the illumination component of the original steel structure welding image in formula (1):

[0073] (1);

[0074] In formula (1), is the estimated value of the illumination component of the original steel structure welding image, is the luminance component of the original steel structure welding image, is the Gaussian filter, represents a two-dimensional convolution operation.

[0075] For the original welding image obtained under complex on-site environmental conditions , when calculating the estimated value of the illumination component , the multi-scale filtering method of formula (2) can be used:

[0076] (2);

[0077] In formula (2), is the estimated value of the illumination component of the original steel structure welding image, where is a shorthand form of is the number of filters, is the standard deviation of the Gaussian function value of. In steel structure welding, good results can be obtained by taking 2, 16, and 32 respectively.

[0078] Among them, the Gaussian filter can be calculated using formula (3):

[0079] (3);

[0080] In formula (3), is the Gaussian filter, 、 is the coordinate position of the current pixel, is the standard deviation of the Gaussian function. It is recommended to take 32 to obtain the low-frequency illumination component, so as to accurately obtain the characteristics of the darker area of the weld.

[0081] Step S212, calculate the initial reflection component according to formula (4) to remove the influence of uneven illumination and retain the color and texture information of the weld area:

[0082] (4);

[0083] In formula (4), is the initial reflection component, is the estimated value of the illumination component of the original steel structure welding image, is the luminance component of the original steel structure welding image, is a constant to avoid division by zero error in the calculation process.

[0084] According to formula (5), enhance the initial reflection component , to highlight the edge details in the welding image:

[0085] (5);

[0086] In formula (5), is the enhanced reflection component; is the reflection coefficient of the steel structure material, used to compensate for the high reflection characteristics of the steel structure surface, and can be obtained by measuring the reflection spectrum of the steel; represents element-wise multiplication; is the estimated value of the illumination component of the gradient; is the sensitivity control parameter, which can be taken as 0.3 for the steel structure; is a constant, taken as 10 for the steel structure -5 that's it, to prevent gradient explosion in the calculation process.

[0087] Step S213, correct the luminance component of the original steel structure welding image according to formula (6), that is, correct the reflection characteristics of the steel structure:

[0088] (6);

[0089] In formula (6), is the luminance component of the corrected original steel structure welding image, is the enhanced reflection component, is the estimated value of the illumination component of the original steel structure welding image, is the mean normalization value of the estimated value of the illumination component .

[0090] Recombine the corrected luminance component with the original chromaticity information ( U and V channels) to obtain the steel structure welding image after illumination correction.

[0091] For the original steel structure welding image By performing illumination correction, it is possible to suppress abnormal brightness distribution (such as local overexposure or underexposure) caused by environmental illumination differences or uneven sensor responses, and restore the overall brightness uniformity of the image; it can enhance the details in the dark or highlight areas, improve the image quality and usability; it can make the image color closer to the real scene and restore the real color and physical properties.

[0092] Step S220, for the steel structure welding image after illumination correction Perform preprocessing of multi-scale noise suppression on the steel structure welding image after illumination correction Perform preprocessing of multi-scale noise suppression on the steel structure welding image after illumination correction to significantly improve the clarity and signal-to-noise ratio of the image, and provide a more reliable data basis for subsequent processing. Specifically, it includes:

[0093] Step S221, based on the Hough transform principle, detect the weld center line and the weld area boundary from the steel structure welding image after illumination correction and generate a binary mask of formula (7) in the weld area :

[0094] (7);

[0095] In formula (7), is the binary mask of the weld area, is the coordinate of the current pixel, is the distance from the current pixel point to the weld center line, is the width of the weld, which is determined by the detected weld boundary line.

[0096] Step S222, perform transformation on the steel structure welding image after illumination correction by a multi-scale decomposition method (such as discrete wavelet transform DWT, etc.), and extract the low-frequency component , intermediate-frequency component and high-frequency component from the image, and perform noise suppression processing on the low, intermediate, and high-frequency components respectively. Perform noise suppression processing on the low-frequency component through formula (8):

[0097] (8);

[0098] In formula (8),

[0099] is the low-frequency component after noise suppression processing, is the low-frequency component of the steel structure welding image after illumination correction, is the Gaussian smoothing filter kernel, represents a two-dimensional convolution operation.

[0100] ​The intermediate frequency component is processed for noise suppression through formula (9):

[0101] (9);

[0102] In formula (9), is the intermediate frequency component after noise suppression processing, is the double threshold function, is the intermediate frequency component of the steel structure welding image after illumination correction, is the binary mask of the weld area, represents the dot product operation.

[0103] Among them is calculated according to formula (10):

[0104] (10);

[0105] In formula (10), is the double threshold function, is the intermediate frequency component , is the preset low threshold for distinguishing the background and the edge area, is the preset high threshold for distinguishing the edge and the defect area, which is determined by the noise distribution of the preset standard welding database; and are the threshold processing functions.

[0106] To effectively highlight the edges and defects in the weld area and facilitate subsequent defect detection, the double threshold function can be calculated using formula (11);

[0107] (11);

[0108] In formula (11), is the double threshold function, is the value of the intermediate frequency component , is the sign function that returns the sign of the input value; is used to increase the retention rate of weak defects in the weld area, and 0.3 can be taken for steel structures; is used to enhance the significant defect features, and 0.15 can be taken for steel structures; is the preset low threshold for distinguishing the background and the edge area, is the preset high threshold for distinguishing the edge and the defect area; is the binary mask of the weld area.

[0109] The high frequency component is processed for noise suppression through formula (12):

[0110] (12);

[0111] In formula (12), is the high-frequency component after noise suppression processing, is the high-frequency component of the steel structure welding image after illumination correction, is the high-frequency component retention parameter for the weld zone, taking 0.2 for steel structures, is the binary mask of the weld area, represents the dot product operation.

[0112] Among them, the processes of noise suppression processing for the high, middle, and low-frequency components in formulas (8), (9), and (12) are sub-band noise processing. By suppressing noise in the high, middle, and low-frequency bands, different frequency band noise interferences can be effectively dealt with, and at the same time, edge blurring or texture loss caused by excessive smoothing can be avoided, and the structural and detail information of the image can be retained to the greatest extent.

[0113] Step S223, recombine the processed components , and through the inverse process of the said transformation (such as wavelet reconstruction) to obtain the denoised steel structure welding image .

[0114] Considering that the differences in the brightness channel of welding defects are the most significant, and gray-scale processing can reduce the computational complexity, the steel structure welding image after illumination correction can be first converted into a gray-scale image and then noise suppression processing is performed.

[0115] Step S230, perform preprocessing of adaptive contrast enhancement on the denoised steel structure welding image , which can improve the accuracy and robustness of feature extraction of the deep neural network model. Specifically, it includes:

[0116] Step S231, perform gray-scale processing on the denoised steel structure welding image through formula (13), that is, image gray-scale conversion:

[0117] (13);

[0118] In formula (13), is the eigenvalue obtained by gray-scale processing of the denoised steel structure welding image, , , are the red, green, and blue channel values of the image, is used to enhance the porosity defect features of steel structure welding.

[0119] Step S232, introduce a closing operation function (such as Morph_Close, etc.) to perform smoothing processing on the weld zone mask to obtain a mask , so as to eliminate the serrations at the mask edge and avoid artifacts in the transition zone between the weld and the base metal in the enhanced image. Introduce a contrast enhancement algorithm (such as Contrast Limited Adaptive Histogram Equalization, CLAHE), and use the mask to segment the weld zone and the non-weld zone. That is, the determination of the weld zone.

[0120] Step S233, highlight the weld detail features in the weld zone by enhancing the contrast (such as selecting a higher clipLimit parameter and a smaller tileGridSize parameter in the CLAHE function), and smooth the background area and reduce noise in the non-weld zone by reducing the contrast (such as selecting a lower clipLimit parameter and a larger tileGridSize parameter in the CLAHE function). The enhanced weld zone and non-weld zone are weighted and fused according to formula (14) to obtain a steel structure welding image with enhanced contrast :

[0121] (14);

[0122] In formula (14), is the steel structure welding image with enhanced contrast, is the image with enhanced contrast in the weld zone, is the image with adjusted contrast in the weld zone, is the mask is the mask obtained after Gaussian blur processing of the mask.

[0123] In the above step S200, aiming at the complex construction environment on the construction site of building engineering, through the three-step preprocessing process of "illumination correction - multi-scale noise suppression - adaptive contrast enhancement" on the original steel structure welding image to improve the accurate recognition of the steel structure welding area and avoid misjudgment. It overcomes the problem of low accuracy of detection results caused by insufficient sensitivity to low-contrast defects in the prior art.

[0124] Step S300, extract the steel structure welding defect features from the preprocessed steel structure welding image through a deep neural network model. Including:

[0125] Step S310, construct a multi-branch feature encoder based on a deep learning architecture (such as HRNet), and decompose the steel structure welding image with enhanced contrast into different branches of high, medium and low levels of the deep learning architecture (such as HRNet) through multi-scale convolutional encoding.

[0126] In the high-scale branch (i.e., the high-resolution branch), the enhanced convolution algorithm formula (15) considering the steel characteristics is added to calculate the welding defect characteristics of the high-scale branch:

[0127] (15);

[0128] In formula (15), is the output feature of this scale branch, is the processing function considering local and global features, is the input feature of the high-scale branch, is the texture direction coefficient, is the texture scale coefficient; and are used to consider the overall texture characteristics of the steel structure weld.

[0129] Specifically, the welding defect characteristics of the high-scale branch can be calculated by formula (16):

[0130] (16);

[0131] In formula (16), is the output feature of this scale branch, represents standard convolution calculation; is the steel feature enhancement coefficient; is the linear filter, is the texture direction coefficient, is the texture scale coefficient; Based on (which can take values of 0°, 45°, 90°, 135°) to match the weld texture direction, based on to control the texture scale matching; represents the convolution operation. Among them, formula (16) is a preferred expression form of formula (15).

[0132] Through the high-scale branch, the steel structure texture characteristics of the steel structure welding image can be enhanced of the steel structure.

[0133] In the middle-scale branch (i.e., the middle-resolution branch), a spatial attention mechanism guided by the weld area is designed, and the welding defect characteristics of the middle-scale branch are calculated by formula (17):

[0134] (17);

[0135] In formula (17), is the enhanced output feature in the middle-scale branch, is the output feature of this scale branch, is the attention weight map, is the weight adjustment coefficient, which is used to adjust the influence degree of the attention weight map on the output features of this branch. By the features in the weld area are enhanced, helping the model to focus more on the feature extraction of the weld area;

[0136] Among them, the attention weight map in formula (17) is calculated by formula (18):

[0137] (18);

[0138] In formula (18), is the attention weight map, which is used to dynamically adjust the feature response weight; is the S-shaped non-linear activation function; is the convolution function; is the output feature of the high-scale branch; is the mask the mask obtained after Gaussian blur processing; represents element-wise multiplication; is the abbreviated form of;

[0139] Among them, the weight adjustment coefficient in formula (17) can be calculated according to formula (19):

[0140] (19);

[0141] In formula (19), is the weight adjustment coefficient, is the mask the sum of all elements in, is the total number of pixels in the image.

[0142] Through the calculation method of formula (19), the value is adaptively adjusted, improving the feature extraction efficiency and accuracy.

[0143] In the low-scale branch (i.e., the low-resolution branch), the welding defect features corresponding to the low-scale branch are output, and at the same time, dilated convolutions are added to improve the ability of this branch to capture global information, avoiding the premature loss of information during the iteration process.

[0144] Step S320, fuse the feature maps obtained from different branches of high, medium, and low scales, that is, multi-scale feature fusion, to obtain the multi-dimensional fusion features for defect detection in formula (20) :

[0145] (20);

[0146] In formula (20), For the multi-dimensional fusion features of defect detection, is a certain scale branch, is the set of all scales, is the scale branch weight, is the scale branch output feature.

[0147] Step S400: Detect the steel structure welding defects through a deep neural network model.

[0148] Construct a new multi-task decision function module based on the deep learning architecture. This module has three branches: defect classification, defect localization, and confidence evaluation. The three branches share the multi-dimensional fusion feature layer for collaborative decision-making; calculate the overall multi-task decision loss function through formula (21) :

[0149] (21);

[0150] In formula (21), is the overall multi-task decision loss function, , and are weight coefficients respectively, is the defect classification loss function, is the defect localization loss function; is the confidence loss function;

[0151] Simultaneously backpropagate the losses of the three branches of defect classification, defect localization, and confidence evaluation, continuously adjust the parameters of the deep neural network, and achieve collaborative optimization of the branch parameters to improve the interpretability of the detection results;

[0152] After completing the detection of steel structure welding defects, the defect classification branch outputs a multi-dimensional probability vector, and each dimension corresponds to a type of steel structure welding defect; the defect types include but are not limited to pores, cracks, lack of fusion, etc.; the defect localization branch outputs a detection image with the corresponding defect position marked on the original welding image ; the confidence evaluation branch outputs the detection confidence evaluation result.

[0153] When calculating the detection confidence evaluation result, the confidence of the steel structure welding quality detection output by formula (22) is:

[0154] (22);

[0155] In formula (22), is the confidence of the steel structure welding quality detection; A function for normalizing the output to between 0 and 1; is the predicted probability of the model for the positive class; is the bounding box boundary overlap coefficient; is the branch gradient credibility compensation parameter.

[0156] Among them, the branch gradient credibility compensation parameter in formula (22) can be calculated using formula (23):

[0157] (23);

[0158] In formula (23), is the branch gradient credibility compensation parameter, is the overall multi-task decision loss function, is the weight coefficient, is a constant, represents the indicator function, when the value of is greater than the threshold returns 1, otherwise returns 0; is the gradient of the total loss function with respect to the weight coefficient

[0159] In the defect classification branch, an independent fully connected network is constructed, and the defect classification loss function is calculated using formula (25) to ensure accurate classification of defect types;

[0160] (25);

[0161] In formula (25), is the defect classification loss function, is the defect category in the preset set of steel structure welding defect types, is the steel defect sensitivity coefficient, is the predicted probability of the model for the positive class, used to suppress the gradient of easily classified samples.

[0162] In the defect localization branch, an independent deconvolution network is constructed, and the defect localization loss function is calculated using formula (26) to accurately locate the position of the steel structure welding defect;

[0163] (26);

[0164] In formula (26), is the defect localization loss function, used to measure the degree of overlap of the bounding box boundary, is the Euclidean distance,​ is the length of the diagonal of the smallest closed region, is the adjustment coefficient for steel structure defects.

[0165] Among them, in the defect location branch, the input feature can be locally weighted processed through formula (27) to reduce the false detection rate of the weld boundary area:

[0166] (27);

[0167] In formula (27), is the locally weighted feature, is the original local feature where a defect is recognized, exp is the natural exponential function, is the influence coefficient of the weld type (such as butt weld, fillet weld, etc.), is the distance from the current pixel point to the weld center line.

[0168] In the confidence evaluation branch, an independent network composed of 2 layers of long short-term memory network (LSTM) and 1 layer of Attention mechanism is constructed, and the confidence loss function of formula (28) is used to improve the credibility and generalization ability of the detection results.

[0169] (28);

[0170] In formula (28), is the confidence loss function, is the local area feature where a welding defect is detected, is the local feature of the weld area detected as qualified.

[0171] To implement the adaptive weight adjustment strategy, a dynamic gradient-aware feature fusion mechanism can be established to allocate the contribution degree of each scale feature to the fused feature. According to the importance of each scale feature to the classification task, the weight of the scale branch in formula (20) is adaptively adjusted through formula (29) :

[0172] (29);

[0173] In formula (29), is the weight of the scale branch , is the normalized exponential function used to convert the gradient into a probability distribution, T is the gradient temperature coefficient used to suppress the problem of gradient disappearance of small-scale features, is the defect classification loss function, is the derivative of the defect classification loss function with respect to the feature The gradient reflects the impact of features on the classification task.

[0174] The weight coefficients are dynamically adjusted during the model training process by monitoring the degree of loss of each branch to prevent a single branch from dominating the training process. The weight coefficients in formula (21) are dynamically adjusted through formula (30):

[0175] (30);

[0176] In formula (30), is the weight value of the weight coefficient at the iteration number when, is the branch The loss reduction rate at this time (the gradient of the last 50 iterations can be calculated using a sliding window), is used to control the steepness of the weight distribution.

[0177] In the above step S400, the steel structure welding defects are detected by the defect classification branch established by the deep neural network model, solving the problem in the prior art that different types of welding defects cannot be accurately distinguished. Through the confidence evaluation branch established by the deep neural network model, the accuracy of the detection result can be reasonably evaluated. Through the defect localization branch established by the deep neural network model, the position of the steel structure welding defect can be accurately identified and located.

[0178] Step S500, output the steel structure welding quality detection result through the deep neural network model. Specifically: the output results of the defect classification, defect localization, and confidence evaluation branches are integrated through the deep neural network model to generate a steel structure welding quality detection report. The deep neural network model includes but is not limited to a deep learning model.

[0179] In the embodiments of the present invention, the same operation symbols in the above formulas have the same meanings. Among them, "*" all represents a two-dimensional convolution operation, "•" all represents a dot product, "⊙" all represents an element-wise multiplication, and "⊗" all represents a convolution operation.

[0180] The method for detecting the welding quality of steel structures in building engineering provided by the embodiments of the present invention extracts features, detects welding defects, and outputs detection results for the preprocessed original steel structure welding images through a deep neural network. By integrating the deep neural network, the intelligent level of detecting the welding quality of steel structures in building engineering is improved, the detection efficiency and accuracy of the welding quality of steel structures in building engineering can be significantly improved, the reliability and safety of steel structures in building engineering are guaranteed, and the labor cost and material cost in the process of detecting the welding quality of steel structures in building engineering are reduced, having good economic benefits.

[0181] The method for detecting the welding quality of steel structures in building engineering based on a deep neural network provided by an embodiment of the present invention directly detects the original steel structure welding image through the deep neural network, avoiding the problem that the method of using an adversarial network to filter the arc light interference during the welding process to improve the welding image quality detection means must rely on time series parameters and cannot meet the requirements of pure vision detection.

[0182] The method for detecting the welding quality of steel structures in building engineering based on a deep neural network provided by an embodiment of the present invention uses common image acquisition devices to obtain the original image of the welding area of the steel structure to be tested. By introducing innovative technical means such as a multi-scale feature fusion mechanism, a dual-domain filtering algorithm, a hierarchical feature decoupling network, and a dynamic parameter adjustment strategy, it provides a high-precision, strong-robustness, and fully automatic vision detection method for the welding quality detection of steel structures in building engineering, aiming to solve the technical problems existing in the welding quality detection of steel structures in building engineering, such as missed detection of tiny defects, poor adaptability to complex environments, lack of defect classification ability, and poor balance between detection efficiency and accuracy.

[0183] The method for detecting the welding quality of steel structures in building engineering based on a deep neural network provided by an embodiment of the present invention detects welding defects through the deep neural network for steel structure welding images, can judge the types of welding defects, can formulate repair measures for the types of steel structure welding quality defects, and can provide a reference for the subsequent welding operations of steel structures to avoid the recurrence of such problems.

[0184] The method for detecting the welding quality of steel structures in building engineering based on a deep neural network provided by an embodiment of the present invention helps to promote the deep integration of machine learning and building engineering and has significant innovation and industrialization value.

[0185] The present invention is not limited to the above specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present invention. Those skilled in the art can make other-level modifications and changes to the present invention. Thus, if these modifications and changes of the present invention fall within the scope of the claims of the present invention, the present invention also intends to include these modifications and changes.

Claims

1. A method for detecting the welding quality of steel structures in construction projects based on a deep neural network, characterized in that, Including: Step S100, obtaining the original steel structure welding image; Step S200, preprocessing the original steel structure welding image; Step S300, extracting the steel structure welding defect features from the preprocessed steel structure welding image through a deep neural network model; In step S300, the method for extracting the steel structure welding defect features from the preprocessed steel structure welding image through a deep neural network model includes: Step S310, construct a multi-branch feature encoder based on a deep learning architecture, and decompose the preprocessed steel structure welding image into different high, middle, and low branches of the deep learning architecture through multi-scale convolutional encoding ; In the high-scale branch, adding an enhanced convolution algorithm considering the steel characteristics, formula (15), to calculate and output the welding defect features of the high-scale branch: （15）； In Equation (15), is the output feature of this scale branch, is a processing function that comprehensively considers local features and global features, is the input feature of the high-scale branch, is the texture orientation coefficient, is the texture scale coefficient; In the middle-scale branch, calculating and outputting the welding defect features of the middle-scale branch through formula (17): （17）； In formula (17), is the enhanced output feature in the mesoscale branch, is the output feature of this scale branch, is the attention weight map, is the weight adjustment coefficient; Among them, the attention weight map in formula (17) is calculated by formula (18): （18）； In formula (18), is the attention weight map, is the S-shaped non-linear activation function, is the convolution function, is the output feature of the high-scale branch, is the mask is the mask obtained after Gaussian blur processing, represents element-wise multiplication; Among them, the weight adjustment coefficient in formula (17) can be calculated according to formula (19): （19）； In formula (19), is the weight adjustment coefficient, is the sum of all elements in the mask , and is the total number of pixels in the image; In the low-scale branch, adding dilated convolution to output the welding defect features of the low-scale branch; Step S320, fusing the feature maps obtained from different high, middle, and low branches to obtain the multi-dimensional fusion features for defect detection in formula (20): （20）； In formula (20), is the multi-dimensional fusion feature for defect detection, is a certain scale branch, is the set of all scales, is the weight of the scale branch ; is the output feature of the scale branch . Step S400: Detecting the steel structure welding defects through a deep neural network model; Step S500, outputting the steel structure welding quality detection result through a deep neural network model.

2. The method for detecting the welding quality of steel structures in construction projects based on a deep neural network according to claim 1, wherein, In step S320, it also includes: According to the importance of each scale feature for the classification task, the weight of the scale branch in formula (21) is adaptively adjusted by formula (29): （29）； In Equation (29), is the weight of the scale branch , is the normalization exponential function, T is the gradient temperature coefficient, is the defect classification loss function, is the gradient of the defect classification loss function with respect to the feature .

3. The method for detecting the welding quality of steel structures in building engineering based on a deep neural network according to claim 1, characterized in that, In step S400, the method for detecting the steel structure welding defects through a deep neural network model includes: Construct a new multi-task decision function module based on the deep learning architecture, which has three branches: defect classification, defect localization, and confidence evaluation. The three branches share a multi-dimensional fusion feature layer for collaborative decision-making; calculate the overall multi-task decision loss function through formula (21) : （21）； In formula (21), is the overall multi-task decision loss function, , and are the weight coefficients respectively, is the defect classification loss function, is the defect localization loss function; is the confidence loss function; Simultaneously backpropagating the losses of the three branches of defect classification, defect localization, and confidence evaluation, continuously adjusting the parameters of the deep neural network, realizing the collaborative optimization of the branch parameters, and improving the interpretability of the detection results; After completing the detection of steel structure welding defects, the defect classification branch outputs a multi-dimensional probability vector, with each dimension corresponding to a type of steel structure welding defect; the defect location branch outputs a detection image with the corresponding defect location marked on the original welding image ; the confidence evaluation branch outputs the detection confidence evaluation result.

4. The method for detecting the welding quality of steel structures in construction projects based on a deep neural network according to claim 3, wherein In step S400, it also includes: In the defect classification branch, an independent fully-connected network is constructed, and the defect classification loss function is calculated using formula (25) , to ensure accurate classification of defect types; （25）； In formula (25), is the defect classification loss function, is the defect category in the preset set of steel structure welding defect types, is the steel defect sensitivity coefficient, is the predicted probability of the model for the positive category, used to suppress the gradient of easily classified samples; In the defect localization branch, an independent deconvolution network is constructed, and the defect localization loss function is calculated using formula (26) to accurately locate the position of steel structure welding defects; （26）； In formula (26), is the defect localization loss function, which is used to measure the overlapping degree of the bounding box boundaries, is the Euclidean distance, is the diagonal length of the minimum closed region, is the steel structure defect adjustment coefficient, N is the number of samples of the local region features; In the confidence evaluation branch, an independent network consisting of two layers of long short-term memory networks and one layer of Attention mechanism is constructed, and the confidence loss function is calculated using formula (28) , which is used to improve the credibility and generalization ability of the detection results; （28）； In formula (28), is the confidence loss function, is the local region feature of the detected welding defect, is the local feature of the detected qualified weld region, N is the number of samples of the local region feature.

5. The method for detecting the welding quality of steel structures in construction projects based on a deep neural network according to claim 3 or 4, characterized in that In step S400, it also includes: In the defect location branch, the input features are locally weighted by formula (27) to reduce the false detection rate of the weld boundary area: （27）； In formula (27), is the weighted local feature, is the original local feature where a defect is recognized, is the weld type influence coefficient, exp is the natural exponential function, is the distance from the current pixel point to the weld center line.

6. The method for detecting the welding quality of steel structures in construction engineering based on a deep neural network according to claim 3, characterized in that, In step S400, the method for the confidence evaluation branch to output the detection confidence evaluation result includes: The confidence of the steel structure welding quality detection output through formula (22): （22）； In formula (22), is the confidence level of steel structure welding quality inspection; is a function used to normalize the output between 0 and 1; is the predicted probability of the model for the positive class; is the localization box boundary overlap coefficient; is the branch gradient credibility compensation parameter.

7. The method for detecting the welding quality of steel structures in building engineering based on a deep neural network according to claim 6, wherein Among them, the branch gradient credibility compensation parameter in formula (22) can be calculated using formula (23): （23）； In formula (23), is the branch gradient credibility compensation parameter, is the overall multi-task decision loss function, is the weight coefficient, is a constant, represents the indicator function, when is greater than the threshold returns 1, otherwise returns 0; is the gradient of the total loss function with respect to the weight coefficient ​ 8. The method for detecting the welding quality of steel structures in construction projects based on a deep neural network according to claim 3, characterized in that, In step S500, the method for outputting the steel structure welding quality detection result through a deep neural network model includes: integrating the output results of the defect classification, defect localization, and confidence evaluation branches through the deep neural network model to generate a steel structure welding quality detection report.

Images