Construction engineering steel structure welding quality detection method based on deep neural network

Abstract

The invention discloses a construction engineering steel structure welding quality detection method based on a deep neural network. The method comprises the steps that an original steel structure welding image is acquired; preprocessing the original steel structure welding image; performing steel structure welding defect feature extraction on the preprocessed steel structure welding image through a deep neural network model; the welding defects of the steel structure are detected through the deep neural network model; and outputting a steel structure welding quality detection result through the deep neural network model. According to the invention, intelligent detection can be carried out on the welding quality of the constructional engineering steel structure, the detection efficiency and accuracy can be improved, the reliability and safety of the constructional engineering steel structure are guaranteed, and the labor cost and material cost of welding quality detection of the constructional engineering steel structure are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of welding quality detection of steel structures in construction projects, and in particular to a welding quality detection method for steel structures in construction projects based on a deep neural network. Background Art

[0002] The welding connection area of ​​a building is a potential weak area of ​​the steel structure in the construction project. It is easy to suffer fatigue damage and fracture during the service life of the steel structure. The quality of the welding area is directly related to the safety and reliability of the entire building structure. Therefore, it is very necessary to detect the welding quality of steel structures in construction projects. Traditional steel structure welding quality detection mainly relies on manual experience judgment or simple image processing and recognition, which has the disadvantages of low detection efficiency and limited evaluation accuracy. In order to improve the detection efficiency, ultrasonic, X-ray and other detection equipment are used to detect the welding quality of steel structures in this field, but the operator still needs to participate in the whole process of the detection process, and the material costs such as manpower and equipment are high. Summary of the invention

[0003] The purpose of the present invention is to provide a method for detecting the welding quality of steel structures in construction projects based on deep neural networks, so as to solve the problems of low detection efficiency, poor accuracy, high manpower cost and high material cost in welding quality detection of steel structures in construction projects.

[0004] In order to solve the above technical problems, the present invention provides a method for detecting welding quality of steel structures in construction projects based on a deep neural network, comprising: Step S100, acquiring the original steel structure welding image; Step S200, preprocessing the original steel structure welding image; Step S300, extracting steel structure welding defect features from the preprocessed steel structure welding image through a deep neural network model; In step S300, the method for extracting steel structure welding defect features from the preprocessed steel structure welding image through a deep neural network model includes: Step S310, construct a multi-branch feature encoder based on a deep learning architecture, and convert the pre-processed steel structure welding image into Decompose into different branches of deep learning architecture; Step S320, the feature maps obtained from different branches of high, medium and low are fused to obtain the multi-dimensional fusion feature for defect detection in formula (20): (20); In formula (20), Multi-dimensional fusion features for defect detection. is a branch of a certain scale, is the set of all scales, Scale branch The weight of Scale branch Output features of Step S400: Detecting steel structure welding defects through a deep neural network model; Step S500: output the steel structure welding quality inspection result through the deep neural network model.

[0005] Furthermore, the method for detecting welding quality of steel structures in construction projects based on deep neural networks provided by the present invention, in step S310, further includes: In the high-scale branch, the enhanced convolution algorithm formula (15) considering the steel characteristics is added to calculate the welding defect characteristics of the output high-scale branch: (15); In formula (15), is the output feature of the scale branch, To comprehensively consider the processing function of local features and overall features, is the input feature of the high-scale branch, is the texture orientation coefficient, is the texture scale coefficient; In the mesoscale branch, the welding defect characteristics of the output mesoscale branch are calculated by formula (17): (17); In formula (17), is the enhanced output feature in the mesoscale branch, is the output feature of the scale branch, is the attention weight map, is the weight adjustment coefficient; The attention weight map in formula (17) Calculated by formula (18): (18); In formula (18), is the attention weight map, is a S-type nonlinear activation function, is the convolution function, is the output feature of the high-scale branch, For mask The mask obtained after Gaussian blur processing, represents element-wise multiplication; The weight adjustment coefficient in formula (17) can be calculated according to formula (19): (19); In formula (19), is the weight adjustment coefficient, For mask The sum of all elements in , is the total number of pixels in the image; In the low-scale branch, a hole convolution is added to output the welding defect features of the low-scale branch.

[0006] Furthermore, the method for detecting welding quality of steel structures in construction projects based on deep neural networks provided by the present invention further includes, in step S320: According to the importance of each scale feature to the classification task, the scale branch in formula (21) The weight of is adaptively adjusted by formula (29): (29); In formula (29), Scale branch The weight of is the normalized exponential function, T is the gradient temperature coefficient, is the defect classification loss function, The loss function for defect classification is gradient.

[0007] Furthermore, the method for detecting welding quality of steel structures in construction projects based on a deep neural network provided by the present invention includes, in step S400, detecting welding defects of steel structures by using a deep neural network model: A new multi-task decision-making function module is built based on the deep learning architecture, and it has three branches: defect classification, defect location, and confidence assessment. The three branches share a multi-dimensional fusion feature layer. Make collaborative decisions; calculate the overall multi-task decision loss function through formula (21) : (twenty one); In formula (21), is the overall multi-task decision loss function, , and are weight coefficients, is the defect classification loss function, Loss function for defect localization; is the confidence loss function; The losses of the three branches of defect classification, defect location and confidence assessment are back-propagated simultaneously, and the parameters of the deep neural network are continuously adjusted to achieve the coordinated optimization of branch parameters and improve the interpretability of the detection results. After completing the steel structure welding defect detection, the defect classification branch outputs a multi-dimensional probability vector, each dimension corresponds to a type of steel structure welding defect; the defect location branch outputs the corresponding defect location marked on the original welding image. The confidence assessment branch outputs the detection confidence assessment result.

[0008] Furthermore, the method for detecting welding quality of steel structures in construction projects based on a deep neural network provided by the present invention, in step S400, further includes: In the defect classification branch, an independent fully connected network is constructed, and the defect classification loss function is calculated using formula (25): , used to ensure accurate classification of defect types; (25); In formula (25), is the defect classification loss function, It is a defect category in the preset steel structure welding defect type set. is the steel defect sensitivity coefficient, is the model’s predicted probability for the positive category, Used to suppress the gradient of easily classified samples; In the defect localization branch, an independent deconvolution network is constructed, and the defect localization loss function is calculated using formula (26): , used to accurately locate the position of welding defects in steel structures; (26); In formula (26), is the defect localization loss function, Used to measure the degree of overlap of the positioning box boundaries. is the Euclidean distance, is the minimum closed region diagonal length, is the steel structure defect adjustment coefficient; In the confidence evaluation branch, an independent network consisting of a 2-layer long short-term memory network and a 1-layer attention mechanism is constructed, and the confidence loss function is calculated using formula (28): , used to improve the credibility and generalization ability of detection results; (28); In formula (28), is the confidence loss function, In order to detect the local area characteristics of welding defects, It is the local features of the weld area detected as qualified.

[0009] Furthermore, the method for detecting welding quality of steel structures in construction projects based on a deep neural network provided by the present invention, in step S400, further includes: In the defect localization branch, the input features The local weighted processing is performed by formula (27) to reduce the false detection rate of the weld boundary area: (27); In formula (27), is the weighted local feature, In order to identify the original local features of the defect, is the influence coefficient of weld type, exp is the natural exponential function, It is the distance from the current pixel to the center line of the weld.

[0010] Furthermore, in the method for detecting welding quality of steel structures in construction projects based on deep neural networks provided by the present invention, in step S400, the method for the confidence assessment branch to output the detection confidence assessment result includes: The confidence level of steel structure welding quality inspection output by formula (22) is: (twenty two); In formula (22), To test the confidence level of steel structure welding quality; Function, used to normalize the output to between 0 and 1; is the predicted probability of the model for the positive category; is the overlap coefficient of the positioning box boundary; is the branch gradient credibility compensation parameter.

[0011] Furthermore, the present invention provides a method for detecting welding quality of steel structures in construction projects based on a deep neural network, wherein the branch gradient credibility compensation parameter in formula (22) is It can be calculated using formula (23): (twenty three); In formula (23), is the branch gradient credibility compensation parameter, is the overall multi-task decision loss function, is the weight coefficient, is a constant, represents the indicator function, when The value is greater than the threshold Returns 1 if yes, otherwise returns 0; is the total loss function for the weight coefficient gradient.

[0012] Furthermore, in the method for detecting welding quality of steel structures in construction projects based on deep neural networks provided by the present invention, in step S500, the method for outputting the welding quality detection results of steel structures through a deep neural network model includes: integrating the output results of defect classification, defect location and confidence assessment branches through a deep neural network model to generate a steel structure welding quality inspection report.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a method for detecting the welding quality of steel structures in construction projects based on a deep neural network. The method extracts features from the preprocessed original steel structure welding image, detects welding defects and outputs the detection results through the deep neural network. The intelligent level of the welding quality detection of steel structures in construction projects is improved by integrating the deep neural network. The detection efficiency and accuracy of the welding quality of steel structures in construction projects can be significantly improved, the reliability and safety of the steel structures in construction projects can be guaranteed, the manpower and material costs in the welding quality detection process of steel structures in construction projects can be reduced, and good economic benefits can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Flowchart of the welding quality detection method of steel structure in construction engineering based on deep neural network; Figure 2 An example diagram of the original steel structure welding image preprocessing process; Figure 3 Example diagram of the feature extraction process of preprocessing steel structure welding images; Figure 4 Example diagram of steel structure welding defect detection process. DETAILED DESCRIPTION

[0015] The present invention is described in detail below in conjunction with the accompanying drawings: The advantages and features of the present invention will become more apparent from the following description. It should be noted that the accompanying drawings are in very simplified form and in non-precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0016] Please refer to Figures 1 to 4 The embodiment of the present invention provides a method for detecting welding quality of steel structures in construction projects based on a deep neural network, which may include the following steps: Step S100, original steel structure welding image Specifically, the image of the welding area of ​​the steel structure can be captured by using image acquisition devices such as mobile phones, cameras, and drones equipped with cameras.

[0017] Step S200, the original steel structure welding image Perform preprocessing. Including: Step S210, the original steel structure welding image Perform preprocessing for light correction. This may include: Step S211, extracting welding image in YUV color space The brightness component , while preserving the chromaticity information ( U and V YUV is a color encoding method that decomposes image information into brightness (Y) and chrominance (U, V).

[0018] For the original welding images acquired under general on-site environmental conditions , the brightness component of the original steel structure welding image is converted into With Gaussian filter Convolution is performed to obtain the estimated value of the illumination component of the original steel structure welding image of formula (1): Convolution is performed to obtain the estimated value of the illumination component of the original steel structure welding image of formula (1): (1); In formula (1), is the estimated value of the illumination component of the original steel structure welding image, is the brightness component of the original steel structure welding image, is a Gaussian filter, Represents a two-dimensional convolution operation.

[0019] For the original welding images obtained under complex on-site environmental conditions , in calculating the estimated value of the illumination component The multi-scale filtering method of formula (2) can be used: (2); In formula (2), is the estimated value of the illumination component of the original steel structure welding image, where for The abbreviation of is the number of filters, is the standard deviation of the Gaussian function In steel structure welding, the values ​​of 2, 16, and 32 can achieve better results.

[0020] The Gaussian filter It can be calculated using formula (3): (3); In formula (3), is a Gaussian filter, 、 is the coordinate position of the current pixel, is the standard deviation of the Gaussian function, and a value of 32 is recommended to obtain the low-frequency illumination component, thereby accurately obtaining the characteristics of the darker areas of the weld.

[0021] Step S212, calculate the initial reflection component according to formula (4) , remove the influence of uneven lighting and retain the color and texture information of the weld area: (4); In formula (4), is the initial reflection component, is the estimated value of the illumination component of the original steel structure welding image, is the brightness component of the original steel structure welding image, is a constant to avoid division by zero errors in the calculation process.

[0022] According to formula (5), the initial reflection component , perform enhancement processing to highlight the edge details in the welding image: (5); In formula (5), is the enhanced reflection component; The reflection coefficient of the steel structure material is used to compensate for the high reflection characteristics of the steel structure surface and can be obtained by measuring the reflection spectrum of the steel; represents element-wise multiplication; Estimated value of the illumination component The gradient of is the sensitivity control parameter, which can be taken as 0.3 for steel structures; is a constant, and for steel structures it is 10 -5 That is, to prevent the gradient from exploding during the calculation process.

[0023] Step S213, according to formula (6), the brightness component of the original steel structure welding image is calculated. Correction is made, i.e. correction of the reflection characteristics of the steel structure: (6); In formula (6), is the brightness component of the corrected original steel structure welding image, is the enhanced reflection component, is the estimated value of the illumination component of the original steel structure welding image, Estimated value of the illumination component The mean normalized value of .

[0024] The corrected brightness component With the original chromaticity information ( U andV channel) to obtain the steel structure welding image after illumination correction .

[0025] Welding images of original steel structures By performing illumination correction, it is possible to suppress abnormal brightness distribution (such as local overexposure or underexposure) caused by differences in ambient lighting or uneven sensor response, and restore the overall brightness balance of the image; it is possible to enhance the details of dark or highlight areas, improve image quality and usability; and it is possible to make image colors closer to real scenes, restoring real colors and physical properties.

[0026] Step S220: Correct the illumination of the steel structure welding image. Perform multi-scale noise suppression preprocessing on the steel structure welding image after illumination correction Multi-scale noise suppression preprocessing is performed to significantly improve the clarity and signal-to-noise ratio of the image, providing a more reliable data basis for subsequent processing. Specifically, it includes: Step S221, based on the Hough transform principle, the steel structure welding image after illumination correction is obtained. The weld centerline and weld area boundary are detected, and a binary mask of formula (7) is generated in the weld area. : (7); In formula (7), is the binary mask of the weld area, is the coordinate of the current pixel, is the distance from the current pixel to the center line of the weld, is the width of the weld, which is determined by the weld boundary line obtained by detection.

[0027] Step S222: Correct the illumination of the steel structure welding image. Through multi-scale decomposition methods (such as discrete wavelet decomposition DWT, etc.), the image is transformed Extract low frequency components from , intermediate frequency component and high frequency components , noise suppression is performed on low, medium and high frequency components respectively.

[0028] The low-frequency component is subjected to noise suppression processing by formula (8): (8); In formula (8), is the low-frequency component after noise suppression processing, is the low-frequency component of the steel structure welding image after illumination correction, is the Gaussian smoothing filter kernel, Represents a two-dimensional convolution operation.

[0029] The noise suppression process of the intermediate frequency component is performed by formula (9): (9); In formula (9), is the intermediate frequency component after noise suppression processing, is a double threshold function, is the intermediate frequency component of the steel structure welding image after illumination correction, is the binary mask of the weld area, Represents the dot product operation.

[0030] in Calculate according to formula (10): (10); In formula (10), is a double threshold function, The intermediate frequency component , A preset low threshold is used to distinguish the background and edge areas. A high threshold is preset to distinguish between edge and defect areas, which is determined by the noise distribution of a preset standard welding database; and is the threshold processing function.

[0031] In order to effectively highlight the edges and defects of the weld area and facilitate subsequent defect detection, the dual threshold function It can be calculated using formula (11); (11); In formula (11), is a double threshold function, The intermediate frequency component The value of is a sign function that returns the sign of the input value; Used to improve the retention rate of weak defects in the weld area, 0.3 can be used for steel structures; Used to enhance significant defect features, 0.15 can be used for steel structures; A preset low threshold is used to distinguish the background and edge areas. A high threshold is preset to distinguish between edge and defect areas; is the binary mask of the weld area.

[0032] The high frequency components are subjected to noise suppression processing by formula (12): (12); In formula (12), is the high frequency component after noise suppression processing, is the high-frequency component of the steel structure welding image after illumination correction, The parameter for retaining high-frequency components in the weld zone is 0.2 for steel structures. is the binary mask of the weld area, Represents the dot product operation.

[0033] The process of noise suppression on high, medium and low frequency components in formulas (8), (9) and (12) is called sub-band noise processing. By suppressing the noise on high, medium and low frequency bands, we can effectively deal with the noise interference of different frequency bands, avoid edge blur or texture loss caused by over-smoothing, and retain the structure and detail information of the image to the maximum extent.

[0034] Step S223: the processed components , and The denoised steel structure welding image is obtained by recombining the inverse process of the transformation (such as wavelet reconstruction). .

[0035] Considering that the difference of welding defects in the brightness channel is most significant, and grayscale processing can reduce the computational complexity, the illumination-corrected steel structure welding image can be First convert it to grayscale image and then perform noise suppression.

[0036] Step S230: De-noising the steel structure welding image Adaptive contrast enhancement preprocessing can improve the accuracy and robustness of deep neural network model feature extraction. Specifically, it includes: Step S231: De-noised steel structure welding image Grayscale processing is performed through formula (13), that is, image grayscale conversion: (13); In formula (13), is the eigenvalue obtained by graying the denoised steel structure welding image. , , are the red, green, and blue channel values ​​of the image, Used to strengthen the porosity defect characteristics of steel structure welding.

[0037] Step S232, introduce a closing function (such as Morph_Close, etc.) to mask the weld area Smoothing is performed to obtain the mask , in order to eliminate the jagged edges of the mask and avoid artifacts in the transition area between the weld and the base material in the enhanced image. Introduce contrast enhancement algorithms (such as contrast-limited adaptive histogram equalization CLANE) to use the mask Segment the weld area and non-weld area, i.e. weld area determination.

[0038] Step S233, the weld area is enhanced by contrast enhancement to highlight weld detail features (such as selecting a higher clipLimit parameter and a smaller tileGridSize parameter in the CLANE function), and the non-weld area is reduced by contrast reduction to smooth the background area and reduce noise (such as selecting a lower clipLimit parameter and a larger tileGridSize parameter in the CLANE function). The enhanced weld area and non-weld area are weighted fused according to formula (14) to obtain a contrast-enhanced steel structure welding image. : (14); In formula (14), This is the contrast-enhanced steel structure welding image. This is the image after contrast enhancement in the weld area. This is the image after contrast adjustment of the weld area. For mask The mask obtained after Gaussian blurring.

[0039] In the above step S200, in view of the complex construction environment of the construction site, the original steel structure welding image is The three-step preprocessing process of "illumination correction-multi-scale noise suppression-adaptive contrast enhancement" is implemented to improve the accurate recognition of the welding area of ​​the steel structure and avoid misjudgment. This overcomes the problem of low accuracy of detection results caused by insufficient sensitivity to low-contrast defects in the existing technology.

[0040] Step S300: The pre-processed steel structure welding image is processed by a deep neural network model. Extract the characteristics of steel structure welding defects. Including: Step S310, construct a multi-branch feature encoder based on a deep learning architecture (such as HRNet), and convert the contrast-enhanced steel structure welding image into Decompose into different branches of deep learning architecture (such as HRNet).

[0041] In the high-scale branch (i.e., high-resolution branch), the enhanced convolution algorithm formula (15) that considers the steel characteristics is added to calculate the welding defect characteristics of the high-scale branch: (15); In formula (15), is the output feature of the scale branch, is the processing function that comprehensively considers local features and overall features, is the input feature of the high-scale branch, is the texture orientation coefficient, is the texture scale coefficient; and Used to consider the overall texture characteristics of steel structure welds.

[0042] Specifically, the welding defect characteristics of the output high-scale branches can be calculated by formula (16): (16); In formula (16), is the output feature of the scale branch, express Standard convolution calculation; is the characteristic enhancement factor of steel; is a linear filter, is the texture orientation coefficient, is the texture scale coefficient; based on (Possible values ​​are 0°, 45°, 90°, 135°) Match the weld texture direction, based on Control texture scale matching; Represents the convolution operation. Formula (16) is a preferred expression of formula (15).

[0043] Enhanced steel structure welding images through high-scale branching Steel structure texture characteristics.

[0044] In the mesoscale branch (i.e., medium-resolution branch), a spatial attention mechanism guided by the weld area is designed, and the welding defect features of the output mesoscale branch are calculated by formula (17): (17); In formula (17), is the enhanced output feature in the mesoscale branch, is the output feature of the scale branch, is the attention weight map, is the weight adjustment coefficient, which is used to adjust the influence of the attention weight map on the output characteristics of the branch. The features of the weld area are enhanced, helping the model to focus more on feature extraction of the weld area; The attention weight map in formula (17) Calculated by formula (18): (18); In formula (18), is the attention weight map, which is used to dynamically adjust the feature response weight; is a S-type nonlinear activation function; is the convolution function; is the output feature of the high-scale branch; For mask The mask obtained after Gaussian blur processing; represents element-wise multiplication; for The abbreviation of The weight adjustment coefficient in formula (17) can be calculated according to formula (19): (19); In formula (19), is the weight adjustment coefficient, For mask The sum of all elements in is the total number of pixels in the image.

[0045] The calculation method of formula (19) realizes Adaptive adjustment of the value can improve the efficiency and accuracy of feature extraction.

[0046] In the low-scale branch (i.e., low-resolution branch), the welding defect features corresponding to the low-scale branch are output, and the ability of the branch to capture global information is improved by adding void convolution to avoid premature loss of information during the iteration process.

[0047] Step S320, the feature maps obtained from different high, medium and low branches are fused, that is, multi-scale feature fusion, to obtain the multi-dimensional fusion feature for defect detection in formula (20) : (20); In formula (20), Multi-dimensional fusion features for defect detection. is a branch of a certain scale, is the set of all scales, Scale branch The weight of Scale branch The output features of .

[0048] Step S400: Detect steel structure welding defects through a deep neural network model.

[0049] A new multi-task decision-making function module is built based on the deep learning architecture. The module has three branches: defect classification, defect location, and confidence assessment. The three branches share a multi-dimensional fusion feature layer. Make collaborative decisions; calculate the overall multi-task decision loss function through formula (21) : (twenty one); In formula (21), is the overall multi-task decision loss function, , and are weight coefficients, is the defect classification loss function, Loss function for defect localization; is the confidence loss function; The losses of the three branches of defect classification, defect location and confidence assessment are back-propagated simultaneously, and the parameters of the deep neural network are continuously adjusted to achieve the coordinated optimization of branch parameters and improve the interpretability of the detection results. After completing the steel structure welding defect detection, the defect classification branch outputs a multi-dimensional probability vector, where each dimension corresponds to a type of steel structure welding defect; the defect types include but are not limited to pores, cracks, lack of fusion, etc.; the defect location branch outputs the corresponding defect location marked on the original welding image. The confidence assessment branch outputs the detection confidence assessment result.

[0050] When calculating the test confidence evaluation results, the steel structure welding quality test confidence output by formula (22) can be for: (twenty two); In formula (22), To test the confidence level of steel structure welding quality; Function, used to normalize the output to between 0 and 1; is the predicted probability of the model for the positive category; is the overlap coefficient of the positioning box boundary; is the branch gradient credibility compensation parameter.

[0051] The branch gradient credibility compensation parameter in formula (22) is It can be calculated using formula (23): (twenty three); In formula (23), is the branch gradient credibility compensation parameter, is the overall multi-task decision loss function, is the weight coefficient, is a constant, represents the indicator function, when The value is greater than the threshold Returns 1 if yes, otherwise returns 0; is the total loss function for the weight coefficient gradient.

[0052] In the defect classification branch, an independent fully connected network is constructed, and the defect classification loss function is calculated using formula (25): , used to ensure accurate classification of defect types; (25); In formula (25), is the defect classification loss function, It is a defect category in the preset steel structure welding defect type set. is the steel defect sensitivity coefficient, is the model’s predicted probability for the positive category, Used to suppress gradients of easily classified samples.

[0053] In the defect localization branch, an independent deconvolution network is constructed, and the defect localization loss function is calculated using formula (26): , used to accurately locate the position of welding defects in steel structures; (26); In formula (26), is the defect localization loss function, Used to measure the degree of overlap of the positioning box boundaries. is the Euclidean distance, is the minimum closed region diagonal length, It is the steel structure defect adjustment coefficient.

[0054] Among them, in the defect location branch, the input features can be first The local weighted processing is performed by formula (27) to reduce the false detection rate of the weld boundary area: (27); In formula (27), is the weighted local feature, In order to identify the original local features of the defect, exp is a natural exponential function. is the influence coefficient of weld type (such as butt weld, fillet weld, etc.), It is the distance from the current pixel to the center line of the weld.

[0055] In the confidence evaluation branch, an independent network consisting of a 2-layer long short-term memory network (LSTM) and a 1-layer attention mechanism is constructed, and the confidence loss function of formula (28) is used. , which is used to improve the credibility and generalization ability of the detection results.

[0056] (28); In formula (28), is the confidence loss function, In order to detect the local area characteristics of welding defects, It is the local features of the weld area detected as qualified.

[0057] In order to implement the adaptive weight adjustment strategy, a dynamic gradient-aware feature fusion mechanism can be established to allocate the contribution of each scale feature to the fusion feature. According to the importance of each scale feature to the classification task, the scale branch in formula (20) is adjusted by formula (29). The weights are adaptively adjusted: (29); In formula (29), Scale branch The weight of is a normalized exponential function used to convert the gradient into a probability distribution, T is the gradient temperature coefficient, which is used to suppress the gradient annihilation problem of small-scale features, is the defect classification loss function, The loss function for defect classification is The gradient of , reflects the influence of the feature on the classification task.

[0058] The weight coefficient is dynamically adjusted during the model training process by monitoring the degree of loss of each branch to prevent a single branch from dominating the training process. The weight coefficient in formula (21) is dynamically adjusted through formula (30): (30); In formula (30), is the weight coefficient in the number of iterations The weight value when For branch The loss decrease rate at this time (the sliding window can be used to calculate the gradient of the last 50 iterations), Used to control the steepness of the weight distribution.

[0059] In the above step S400, the steel structure welding defects are detected by the defect classification branch established by the deep neural network model, which solves the problem that different types of welding defects cannot be accurately distinguished in the prior art. The confidence assessment branch established by the deep neural network model can reasonably evaluate the accuracy of the detection results. The defect location branch established by the deep neural network model can accurately identify and locate the location of the steel structure welding defects.

[0060] Step S500: Output the steel structure welding quality inspection results through the deep neural network model. Specifically, the output results of the defect classification, defect location and confidence assessment branches are integrated through the deep neural network model to generate a steel structure welding quality inspection report. The deep neural network model includes but is not limited to a deep learning model.

[0061] The same operation symbols in the above formulas in the embodiments of the present invention have the same meanings, where “*” represents a two-dimensional convolution operation, “•” represents a dot product, “⊙” represents an element-by-element multiplication, and “⊗” represents a convolution operation.

[0062] The embodiment of the present invention provides a method for detecting the welding quality of steel structures in construction projects based on a deep neural network. The method extracts features from the preprocessed original steel structure welding image, detects welding defects and outputs the detection results through the deep neural network. The intelligent level of the welding quality detection of steel structures in construction projects is improved by integrating the deep neural network. The detection efficiency and accuracy of the welding quality of steel structures in construction projects can be significantly improved, the reliability and safety of steel structures in construction projects can be ensured, the manpower and material costs in the welding quality detection process of steel structures in construction projects can be reduced, and good economic benefits can be achieved.

[0063] The method for detecting welding quality of steel structures in construction projects based on deep neural networks provided in an embodiment of the present invention directly detects the original steel structure welding image through a deep neural network, thereby avoiding the problem that the detection method that uses an adversarial network to filter arc interference during the welding process to improve the welding image quality must rely on timing parameters and cannot meet the needs of pure visual detection.

[0064] The embodiment of the present invention provides a method for detecting the welding quality of steel structures in construction projects based on deep neural networks. It uses common image acquisition equipment to obtain the original image of the welding area of ​​the steel structure to be tested. By introducing innovative technical means such as multi-scale feature fusion mechanism, dual-domain filtering algorithm, hierarchical feature decoupling network, and dynamic parameter adjustment strategy, it provides a high-precision, highly robust, and fully automatic visual inspection method for the welding quality inspection of steel structures in construction projects. The method aims to solve the technical problems existing in the welding quality inspection of steel structures in construction projects, such as missed detection of tiny defects, poor adaptability to complex environments, lack of defect classification ability, and poor balance between detection efficiency and accuracy.

[0065] The method for detecting welding quality of steel structures in construction projects based on deep neural networks provided in an embodiment of the present invention can detect welding defects through deep neural network steel structure welding images, can judge the type of welding defects, can formulate repair measures for the type of welding quality defects of steel structures, and can provide a reference for subsequent welding operations of steel structures to avoid the recurrence of such problems.

[0066] The method for detecting welding quality of steel structures in construction projects based on deep neural networks provided in an embodiment of the present invention is helpful to promote the deep integration of machine learning and construction projects, and has significant innovation and industrial value.

[0067] The present invention is not limited to the above-mentioned specific implementation modes. Obviously, the above-mentioned embodiments are only some embodiments of the embodiments of the present invention, but not all embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention. Those skilled in the art can make other levels of modifications and changes to the present invention. In this way, if these modifications and changes of the present invention fall within the scope of the claims of the present invention, the present invention is also intended to include these changes and changes.

Claims

1. A method for detecting welding quality of steel structures in construction projects based on deep neural networks, characterized in that: include: Step S100, acquiring the original steel structure welding image; Step S200, preprocessing the original steel structure welding image; Step S300, extracting steel structure welding defect features from the preprocessed steel structure welding image using a deep neural network model; In step S300, the method for extracting the preprocessed steel structure welding defect features through a deep neural network model includes: Step S310, construct a multi-branch feature encoder based on a deep learning architecture, and convert the pre-processed steel structure welding image into Decompose into different branches of deep learning architecture; Step S320, the feature maps obtained from different branches of high, medium and low are fused to obtain the multi-dimensional fusion feature for defect detection in formula (20): （20）； In formula (20), Multi-dimensional fusion features for defect detection. is a branch of a certain scale, is the set of all scales, Scale branch The weight of Scale branch Output features of Step S400: Detecting steel structure welding defects through a deep neural network model; Step S500: output the steel structure welding quality inspection result through the deep neural network model.

2. The method for detecting welding quality of steel structures in construction projects based on deep neural networks according to claim 1 is characterized in that: In step S310, it also includes: In the high-scale branch, the enhanced convolution algorithm formula (15) considering the steel characteristics is added to calculate the welding defect characteristics of the output high-scale branch: （15）； In formula (15), is the output feature of the scale branch, To comprehensively consider the processing function of local features and overall features, is the input feature of the high-scale branch, is the texture orientation coefficient, is the texture scale coefficient; In the mesoscale branch, the welding defect characteristics of the output mesoscale branch are calculated by formula (17): （17）； In formula (17), is the enhanced output feature in the mesoscale branch, is the output feature of the scale branch, is the attention weight map, is the weight adjustment coefficient; The attention weight map in formula (17) Calculated by formula (18): （18）； In formula (18), is the attention weight map, is a S-type nonlinear activation function, is the convolution function, is the output feature of the high-scale branch, For mask The mask obtained after Gaussian blur processing, represents element-wise multiplication; The weight adjustment coefficient in formula (17) can be calculated according to formula (19): （19）； In formula (19), is the weight adjustment coefficient, For mask The sum of all elements in is the total number of pixels in the image; In the low-scale branch, a hole convolution is added to output the welding defect features of the low-scale branch.

3. The method for detecting welding quality of steel structures in construction projects based on deep neural networks according to claim 2 is characterized in that: In step S320, it also includes: According to the importance of each scale feature to the classification task, the scale branch in formula (21) The weight of is adaptively adjusted by formula (29): （29）； In formula (29), Scale branch The weight of is the normalized exponential function, T is the gradient temperature coefficient, is the defect classification loss function, The loss function for defect classification is gradient.

4. The method for detecting welding quality of steel structures in construction projects based on deep neural networks according to claim 1 is characterized in that: In step S400, the method for detecting welding defects of steel structures by using a deep neural network model includes: A new multi-task decision-making function module is built based on the deep learning architecture, and it has three branches: defect classification, defect location, and confidence assessment. The three branches share a multi-dimensional fusion feature layer. Make collaborative decisions; calculate the overall multi-task decision loss function through formula (21) : （21）； In formula (21), is the overall multi-task decision loss function, , and are weight coefficients, is the defect classification loss function, Loss function for defect localization; is the confidence loss function; The losses of the three branches of defect classification, defect location and confidence assessment are back-propagated simultaneously, and the parameters of the deep neural network are continuously adjusted to achieve the coordinated optimization of branch parameters and improve the interpretability of the detection results. After completing the steel structure welding defect detection, the defect classification branch outputs a multi-dimensional probability vector, each dimension corresponds to a type of steel structure welding defect; the defect location branch outputs the corresponding defect location marked on the original welding image. The confidence assessment branch outputs the detection confidence assessment result.

5. The method for detecting welding quality of steel structures in construction projects based on deep neural networks according to claim 4 is characterized in that: In step S400, it also includes: In the defect classification branch, an independent fully connected network is constructed, and the defect classification loss function is calculated using formula (25): , used to ensure accurate classification of defect types; （25）； In formula (25), is the defect classification loss function, It is a defect category in the preset steel structure welding defect type set. is the steel defect sensitivity coefficient, is the model’s predicted probability for the positive category, Used to suppress the gradient of easily classified samples; In the defect localization branch, an independent deconvolution network is constructed, and the defect localization loss function is calculated using formula (26): , used to accurately locate the position of welding defects in steel structures; （26）； In formula (26), is the defect localization loss function, Used to measure the degree of overlap of the positioning box boundaries. is the Euclidean distance, is the minimum closed region diagonal length, is the steel structure defect adjustment coefficient; In the confidence evaluation branch, an independent network consisting of a 2-layer long short-term memory network and a 1-layer attention mechanism is constructed, and the confidence loss function is calculated using formula (28): , used to improve the credibility and generalization ability of detection results; （28）； In formula (28), is the confidence loss function, In order to detect the local area characteristics of welding defects, It is the local features of the weld area detected as qualified.

6. The method for detecting welding quality of steel structures in construction projects based on deep neural networks according to claim 4 or 5, characterized in that: In step S400, it also includes: In the defect localization branch, the input features The local weighted processing is performed by formula (27) to reduce the false detection rate of the weld boundary area: （27）； In formula (27), is the weighted local feature, In order to identify the original local features of the defect, is the influence coefficient of weld type, exp is the natural exponential function, It is the distance from the current pixel to the center line of the weld.

7. The method for detecting welding quality of steel structures in construction projects based on deep neural networks according to claim 4 is characterized in that: In step S400, the method for the confidence assessment branch to output the detection confidence assessment result includes: The confidence level of steel structure welding quality inspection output by formula (22) is: （22）； In formula (22), To test the confidence level of steel structure welding quality; Function, used to normalize the output to between 0 and 1; is the predicted probability of the model for the positive category; is the overlap coefficient of the positioning box boundary; is the branch gradient credibility compensation parameter.

8. The method for detecting welding quality of steel structures in construction projects based on deep neural networks according to claim 7 is characterized in that: The branch gradient credibility compensation parameter in formula (22) is It can be calculated using formula (23): （23）； In formula (23), is the branch gradient credibility compensation parameter, is the overall multi-task decision loss function, is the weight coefficient, is a constant, represents the indicator function, when The value is greater than the threshold Returns 1 if yes, otherwise returns 0; is the total loss function for the weight coefficient gradient.

9. The method for detecting welding quality of steel structures in construction projects based on deep neural networks according to claim 4 is characterized in that: In step S500, the method for outputting steel structure welding quality inspection results through a deep neural network model includes: integrating the output results of defect classification, defect location and confidence assessment branches through a deep neural network model to generate a steel structure welding quality inspection report.

Images