A steel structure welding defect intelligent detection method based on deep learning

Abstract

The application discloses a steel structure welding defect intelligent detection method based on deep learning and relates to the technical field of welding defect detection. In order to reserve the information of small defects at the original resolution and reduce the calculation amount of normal areas, a welding area image is acquired and saved at the original resolution, the image is subjected to welding seam positioning and topological normalization to obtain a strip-shaped area, the resolution of the strip-shaped area is reduced to divide the strip-shaped area into a plurality of local areas, a first deep learning network is used to obtain defect existence probability, category probability and feature information, the risk values of the areas are determined in combination with prediction uncertainty and detail residual error, part of the areas is selected under the calculation budget constraint, corresponding high-resolution areas are read from the original resolution image, a second deep learning network is used for fine detection, the areas are inversely mapped back to the original image coordinates and corrected according to the welding seam position, and finally, the defect category and position are output.

Description

Technical Field

[0001] This invention relates to the field of welding defect detection technology, and in particular to an intelligent detection method for welding defects in steel structures based on deep learning. Background Technology

[0002] Steel structures are widely used in the manufacture and installation of bridges, factories, pressure vessels, and large steel components. The quality of welds directly affects the load-bearing capacity and service safety of these components. After welding, defect inspection of the weld body and its adjacent heat-affected zone is necessary. Manual visual inspection and manual X-ray interpretation are affected by the inspector's experience, fatigue level, and ambient lighting conditions, resulting in poor consistency in judgment and difficulty in matching the pace of mass production. With the widespread use of industrial cameras and digital X-ray imaging equipment on production lines, automatic detection methods that use weld surface images or X-ray grayscale images as input and output defect categories and locations through deep learning detection networks are gradually replacing manual interpretation and have become the main technical route for online weld quality inspection.

[0003] Existing deep learning-based weld defect detection solutions mostly follow the approach of single-level full-image detection, which involves scaling the entire acquired image to the network input size and then performing inference in one go. Among common defects in steel structure welds, fine cracks, pinholes, and small-sized undercuts occupy a very small proportion of pixels. The higher the original acquisition resolution, the fewer effective pixels remain after scaling. Coupled with the inherent low grayscale contrast and blurred boundaries of the weld area, these targets are very easy to miss. To improve the detection capability of small targets, common improvement methods include increasing the high-resolution detection scale, constructing more complex multi-scale feature fusion paths, introducing attention modules, or adding parallel input branches. These methods can indeed improve the small defect indicators to some extent, but they all come at the cost of higher inference computation, larger memory usage, and longer single-frame latency, making them difficult to implement on production line equipment with limited computing power.

[0004] Another approach is to maintain the original resolution by cutting the entire image into fixed-size tiles and feeding them into the detection network one by one. This method preserves the original details of minor defects, but at the cost of all tiles being subjected to the same intensity of computation, regardless of whether there are weld anomalies. In actual production, most tiles correspond to normal welds or the background of the base material, resulting in a high proportion of invalid calculations. The processing time and equipment cost per image increase rapidly with the increase in acquisition resolution. Piece-based detection also severs the spatial relationship between tiles. Background textures such as scratches on the base material, component edges, and inkjet characters are similar to real defects in the local field of view, making them prone to false alarms.

[0005] To reduce background interference, some solutions first extract the rectangular region of interest (ROI) containing the weld before performing detection. However, rectangular clipping still includes a significant portion of the base material for welds that are tilted, curved, or subject to perspective distortion. The internal traversal of this region remains fixed and independent of the image content, meaning the allocation of computational resources does not change with the actual state of the weld. While cascading coarse and fine detection can reduce the computational load of subsequent stages, the coarse stage typically uses a fixed threshold to determine whether to continue processing. Small, blurry real defects exhibit low confidence at the coarse level; once eliminated in the coarse stage, subsequent networks have no chance to recover. Furthermore, the number of regions entering subsequent stages fluctuates entirely with the image content, and the single-frame latency lacks an upper bound, hindering production line cycle control.

[0006] Therefore, it is evident that in the inspection of steel structure welds, how to preserve the minute defect details contained in the original resolution to suppress missed detection, while avoiding the application of the same intensity of high-resolution calculations to the large proportion of normal areas, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] The purpose of this invention is to provide a deep learning-based intelligent detection method for welding defects in steel structures, which solves the problem of balancing the need to retain the micro-defect information contained in the original resolution with the need to avoid investing the same intensity of high-resolution computation in the normal area that accounts for a large proportion in existing weld defect detection methods. This method can suppress the missed detection of micro-defects while reducing the invalid computation of normal areas, and make the high-resolution detection load of a single image have a controllable upper limit.

[0008] To achieve the above objectives, the present invention adopts the following technical solution.

[0009] A deep learning-based intelligent detection method for welding defects in steel structures includes: acquiring an image of the welded area of ​​the steel structure and saving the image at its original resolution; locating the weld seam in the image to obtain a weld seam probability map, determining the weld centerline and local weld width from the weld seam probability map, resampling the weld seam along the weld seam direction into a normalized strip-shaped weld detection region, and saving the coordinate mapping relationship between the strip-shaped weld detection region and the image; dividing the strip-shaped weld detection region into multiple overlapping local regions after reducing its resolution, and obtaining at least the defect presence probability, category probability, and feature information of each local region through a first deep learning network; for each local region, calculating the defect presence probability and the prediction uncertainty obtained from the category probability... The risk value of a local region is determined by the detailed residual obtained from the feature information. Based on the risk value of each local region and a preset calculation budget, some local regions are selected from the plurality of local regions, and the number of selected local regions is not less than the minimum protection budget and not more than the maximum budget. The high-resolution image region corresponding to the selected local region is obtained from the image saved at the original resolution. Welding defect detection is performed on the high-resolution image region through a second deep learning network to obtain the defect detection result. The defect detection result is restored to the coordinates of the image according to the coordinate mapping relationship, and the restored defect detection result is corrected for consistency according to the positional relationship of the defect relative to the weld, and the category and location of the welding defect are output. Thus, the risk assessment of normal weld areas is completed at a low cost in the low-resolution stage, and only high-risk areas are read back to the original resolution and sent to high-cost detection, thereby reducing repeated high-resolution calculations on normal areas while preserving the details of small defects.

[0010] As a preferred embodiment of the present invention, the prediction uncertainty is the normalized entropy calculated from the category probability. When there are multiple configured defect categories, the normalized entropy increases as the distribution of the category probability tends to be uniform. When there is only one configured defect category, the prediction uncertainty is obtained from the binary entropy of the defect existence probability. Thus, the uncertainty of whether there is a defect in the local area can be measured under different category configurations.

[0011] As a preferred technical solution of the present invention, the detailed residual is obtained by the difference between the feature information and the result obtained by downsampling and then upsampling the feature information, in order to characterize the local information in the local area that is difficult to be explained by the low-frequency structure, so that areas with obvious local abnormal textures but low defect probability can still obtain a high risk assessment.

[0012] As a preferred technical solution of the present invention, determining the risk value of the local region includes: weighting the probability of the defect, the prediction uncertainty, and the normalized detail residual according to their respective weights, and then obtaining the risk value through monotonically saturated mapping. The respective weights are obtained through training, so that local regions with a high probability of the defect, local regions with a moderate probability of the defect and high prediction uncertainty, and local regions with a low probability of the defect and significant detail residual all obtain higher risk values, thus avoiding premature exclusion of minor defects with low confidence based solely on the probability of the defect.

[0013] As a preferred embodiment of the present invention, the step of selecting a portion of local regions from the plurality of local regions includes: determining a dynamic threshold by the median and absolute median difference of the risk values ​​of each of the local regions in the current image, selecting local regions whose risk values ​​reach the dynamic threshold; when the number of local regions whose risk values ​​reach the dynamic threshold is greater than the maximum budget, retaining the number of local regions equal to the maximum budget in descending order of risk value, so that the selected regions are adaptively determined according to the risk distribution of each image itself.

[0014] As a preferred technical solution of the present invention, the minimum protection budget is not less than 1. When the risk value of each local area does not reach the dynamic threshold, local areas with a number not less than the minimum protection budget are still selected according to the risk value from high to low and sent to the second deep learning network for welding defect detection, so as to avoid missing the tiny defects that happen to exist due to zero selection when the risk of the whole image is generally low.

[0015] As a preferred technical solution of the present invention, the maximum budget is dynamically adjusted according to the load of the processor executing the method or the queuing status of the images to be processed. The maximum budget is increased when the focus is on reducing missed detections, and decreased when the focus is on reducing processing latency, so that the same model can adapt to different computing power conditions and different detection strategies without changing the network structure.

[0016] As a preferred technical solution of the present invention, the first deep learning network, the determination of the risk value, and the second deep learning network are jointly trained. During training, the selection is represented as a differentiable soft gating and constrained by budget loss and routing supervision loss. The budget loss makes the average proportion of the selected local regions approach the preset target proportion, and the routing supervision loss makes the local regions that intersect with the real defect labels preferentially selected. Thus, the control of computational load is included as a component of the training objective without the need for separate pruning after training.

[0017] As a preferred embodiment of the present invention, the consistency correction includes: determining the degree of deviation of the center of the defect detection result from the weld centerline, normalized by the local weld width, and thereby reducing the score of the defect detection result that deviates from the weld; for defect categories whose location is expected to be near the weld toe, adopting geometric constraints related to the defect category so that the defect detection result of the defect category is not reduced in score due to being near the weld toe, thereby suppressing false defects far from the weld while avoiding the erroneous suppression of real weld toe defects.

[0018] As a preferred technical solution of the present invention, when multiple images are acquired continuously, the high-resolution image regions corresponding to the selected local regions of each of the multiple images are merged into a batch, and a forward computation is performed on the second deep learning network. Each high-resolution image region carries the identifier of its respective image and the mapping parameters required for coordinate recovery. The welding defect detection results are distributed to the corresponding images according to the identifier, thereby reducing the idle time of the computing unit caused by small-batch inference while maintaining the correspondence between each image and its detection conclusion.

[0019] Compared with the prior art, the present invention has the following beneficial effects: This invention establishes curved coordinates along the weld direction based on the weld positioning results and resamples the weld into a standard strip area. Welds that are tilted, bent, or have perspective distortion are flattened into images with consistent direction, center alignment, and uniform relative width. The texture of the far end of the base material, tooling fixtures, and inkjet characters entering the network field of view are reduced accordingly. At the same time, it provides a unified measurement benchmark for subsequent judgment of the deviation of candidate defects from the weld center, which cannot be provided by rectangular region of interest clipping.

[0020] Regional risk is determined by the predicted entropy of defect probability, class probability, and detailed residuals of intermediate features, rather than solely by whether the defect probability exceeds a fixed threshold. Regions with low defect probability but high predicted entropy, as well as regions with low probability but significant local detailed residuals, can also enter the second-level fine detection stage. This mitigates the problem in cascaded structures where low-confidence micro-defects are prematurely eliminated in the first level and cannot be recovered in subsequent levels. Selected regions are reread from the cached original image pixels, rather than being enlarged from a scaled-down image, allowing micro-defects to be identified at native resolution.

[0021] Whether to invoke fine detection is represented as a routing decision constrained by computational budget. The number of regions in a single image entering fine detection is limited between the minimum protection budget and the maximum budget. The minimum protection budget ensures that even when the overall risk of the image is generally low, some of the highest-risk regions are still re-inspected. The maximum budget provides an upper bound on the computational cost and latency per frame. In cases where normal welds constitute the majority, the actual number of regions entering fine detection is significantly lower than the total number of logical regions. The total computational cost is reduced compared to a fixed scheme that performs fine detection on all regions. This reduction comes from shortening the actual execution path and does not rely on compressing network weights or reducing convolutional channels; therefore, it does not sacrifice the expressive power of the fine detection network itself. The routing ratio is constrained during the training phase by calculating both the budget loss and the routing supervision loss, eliminating the need for separate pruning after training.

[0022] Because the budget parameters and network structure are independent, the same set of model weights can be adapted to different computing power conditions such as edge computing platforms, workstations, and servers by adjusting the maximum budget. They can also be dynamically adjusted during runtime based on processor load or queue length; increasing the budget when prioritizing missed detection control and decreasing the budget when prioritizing tick count. None of these adjustments require retraining. Routed regions from multiple images can also be merged into dynamic batches for unified fine detection, reducing idle computing units caused by small-batch inference.

[0023] After the fine detection results are restored to the original image coordinates through inverse mapping, topological consistency is checked based on the ratio of the normal distance from the candidate center to the weld centerline to the local weld width. For categories such as undercut, which should appear near the weld toe, a category-related geometric constraint coefficient is applied. This reduces the score of false defects such as scratches on the base material, component edges, and characters that are far from the weld, while avoiding the erroneous suppression of real weld toe defects, which helps to reduce false alarms in complex backgrounds. Attached Figure Description

[0024] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the joint training process of the detection model in this embodiment of the invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are for illustrative purposes only and are not intended to limit the present invention; the technical features of the various embodiments can be combined with each other without conflict.

[0026] This embodiment provides a deep learning-based intelligent detection method for welding defects in steel structures. This method is implemented in software, with program instructions stored in a computer-readable storage medium, and read and executed by a processor in an edge computing device, industrial control computer, or server. See also... Figure 1 The method sequentially proceeds through image reception, image quality evaluation, weld location, weld topology normalization, coarse risk detection, regional risk calculation, dynamic routing, fine detection of the original image, coordinate restoration, result fusion, and result output, respectively corresponding to... Figure 1 Steps S1 to S11, with steps S5 to S7 forming the most critical computational resource scheduling closed loop in the entire method. The input to be detected is an image of the welded area of ​​the steel structure, which can be either a weld surface image acquired by a visible light industrial camera or a grayscale image of the weld obtained by digital X-ray imaging; both share the same processing flow. The defect category set is designed as a configurable parameter, including at least cracks, porosity, undercut, weld beads, burn-through, and spatter anomalies in the visible light scene, and can be expanded to include internal defect categories such as incomplete fusion, incomplete penetration, and slag inclusions in the digital X-ray scene.

[0027] The input image is denoted as Visible light color image capture ray grayscale image The initial acquisition resolution is recommended to be no less than 1920×1080 pixels. For larger components or when higher requirements are needed for detecting minute defects, higher resolutions such as 2448×2048 pixels or 4096×3000 pixels can be used. These values ​​are engineering recommendations and do not constitute a limitation on the method itself. Step S1 receives the image and verifies its format and number of channels. After verification, it is written to the buffer at the original resolution for later reading during the fine inspection stage. If the format is incorrect or the data is empty, it is rejected directly and an acquisition anomaly message is given. Step S2 calculates the overexposure ratio of the image. Underexposure ratio With clarity index Taking 8-bit quantization as an example, Select the percentage of pixels with a grayscale value of at least 250. The percentage of pixels with a grayscale value no higher than 5 is selected, and the sharpness is calculated using the variance of the Laplacian response.

[0028] when or More than 5%, or When the image quality falls below the empirical threshold calibrated based on on-site data, it is deemed insufficient. Instead of a "no defects" conclusion, a quality anomaly warning is returned, requiring re-acquisition, or a conservative full-image inspection mode is entered. This mode directly performs defect detection on the entire downsampled image and marks the result as requiring manual review. This branch design avoids situations where low model response due to acquisition issues such as defocusing or overexposure leads to misinterpretation as normal workpiece performance.

[0029] Images that meet quality standards are downsampled to coarse resolution images. , The longer side is set to 640-1280 pixels. Step S3 involves using the weld seam positioning network... To perform inference, the network employs a lightweight semantic segmentation structure, such as an encoder consisting of stacked depthwise separable convolutions and a decoder consisting of a network of progressively upsampled layers, outputting a weld probability map. and the reliability of weld position determination. For Binarization is performed with a threshold of 0.5. After morphological closing, the largest connected region is retained as the weld mask. If the mask area is lower than a preset lower limit, the centerline length is significantly abnormal, or the location confidence is lower than the threshold, the weld mask will be rejected. , If a value of 0.5 to 0.7 is used, a backtracking path is executed, expanding the bounding rectangle of the weld probability map outward by 30% to 50% to form an enlarged region of interest for further processing, or the detection is directly performed on the low-resolution full image, and the weld positioning is marked as unreliable in the final result. Figure 1 As shown, image quality evaluation and weld seam location reliability verification constitute two protective branches before the main process. When the weld seam has branches or breaks, centerlines are established for each connected segment and normalization is performed segment by segment. The detection results of each segment are finally merged in the original image coordinate system.

[0030] Step S4 involves obtaining the centerline on the weld mask. This can be done by skeletonizing the mask and removing burrs and branches, or by calculating the centroid at each position along the main weld direction using probability weighting, followed by cubic spline smoothing to obtain the centerline. , These are the centerline parameters sampled at equal intervals based on arc length. The unit normal vector is obtained by rotating the centerline tangentially by 90°. ,along The distances from the centerline to the mask boundary are measured in both the positive and negative directions and averaged to obtain the local half-width. Regarding centerline parameters With normalized horizontal position Define the original image sampling coordinates

[0031] In the formula The weld neighborhood expansion factor is set to 1.5–2.5 to ensure that the standard band covers the weld body and the adjacent heat-affected zone. The original image is sampled using bilinear interpolation to obtain the standard band image.

[0032] Mapping table is saved synchronously during sampling.

[0033] This is used to establish the correspondence between the strip coordinates and the original image coordinates. (Mapping table here) That is, by sampling The given analytical mapping applies to any continuous line within the range of the centerline parameter values. All coordinates can be directly substituted into the original image for calculation. Since they are not stored in a discrete lookup table with a fixed grid, there is no inherent domain boundary; the lateral position is taken as... The result obtained is only the original map points further away from the center line; if an implementation prioritizes efficiency, it will... If a table is pre-made, the coordinates that do not fall on the grid points can be obtained by bilinear interpolation. The sampling interval in the direction is comparable to the pixel scale at the center line of the original image. The number of sampling points in each direction can be taken as 256 to 512, making Within the weld neighborhood, the spatial resolution of the original image is largely maintained. After this transformation, welds that are tilted, curved, or have perspective distortion are flattened into strip-shaped areas with consistent direction, center alignment, and uniform relative width. Interference content such as the texture of the far end of the base material, tooling fixtures, and inkjet characters are largely excluded from the network view. The degree to which any candidate position deviates from the weld center can also be directly measured in a unified coordinate system, a property that simple rectangular area clipping does not possess.

[0034] Standard strip image Scaling Overall downsampling yields coarse-scale images Its longer side also ranges from 640 to 1280 pixels. Logical area , Always defined in the standard zonal coordinate system at the original scale Below this, the width along the weld direction should be 256–384 pixels, with an overlap rate of 20%–40%, ideally around 25%, covering the entire bandwidth laterally; each logical region in the coarse-scale image The corresponding window on the top is multiplied by its strip coordinates. The output of the coarse detection is then aggregated according to this correspondence. Mapping table The domain of definition is the original scale zonal coordinate system, therefore, all coordinates that need to be... All region coordinates converted to the original image coordinates are based on the original scale zonal coordinates; however, in some implementations, for engineering convenience, they may be directly converted to the original image coordinates. To divide the window, you must first divide the window coordinates by... Convert back to the original scale zonal coordinates and then look up the table to avoid index out-of-bounds errors or positional offsets. The logical region is only used for organizing risk assessment and does not mean that every region will call fine detection. Step S5 is performed by the coarse detection network, which serves as the first-level deep learning network and adopts a lightweight detection structure. The backbone can be a depthwise separable convolutional network, a lightweight visual Transformer, or other equivalent structures. This method does not limit the specific network model. A preferred implementation is to apply the entire image... After performing one full convolutional inference, the output is aggregated by logical region; alternatively, region-by-region inference is also possible. Coarse detection is performed on each... Probability of output defects Category probability vector and intermediate features , This represents the number of defect categories currently configured. The coarse detection network in this method is responsible for low-cost computational resource scheduling; its output should not be directly used as the final basis for determining whether a system is defect-free.

[0035] If only Determining a region's inclusion or exclusion based on whether it exceeds a fixed threshold presents a potential problem: extremely subtle or blurry real-world defects often manifest as low confidence levels. If a region is prematurely excluded at the first stage, subsequent network stages have no chance to recover it. Therefore, step S6 constructs a risk score for each logical region, determined by three types of information: probability, prediction uncertainty, and detailed residuals. Prediction uncertainty is calculated using normalized entropy.

[0036] In the formula To prevent dimensionless small quantities from logarithmic divergence, the order of magnitude is taken as... , , The number of defect categories currently configured. For this The normalized distribution over each defect category, with the background or defect-free scenario determined by the probability of defect presence. Carry alone and not included .when The normalized entropy of the category distribution is calculated using the above formula; when only one defect category is configured... At that time, the category distribution degenerates. The above formula no longer applies. In this case, the uncertainty is measured by the binary entropy of the probability of defect existence, i.e.

[0037] Its in It tends to 0 when it is close to 0 or 1. The value reaches its maximum when it is close to 0.5, which is semantically consistent with the entropy in the multi-class case, meaning that the more uncertain the network is about whether there is a defect in that region, the more entropy is affected. The larger it is, the more it avoids... Divide by zero at that time. Detail residuals reflect high-frequency local information within a region that is difficult to explain by the low-frequency structure. After performing average pooling, the sample is upsampled back to the original size using bilinear upsampling, and the average absolute value of the difference between the two samples is taken.

[0038] for The total number of elements, the pooling kernel can be 4×4 or 8×8. Each region Linearly normalized within the current image to Alternatively, it can be truncated after dividing by the reference value obtained from the training set statistics. Regional risk is calculated using the following formula:

[0039] in It is a sigmoid function. Before calculation Cut off to ; , , and The parameters obtained during joint training of the entire network can be initialized to take the following values: , , Under this risk profile, regions with significantly high defect probabilities, regions with moderate probabilities and high prediction entropy, regions with low probabilities but significant local detail residuals, and regions ranking high in the current image risk ranking all have the opportunity to enter the second-level detection stage. Small defects with low confidence will not be simply discarded at the coarse level.

[0040] Step S7 performs dynamic routing based on the risk sequence. The dynamic threshold is set...

[0041] The absolute median

[0042] A value of 1.0 to 1.5 is acceptable. and To limit the threshold drift range, values ​​such as 0.3 and 0.85 are used, with the specific values ​​determined on the validation set. (This satisfies...) Area routing tags Set to 1 if the total number of selected regions is 1, otherwise set to 0. Based on this, apply a two-way budget constraint: the total number of selected regions must satisfy... If it exceeds the limit, press Only the top ones are retained from high to low. One; simultaneously satisfying Even when no region exceeds the threshold, the region with the highest risk is still forcibly selected. Each area is re-inspected to cover situations where the overall risk of the map is generally low, but there happens to be one extremely small defect. Take 1 to 2. Take 10% to 30% of the total number of logical regions in the weld. Divide a specific weld into 20 logical regions. , For example, when there are 3 regions that exceed the dynamic threshold, these 3 regions are precisely detected. When there are 10 regions that exceed the threshold, only the 6 regions with the highest risk are selected. When no region exceeds the threshold, the 2 regions with the highest risk are still selected. The precision detection load of a single image is limited to 2 to 6 regions, while the fixed sliding window scheme requires high-resolution detection of all 20 regions.

[0043] The routed region is not enlarged on the coarse-scale image; instead, the original pixels are read back. Step S8 first follows the logical region. The four boundaries are densely sampled and substituted point by point. This yields the corresponding curved region in the original image; due to the center line Typically, this is a curve, and since the curved area is not a regular rectangle, its axis-aligned circumscribed rectangle in the original image's pixel coordinates is used. This serves as a benchmark for the cutoff range. To avoid truncating defects that happen to be located at the region boundary and to preserve the necessary context for fine inspection, [the following is used as a reference]. The image is obtained by expanding outwards along both the width and height directions by 10% to 25% of its own size within the original image's pixel space. The margin expansion ratio should ideally be around 15%. After expansion, any portion extending beyond the image boundaries should be cropped back to within the image limits. This expansion is performed directly at the original image's pixel coordinates and does not involve any horizontal band crossing over. The value of is used to crop high-resolution patches from the original image.

[0044] After scaling while maintaining aspect ratio and adding necessary edge padding, the input size is standardized to the fine detection network, which can be 640×640 pixels or 960×960 pixels. As a second-level deep learning network, the fine detection network structurally requires at least one high-resolution feature output layer with a stride of 4 or 8 to accommodate small targets. For slender defects such as cracks, an instance segmentation branch can be added to describe the defect morphology using a mask, avoiding the inclusion of too much irrelevant background within a single horizontal rectangle. The backbone and detection head of the fine detection network are not limited to specific models. The network outputs a candidate set for each patch.

[0045] This refers to the bounding box in the local coordinate system of the map. As a defect category, For confidence level, This is an optional defect mask.

[0046] Step S9 restores the candidate results to the coordinates of the original steel component image. It is an axis-aligned tile directly cropped from the original image. The bounding box can be linearly restored according to the tile's scaling ratio and offset in the original image to obtain the original image coordinates. The same process is applied to the mask, and any images that extend beyond the image boundaries are discarded. Then, the center line is used... Perform a weld topology consistency check on each candidate: take the parameter of the closest point from the candidate center to the centerline. Let the normal distance at that point be . Define normalized deviation

[0047] The final score of the candidate is

[0048] In the formula For a zero-prevention quantity measured in pixels, 0.5 pixels can be used, similar to the dimensionless quantity mentioned above used to prevent logarithmic divergence. They are different in physical meaning and dimension, and cannot be substituted for one another. The risk value of the routed area to which the candidate belongs. To accurately detect the mixture coefficient between confidence level and routing risk, a value of 0.6 to 0.8 can be used. These are the geometric constraint coefficients associated with the defect category. For categories such as porosity and burn-through, which typically occur in the middle of the weld, A value of 1.0 to 2.0 is acceptable; for categories such as undercut, which should appear near the weld toe, Take values ​​between 0 and 0.3, or reduce the exponent term to... Replace with , To determine the expected normalization location for this category, undercut is assigned a value of 1, and center-type defects are assigned a value of 0, to prevent genuine weld toe defects from being incorrectly suppressed. This verification significantly reduced the scores of false defects located far from the weld, such as base material scratches, component edges, and inkjet characters, thus suppressing false alarms in complex backgrounds.

[0049] Duplicate candidates resulting from overlapping adjacent tiles are merged in step S10. The merging is performed in the original image coordinates. For two candidates of the same category, if their restored original image coordinate bounding boxes... The crossover ratio is not lower than the merging threshold. , A threshold of 0.5 can be used, or if the two masks intersect and the arc length along the centerline is less than 0.5 times the width of the local weld, they are considered the same defect, and weighted frame fusion or softening non-maximum suppression is used to generate the merged result. Category determination threshold. Instead of using a uniform value across all categories, categorized calibration is performed on the validation set. Safety-critical cracks can be calibrated using a lower threshold of 0.25–0.35 to prioritize recall, while sputtering-related appearance anomalies can be calibrated using a higher threshold of 0.5–0.6 to suppress false alarms. Temperature scaling calibration is applied to the output confidence level when necessary. The candidate constitutes a valid defect set, and a structured result is formed by defect category, original image location, size and confidence level, which is output by step S11. When the set is empty, no defect is detected in the output. When the process has entered the quality abnormality or unreliable weld positioning branch, the corresponding mark is added to the output for the quality inspection system to decide whether to transfer to manual review.

[0050] See Figure 2 The detection model was trained using an end-to-end joint training method. Training data consisted of labeled images of welded areas in steel structures, with annotations including weld area masks and defect bounding boxes. For crack types, additional instance masks could be added. For the engineering model, 10,000–30,000 independent weld area images were recommended, while the proof-of-concept phase could start with 3,000–5,000 images. The dataset was divided into a 70% training set, 15% validation set, and 15% test set based on workpiece or weld number. Adjacent frames of the same weld should not appear across sets to prevent data leakage and inflated metrics. For categories with insufficient sample size, geometrically constrained defect instance replication was performed within the weld band, with the pasting direction consistent with the local weld direction. Poisson blending or edge feathering was used to eliminate splicing marks, supplemented by local contrast adjustment, random blur, exposure perturbation of approximately ±20%, and slight perspective changes.

[0051] In joint training, routing decisions participate in backpropagation in a differentiable form. Continuous soft gating is defined.

[0052] For the temperature parameter, during training, the gating is gradually annealed from 1.0 to 0.1, allowing the gating to smoothly transition to approximate binary selection. (Training phase...) The calculation method is exactly the same as that used in the inference phase, that is, after statistically analyzing the median and absolute median difference of the risk distribution for each image, the same method is applied. and Cut off, and to Set a lower limit of no less than 0.02; this processing is for batches that are all normal welds. At this time, the risk values ​​are highly clustered and the absolute median difference is close to 0. If no lower limit or truncation is added, the threshold will degenerate into the median itself, causing the gate value of nearly half of the area to stay around 0.5. This not only conflicts with the target routing ratio, but also causes budget loss to push down the risk of a large number of normal areas. The results are derived from the order statistics of the risk sequence, and are treated as constants during backpropagation without backpropagating gradients. right The upper bound of the gradient is When the temperature is annealed to 0.1, the value should not exceed 2.5. Combined with gradient norm pruning, convergence stability can be guaranteed. Let the target average route ratio be... Budget loss is defined as

[0053] A value of 0.20 is acceptable, and it can be adjusted within the range of 0.10 to 0.40 according to computing power conditions. Routing label. In the region Set to 1 if it intersects with any real defect annotation box or annotation mask, otherwise set to 0. Route supervision uses a focus-based binary cross-entropy. Let... (when )or (when ), and record the class balance factor. (when )or (when ),but

[0054] in As the focus factor, it can be 2. This is used to balance the ratio of positive to negative areas, given that the number of normal areas is usually much greater than the number of defective areas. A value of 0.75 is acceptable. (Focus Item) This reduces the contribution of the large number of correctly identified normal regions to the loss, thus focusing the training effort on a few defective or difficult-to-discriminate regions. The overall objective function is...

[0055] in For weld segmentation loss, exemplarily, the sum of Dice loss and binary cross-entropy is used to predict the weld probability map. Its truth mask ,

[0056] In the formula To perform element-wise multiplication and summation, iterate through all pixels. This represents the total number of pixels. For coarse detection, there is a region-level classification loss and an optional coarse localization loss. For the classification, bounding box regression, and optional mask loss of the fine detection, the bounding box regression can adopt the cross-union-classification loss; the specific form of these two depends on the selected detector, and they are both mature detection losses in the field. Those skilled in the art can directly construct them according to the selected detection head, so they are not fixed to a specific expression here, so as not to bind the scope of protection to a specific detector. This is a cross-scale consistency loss. This loss requires reducing the dimensionality of the instance-level precise detection output to a region-level scalar. Specifically, this involves: The bounding boxes of each candidate are inversely mapped and scale-aligned back to their original scale zonal coordinates. Any bounding box whose center falls within... The candidate set within Based on this, evidence of the existence of regional-level defects is calculated.

[0057] When it is an empty set, take This formula degenerates into the confidence level of a candidate when there is only one candidate in the set. When there are multiple candidates, it monotonically approaches 1 with the accumulation of evidence. Compared with the method of directly taking the maximum value, it can provide a non-zero gradient for each candidate in the set. In practice, it can also be used directly. The set of routes For statistical purposes, the consistency loss is taken as...

[0058] in This indicates stopping the gradient, i.e. Treating it only as a soft target and not accepting feedback from it avoids the fine detection network reducing its own discriminative power to cater to the coarse detection output; this constraint prompts the coarse detection network to gradually absorb the discriminative information obtained by the fine detection network at the original resolution, thus making the risk estimation more accurate as training progresses. Initial values ​​for the weight coefficients can be chosen... , , , Fine-tuning is performed on the validation set. Early in training, the routing parameters are unreliable. To ensure the fine detection branch can access real defect samples, positive sample regions intersecting with real defects are forcibly incorporated into the selected set for forward and backward propagation in fine detection. The batch size can be 16, and the AdamW optimizer is used with an initial learning rate of [value missing]. , can ~ Adjustments can be made within a certain range, with 200-400 training epochs, adjusted based on convergence. After training, dynamic threshold upper and lower limits, budget parameters, and class decision thresholds are calibrated on the validation set, and these, along with the network weights, are then fixed into a deployment package. The following initial configuration can be used for engineering implementation: coarse image long side 960 pixels, fine detection input 640×640 pixels, logical region width 256 pixels, overlap rate 25%, target route ratio 0.20. Choose 1 or 2, perform fine detection and edge expansion by 15%, and optimize the remaining parameters on the validation set according to the aforementioned range.

[0059] Since the decision to proceed to precision testing is jointly made based on risk and budget, the same model can be adapted to different computing platforms without modifying the network structure. The R&D training platform can use workstations with 32–64 GB of system memory and 12–24 GB of GPU memory; production line deployment can use edge computing platforms with 8–16 GB of available GPU memory or unified memory, in which case the cost will be reduced accordingly. or The server side can use a configuration with at least 24 GB of video memory, and the budget can be appropriately increased. During online operation, the configuration can also be dynamically scaled down based on the current processor load or the length of the pending queue. In a quality control strategy prioritizing defect prevention, the budget can be increased; in a strategy prioritizing cycle time delay, the budget can be decreased. Neither of these adjustments requires retraining. When the production line continuously receives... When there are multiple weld seam images and only a small portion of each image is routed, all routed tiles can be added to the queue.

[0060] The queue has reached its batch capacity limit or the waiting time has reached the limit. Each time, the precision detection network performs one forward calculation. The latency can be 20–50 ms. Since the tiles in the same batch come from different images and the execution order may not be the same as the queuing order, each tile must be attached with a tag during queuing, including the frame tag, region tag, and inverse mapping parameters of the region. The fine detection output is backfilled into the result buffer of the corresponding image according to the frame tag. Only after all the routed tiles of a frame have been backfilled is the result fusion, topology verification, and threshold determination triggered for that frame, and the detection conclusion is issued according to the frame tag, thus ensuring that the batch processing does not disrupt the correspondence between frames and conclusions. At the same time, a separate deadline is set for each frame. When the waiting time of a frame since entering the queue approaches the production line cycle limit, a forward calculation is immediately triggered even if the batch capacity is not full, so that the single frame latency always has an upper bound and is aligned with the rejection and alarm actions on the industrial control computer and programmable logic controller side in terms of timing. Cross-image dynamic batch processing and region-level routing work together to reduce the amount of computation and avoid GPU idleness caused by small batch inference.

[0061] The cost of weld positioning is... The cost of rough testing is The cost of precise detection of a single high-resolution region is The average computational cost of this method is

[0062] For all The fixed procedure for performing precision testing in each region is as follows:

[0063] exist Constituting the main computational overhead and Under typical operating conditions, the computational savings are considerable, and these savings come from shortening the actual execution path, rather than compressing weights or reducing convolution channels. In conjunction with this, fine detection always reads back pixels from the original image rather than magnifying a coarse image, preserving the original details of minute defects such as fine cracks and pinholes. The ability to detect minute defects no longer sacrifices average computational cost.

[0064] In another embodiment, the input is a digital radiographic weld grayscale image. Only the number of input channels in the first layer of each network needs to be adjusted; the rest of the process, risk composition, and routing mechanism remain unchanged. The defect category set is correspondingly expanded to include incomplete fusion, incomplete penetration, and slag inclusion. The coarse and fine inspection networks can also be replaced with other types of deep learning detectors. The processing relationships between weld topology normalization, risk assessment combining probability, uncertainty, and detailed residuals, dynamic routing under budget constraints, original resolution readback, and inverse mapping topology verification remain unchanged due to the backbone replacement.

[0065] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A deep learning-based intelligent detection method for welding defects in steel structures, characterized in that, include: Acquire an image of the welded area of ​​the steel structure and save the image at its original resolution; Weld seam location is performed on the image to obtain a weld seam probability map. The weld seam centerline and local weld seam width are determined from the weld seam probability map. The weld seam is resampled along the weld seam direction into a normalized strip weld detection area, and the coordinate mapping relationship between the strip weld detection area and the image is saved. After reducing the resolution of the strip welding detection area, it is divided into multiple overlapping local regions. At least the probability of defect presence, category probability and feature information of each local region are obtained through a first deep learning network. For each local region, the risk value of the local region is determined based on the probability of the defect presence, the prediction uncertainty obtained from the category probability, and the detail residual obtained from the feature information. Based on the risk value of each local area and in conjunction with a preset calculation budget, some local areas are selected from the plurality of local areas, and the number of selected local areas is not less than the minimum protection budget and not more than the maximum budget. From the image saved at its original resolution, obtain the high-resolution image region corresponding to the selected local region; The high-resolution image region is used to detect welding defects using a second deep learning network to obtain defect detection results. The defect detection results are restored to the coordinates of the image based on the coordinate mapping relationship, and the restored defect detection results are corrected for consistency based on the positional relationship of the defect relative to the weld, and the category and location of the welding defect are output.

2. The method according to claim 1, characterized in that, The prediction uncertainty is the normalized entropy calculated from the category probability. When there are multiple defect categories configured, the normalized entropy increases as the distribution of the category probabilities tends to be uniform. When there is only one defect category configured, the prediction uncertainty is obtained from the binary entropy of the defect existence probability.

3. The method according to claim 1, characterized in that, The detailed residual is obtained by the difference between the feature information and the result obtained by downsampling and then upsampling the feature information, and is used to characterize local information in the local region that is difficult to be explained by the low-frequency structure.

4. The method according to claim 1, characterized in that, Determining the risk value of the local region includes: weighting the probability of the defect, the prediction uncertainty, and the normalized detail residual according to their respective weights, and then obtaining the risk value through monotonically saturated mapping. The respective weights are obtained through training, so that local regions with a high probability of the defect, local regions with a moderate probability of the defect and high prediction uncertainty, and local regions with a low probability of the defect and significant detail residual all obtain higher risk values.

5. The method according to claim 1, characterized in that, The step of selecting a subset of local regions from the plurality of local regions includes: determining a dynamic threshold by the median and absolute median difference of the risk values ​​of each local region in the current image, and selecting local regions whose risk values ​​reach the dynamic threshold; when the number of local regions whose risk values ​​reach the dynamic threshold is greater than the maximum budget, retaining the number of local regions in descending order of risk value as the maximum budget.

6. The method according to claim 5, characterized in that, The minimum protection budget is not less than 1. When the risk value of each local area does not reach the dynamic threshold, local areas with a number not less than the minimum protection budget are still selected according to the risk value from high to low and sent to the second deep learning network for welding defect detection.

7. The method according to claim 5, characterized in that, The maximum budget is dynamically adjusted based on the load of the processor executing the method or the queuing status of the images to be processed. The maximum budget is increased when the focus is on reducing missed detections, and decreased when the focus is on reducing processing latency.

8. The method according to claim 1, characterized in that, The first deep learning network, the determination of the risk value, and the second deep learning network are jointly trained. During training, the selection is represented as a differentiable soft gating and constrained by budget loss and routing supervision loss. The budget loss makes the average proportion of the selected local regions approach the preset target proportion, and the routing supervision loss makes the local regions that intersect with the real defect labels preferentially selected.

9. The method according to claim 1, characterized in that, The consistency correction includes: determining the degree of deviation of the center of the defect detection result from the weld centerline, normalized by the local weld width, and thereby reducing the score of the defect detection result that deviates from the weld; for defect categories whose location is expected to be near the weld toe, applying geometric constraints related to the defect category so that the defect detection result of the defect category is not reduced in score due to being near the weld toe.

10. The method according to claim 1, characterized in that, When multiple images are acquired consecutively, the high-resolution image regions corresponding to the selected local regions of each of the multiple images are merged into a batch, and a forward computation is performed on the second deep learning network. Each high-resolution image region carries the identifier of its respective image and the mapping parameters required for coordinate recovery, and the welding defect detection results are distributed to the corresponding images according to the identifier.

Images