Rebar mesh welding defect identification method based on graph neural network

Abstract

The application discloses a steel bar mesh welding defect identification method based on a graph neural network, generates a standardized reference image, forms a welding area candidate set, extracts a steel bar mesh structure graph, marks a suspicious welding point position, forms a steel bar mesh structure graph, executes a topology keeping graph embedding operation on the steel bar mesh structure graph, constructs a low-dimensional embedding space, outputs a node embedding representation and a connected confidence graph, divides the reference image and the steel bar mesh structure graph into a plurality of image blocks and graph structure blocks, outputs a fusion attention feature graph including local details and cross-cell continuity features, judges whether the connectivity of each candidate welding point is abnormal, locates a welding breakpoint, outputs a defect position and a breakpoint type, and performs consistency verification on a judgment result. The application significantly improves the structural consistency and anti-artifact capability of welding breakpoint detection in a complex scene.

Description

Technical Field

[0001] This invention relates to the field of steel mesh technology, and in particular to a method for identifying welding defects in steel mesh based on graph neural networks. Background Technology

[0002] With the advancement of intelligent construction in civil engineering and automated production of steel mesh, the demand for non-destructive testing of steel mesh welding quality is becoming increasingly prominent. Traditional methods for detecting weld break points in steel mesh mainly rely on manual inspection or machine vision algorithms based on pixel segmentation, edge detection, and key point extraction.

[0003] In real-world construction environments with complex backgrounds, strong reflections, and partial obstructions, traditional methods are easily affected by visual artifacts such as rust spots, site reflections, and mortar coverage, making it difficult to reliably distinguish between real weld breaks and visual false breaks, resulting in high false detection and false detection rates.

[0004] Existing deep learning methods based on global self-attention consume enormous computational resources when dealing with large-format, high-resolution images, severely limiting their real-time performance and deployability. Current pixel-domain methods struggle to effectively represent the "intersection-member-loop" topology of steel mesh, failing to accurately determine the continuity of complex structures and resulting in insufficient stability in locating and classifying welding defects. Summary of the Invention

[0005] One objective of this invention is to propose a method for identifying welding defects in steel mesh based on graph neural networks. This invention significantly improves the structural consistency and anti-spoofing ability of weld break point detection in complex scenarios.

[0006] A method for identifying welding defects in reinforcing mesh based on a graph neural network according to an embodiment of the present invention includes:

[0007] Collect large-format image data of the steel mesh and preprocess it to generate standardized reference images;

[0008] The reference image is coarsely segmented to form a candidate set of welding areas;

[0009] Extract the steel mesh structure diagram, model the steel bar intersections as graph nodes and the steel bar strips as graph edges, and spatially associate the candidate set of welding areas with nodes and edges, mark the positions of suspicious welding points, and form the steel mesh structure diagram.

[0010] Perform a topology-preserving graph embedding operation on the steel mesh structure diagram to construct a low-dimensional embedding space and output the node embedding representation and connectivity confidence graph.

[0011] The reference image and the steel mesh structure diagram are divided into several image blocks and diagram structure blocks.

[0012] A block sparse attention mechanism is applied to image patches and their corresponding graph structure blocks for feature inference, and the output is a fused attention feature map that includes local details and cross-cell continuity features.

[0013] Based on the fusion of attention feature map, steel mesh structure map and welding area candidate set, and combined with the spatial topology and connectivity features of nodes, edges and candidate welding points in each image block and graph structure block, the system determines whether the connectivity of each candidate welding point is abnormal, locates the welding breakpoint and outputs the defect location and breakpoint type, and verifies the consistency of the judgment results.

[0014] Optionally, generating the standardized reference image includes:

[0015] An industrial vision acquisition system is used to acquire images of the steel mesh to be inspected. During acquisition, the imaging time and spatial resolution are set to obtain the original image data set.

[0016] Perform illumination equalization processing on each original image in the original image dataset to obtain an illumination equalized image;

[0017] Perform reflection suppression processing on the image after illumination equalization to obtain the image after reflection suppression;

[0018] The image after reflection suppression is subjected to perspective geometric correction based on the intrinsic and extrinsic parameter matrices pre-calibrated during image acquisition to obtain a standardized reference image;

[0019] Perspective geometric correction maps each original image coordinate to reference coordinates based on intrinsic and extrinsic parameter matrices. Assuming the depth plane height is constant, the correction yields pixel coordinates under the standard reference plane. Perspective geometric correction is used to ensure the geometric consistency of the reinforcing bar strips in the image.

[0020] Optionally, the formation of the candidate set of welding regions includes:

[0021] For each reference image in the standardized reference image set, a multi-scale convolutional feature extraction network is used to calculate the channel-enhanced semantic feature value of each pixel in all semantic feature dimensions to obtain a channel-enhanced semantic feature map.

[0022] Based on the semantic feature map enhanced by each reference image channel, the probability value of each pixel belonging to the steel strip is calculated to obtain the steel strip probability map;

[0023] A threshold segmentation strategy is adopted for the probability map of steel strips. The probability value of the steel strip of each pixel is compared with the steel strip segmentation threshold. If the probability value of the steel strip is greater than or equal to the steel strip segmentation threshold, the pixel belongs to the steel strip region; otherwise, it is a non-steel strip region, thus forming a steel strip region mask.

[0024] Geometric skeleton extraction and endpoint analysis are performed on the mask of the rebar strip area to obtain the rebar strip skeleton map;

[0025] Based on the steel strip skeleton map, all centerline pixels that intersect in different directions are detected to obtain a candidate set of steel strip intersections;

[0026] Using the pixel position of each intersection point in the candidate set of rebar intersections as the center, a neighborhood window with a fixed radius is constructed. Within each neighborhood window, candidate welding areas are extracted, and all candidate welding areas are combined into a candidate welding area set.

[0027] Optionally, forming a steel mesh structure diagram including a set of nodes, a set of edges, and candidate weld point markers includes:

[0028] Based on the steel bar skeleton map, extract the set of pixel positions of all steel bar center lines in each reference image;

[0029] The node set of the rebar mesh is constructed based on the topological connection relationship in the pixel position set of the rebar centerline;

[0030] If there exists a path in the centerline pixel set between any two nodes that contains only the continuous pixels between the two nodes and does not pass through other nodes, then an edge is constructed to represent the steel strip segment on the path. All steel strip segments together form the edge set, and each edge in the edge set represents the steel strip connecting two intersection points.

[0031] Based on the node set, edge set, and welding region candidate set, calculate the spatial centroid position for each welding region candidate region;

[0032] Based on the location of the spatial centroid, all candidate welding areas that meet the spatial correlation conditions are uniformly marked as a set of candidate welding points, and a steel mesh structure diagram corresponding to each reference image is constructed.

[0033] Optionally, the spatial association conditions include:

[0034] If the Euclidean distance between the spatial centroid of a candidate welding area and a node is less than or equal to the spatial radius threshold, the corresponding candidate welding area is associated with the node and recorded as a suspicious welding point.

[0035] If the distance projection error between the spatial centroid of a candidate welding area and a certain edge is less than the spatial radius threshold, the corresponding candidate welding area is associated with the edge and recorded as a suspicious welding point.

[0036] Optionally, the output node embedding representation and connectivity confidence graph include:

[0037] A topology-preserving graph embedding method is performed on the rebar mesh structure diagram. Based on the set of nodes and edges in the rebar mesh structure diagram, an embedding method based on graph neural networks is used to map each node to an embedding vector in a low-dimensional embedding space. The local structural features of each node's embedding representation are constrained to be consistent with the degree of the actual node in the rebar mesh structure diagram by a node degree constraint regularization term. The loop structure consistency regularization term is constrained to ensure that the embedding representations of all nodes in the same loop form a closed or continuous trajectory in space. After training, each node obtains a unique embedding vector. The embedding vectors of all nodes together form the node embedding representation. Combined with connectivity analysis in the embedding space, the connectivity confidence graph of each node and edge is output.

[0038] During the embedding process, the adjacency-maintaining regularization term ensures that the embedding distances of connected nodes in the space are as close as possible. The space explicitly maintains the consistency of node adjacency relationships, node degree constraints, and loop structure. For each pair of nodes connected by an edge, the distance between the embedding vectors is forcibly shortened. Nodes with abnormal degrees are penalized, and nodes with loop interruptions caused by breakpoints or defects are subject to distinguishing constraints.

[0039] Optionally, the step of dividing the reference image and the steel mesh structure diagram into several image blocks and structure blocks includes:

[0040] The standardized reference image is divided into several image blocks, and the spatial boundary range of the image blocks is extracted.

[0041] Perform spatial intersection analysis between the spatial boundary of each image block and the graph elements in the steel mesh structure diagram, and extract all graph structure blocks that intersect with the image block in space.

[0042] A one-to-one mapping relationship is established between image blocks and graph structure blocks. Each image block corresponds to a graph structure block whose spatial boundary range is completely consistent with it. This is used to associate the area covered by each image block in the image domain with the corresponding graph structure block in the graph structure domain.

[0043] Optionally, the output includes a fused attention feature map of local details and cross-cell continuity features, including:

[0044] For each graph structure block, construct a block sparse attention graph structure, which includes local intra-block self-attention modules and remote stripe attention modules;

[0045] Within each graph block, a local self-attention module is applied. An attention matrix is ​​established based on the relative spatial relationships between nodes, and the local attention response values ​​between all nodes within each graph block are calculated.

[0046] For each graph structure block, determine the current graph structure block and its remote graph structure blocks that may have structural connections along the reinforcement direction based on the structural connection confidence value in the connectivity confidence graph.

[0047] A remote striped attention module is applied to graph block pairs that meet the remote connection conditions. Based on the node feature matrix of the current graph block and the node feature matrix of the remote graph block, a remote attention response value is constructed.

[0048] The local attention response values ​​are fused with all remote attention response values ​​to construct a fused attention feature map;

[0049] The fusion attention feature map assigns fusion weights to the feature representations obtained by each node under the local attention response value and the feature representations obtained under the remote attention response value, and then sums them up to obtain the final fusion feature representation of each node. The fusion feature representations of all nodes form the fusion attention feature map, which is used to integrate the local details in the current graph structure block with the directional continuity features between cross graph structure blocks.

[0050] Optionally, the step of locating the weld break point and outputting the defect location, break point type, and confidence level includes:

[0051] Extract the fusion attention feature map and the steel mesh structure map, and for each welding area candidate set, calculate the node degree, number of associated edges, loop closure and cross-graph structural block continuity score of the candidate welding points in the spatial location;

[0052] Based on the connectivity anomaly determination rules, determine whether the connectivity of each candidate welding point is abnormal, where:

[0053] If a node has a degree of 0 and no associated edges, or a loop closure of 0, it is considered a complete break.

[0054] If a node has a degree of 1 and an associated edge, but its loop closure is less than the specified threshold, it is determined to be a cold solder joint or a half solder joint.

[0055] If the node degree is normal and there are associated edges, but the spatial position is offset or the loop structure is damaged, it is judged as misaligned welding or missing welding.

[0056] All other cases are considered to have normal connectivity;

[0057] For each candidate weld point with connectivity anomalies, output the following:

[0058] Defect location: The horizontal and vertical coordinates of the spatial centroid of the weld point in the standard reference plane coordinate system;

[0059] Breakpoint types include complete breakpoints, cold or partial welds, misaligned welds, and missing welds.

[0060] Optionally, the consistency verification of the judgment result includes:

[0061] The pixel domain and structural domain breakpoint detection results of each image block and graph structure block are bidirectionally aligned. If a high uncertainty region is found, a re-inspection prompt is triggered by combining the local structural features of the corresponding image block and graph structure block.

[0062] The beneficial effects of this invention are:

[0063] This invention organically combines topology-preserving graph embedding and block sparse attention mechanism into the weld break detection process of rebar mesh. By skeletonizing and graph modeling structural elements such as rebar intersections and strips in the image, the topology-preserving graph embedding algorithm explicitly maintains node adjacency relationships, node degree constraints, and loop consistency constraints in low-dimensional space, transforming the weld continuity / discontinuity problem into a structural connectivity and topological closure discrimination problem. The reference image and structural graph are jointly divided into natural blocks, and the block sparse attention mechanism is combined to achieve fine-grained self-attention within the block and long-range stripe-like attention along the rebar direction, effectively supporting long-distance continuity judgment across cells. Even in the presence of strong reflections and construction site interference, it can still achieve robust differentiation between real break points and visual pseudo break points, significantly improving the structural consistency and anti-spoofing ability of weld break detection in complex scenarios.

[0064] This invention proposes a block sparse attention mechanism guided by structural prior dynamics. It uses the natural blocks of the rebar mesh as attention units and dynamically schedules sparse attention paths through a connectivity confidence graph. This activates continuous and reliable long-range block pairs on the structure, retaining only the directional dependencies across cells, achieving O(N·k) level sparse attention computation. This effectively breaks through the O(N²) computation bottleneck of traditional dense self-attention in large-format, high-resolution images, and balances the real-time performance and engineering deployability of the detection algorithm, providing a technical foundation for intelligent detection of rebar mesh production lines and on-site mobile terminals. Attached Figure Description

[0065] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0066] Figure 1 This is a flowchart of a method for identifying welding defects in steel mesh based on graph neural networks proposed in this invention. Detailed Implementation

[0067] Example 1:

[0068] refer to Figure 1 A method for identifying welding defects in steel mesh based on graph neural networks, comprising:

[0069] Collect large-format image data of the steel mesh and preprocess it to generate standardized reference images;

[0070] In this embodiment, generating a standardized reference image includes:

[0071] An industrial vision acquisition system is used to acquire images of the steel mesh to be inspected. During acquisition, the imaging time and spatial resolution are set to obtain the original image data set.

[0072] The raw image dataset consists of multiple raw images acquired at different imaging times and spatial resolutions. Each raw image includes the pixel value of each pixel in each channel.

[0073] Perform illumination equalization processing on each original image in the original image dataset to obtain an illumination equalized image;

[0074] The illumination equalization process uses histogram matching and brightness redistribution strategies to convert the brightness value of each pixel into an illumination equalization value after uniform processing. The illumination equalization brightness value is obtained by subtracting the mean of the original image brightness from the original image brightness, dividing by the standard deviation of the original image brightness, multiplying by the standard deviation of the target brightness distribution, and finally adding the mean of the target brightness distribution. The illumination equalization brightness value is used to reduce the impact of the difference between shadows and strong light in the steel reinforcement area.

[0075] Perform reflection suppression processing on the image after illumination equalization to obtain the image after reflection suppression;

[0076] The reflection suppression processing calculates the reflection response value for each pixel based on color saturation, brightness gradient, and local contrast. The reflection response value is the sum of color saturation, the magnitude of the brightness gradient, and local contrast. If the reflection response value exceeds a preset reflection threshold, polarization filtering and region smoothing are applied to the pixel region to suppress the impact of reflection interference on the texture structure of the steel reinforcement image.

[0077] The image after reflection suppression is subjected to perspective geometric correction based on the intrinsic and extrinsic parameter matrices pre-calibrated during image acquisition to obtain a standardized reference image;

[0078] Perspective geometric correction maps each original image coordinate to reference coordinates based on intrinsic and extrinsic parameter matrices. Assuming the depth plane height is constant, the correction yields pixel coordinates under the standard reference plane. Perspective geometric correction is used to ensure the geometric consistency of the reinforcing bar strips in the image.

[0079] The standardized reference image set consists of multiple reference images in a standard reference plane coordinate system, and each reference image includes the pixel value of each pixel in each channel.

[0080] The reference image is coarsely segmented to form a candidate set of welding areas;

[0081] In this embodiment, the formation of the candidate set of welding areas includes:

[0082] For each reference image in the standardized reference image set, a multi-scale convolutional feature extraction network is used to calculate the channel-enhanced semantic feature value of each pixel in all semantic feature dimensions to obtain a channel-enhanced semantic feature map.

[0083] For each reference image in the standardized reference image set, a multi-scale convolutional feature extraction network is used to perform convolution operations on all pixels of the reference image. The spatial neighborhood information of each pixel is extracted under different receptive fields, and channel enhancement is performed on each pixel in all semantic feature dimensions. The channel-enhanced semantic feature values ​​of each pixel in all semantic feature dimensions are combined to obtain a channel-enhanced semantic feature map.

[0084] Based on the semantic feature map enhanced by each reference image channel, the probability value of each pixel belonging to the steel strip is calculated to obtain the steel strip probability map;

[0085] The probability value of the rebar strip is obtained by taking the inner product of the channel-enhanced semantic feature value of each pixel position and the rebar strip classification weight vector, adding a bias, and then inputting it into the Sigmoid activation function. The probability value of the rebar strip is used to describe the possibility of each pixel being a rebar strip region.

[0086] ;

[0087] in, This is the classification weight vector for steel bar strips. For bias, It is the Sigmoid activation function. Channel-enhanced semantic feature map.

[0088] A threshold segmentation strategy is adopted for the probability map of steel strips. The probability value of the steel strip of each pixel is compared with the steel strip segmentation threshold. If the probability value of the steel strip is greater than or equal to the steel strip segmentation threshold, the pixel belongs to the steel strip region; otherwise, it is a non-steel strip region, thus forming a steel strip region mask.

[0089] The rebar strip area mask is used to identify the spatial distribution range of all rebar strips in the reference image.

[0090] Geometric skeleton extraction and endpoint analysis are performed on the mask of the rebar strip area to obtain the rebar strip skeleton map;

[0091] In Example 1, the geometric skeleton extraction and endpoint analysis specifically include refining the rebar strip region to a single pixel width to retain the main structure, obtaining the set of centerline pixel positions for each rebar strip, performing endpoint analysis on the set of centerline pixel positions, identifying endpoint pixels and bifurcation point pixels in the set of centerline pixel positions by detecting the number of neighboring pixels of each centerline pixel, and the information of all centerline pixel positions and endpoint and bifurcation point pixels together forming the rebar strip skeleton map; the rebar strip skeleton map represents the main structure of the rebar strip, and at the same time, the endpoint attributes of each pixel in the set of centerline pixel positions are analyzed.

[0092] Based on the steel strip skeleton map, all centerline pixels that intersect in different directions are detected to obtain a candidate set of steel strip intersections;

[0093] Each intersection pixel in the candidate set of rebar intersections represents the intersection point of two centerline pixel sets. Only when the angle between the two centerlines is within the range of legal intersection angles is the corresponding intersection pixel position included in the candidate set of rebar intersections.

[0094] Using the pixel position of each intersection point in the candidate set of rebar intersections as the center, a neighborhood window with a fixed radius is constructed. Within each neighborhood window, candidate welding areas are extracted, and all candidate welding areas are combined into a candidate welding area set.

[0095] The neighborhood window consists of all pixels whose distance from the intersection point is less than or equal to the fixed radius. Within each neighborhood window, candidate weld regions are extracted, and the candidate weld region set is used to mark all spatial regions that may contain rebar weld points.

[0096] Extract the steel mesh structure diagram, model the steel bar intersections as graph nodes and the steel bar strips as graph edges, and spatially associate the candidate set of welding areas with nodes and edges, mark the positions of suspicious welding points, and form the steel mesh structure diagram.

[0097] In this embodiment, a steel mesh structure diagram containing a set of nodes, a set of edges, and candidate weld point markers is formed, including:

[0098] Based on the steel bar skeleton map, extract the set of pixel positions of all steel bar center lines in each reference image;

[0099] Each centerline pixel position in the set of centerline pixel positions of the reinforcing bars is used to represent the position of the main reinforcing bar in the image after skeleton refinement.

[0100] The node set of the rebar mesh is constructed based on the topological connection relationship in the pixel position set of the rebar centerline;

[0101] Each node in the node set of the rebar mesh diagram corresponds to the intersection pixel position in the rebar strip skeleton diagram where there is an intersection relationship. The construction of the node depends on the intersection angle relationship of the center line pixel in the local neighborhood. If the intersection angle falls within the range of the legal intersection angle, it is determined to be a node.

[0102] If there exists a path in the centerline pixel set between any two nodes that contains only the continuous pixels between the two nodes and does not pass through other nodes, then an edge is constructed to represent the steel strip segment on the path. All steel strip segments together form the edge set, and each edge in the edge set represents the steel strip connecting two intersection points.

[0103] Based on the node set, edge set, and welding region candidate set, calculate the spatial centroid position for each welding region candidate region;

[0104] The spatial centroid position is obtained by summing the x-coordinates and y-coordinates of all pixels within the candidate area of ​​the welding region and dividing each sum by the total number of pixels. The spatial centroid position is used to represent the geometric center of the candidate area of ​​the welding region in the standard reference plane coordinate system.

[0105] Based on the location of the spatial centroid, all candidate welding areas that meet the spatial correlation conditions are uniformly marked as a set of candidate welding points, and a steel mesh structure diagram corresponding to each reference image is constructed.

[0106] Each candidate welding point region in the candidate welding point set is a suspected welding point region covered by nodes or edge spaces. The steel mesh structure diagram consists of three parts: node set, edge set, and candidate welding point set.

[0107] In this embodiment, the spatial association conditions include:

[0108] If the Euclidean distance between the spatial centroid of a candidate welding area and a node is less than or equal to the spatial radius threshold, the corresponding candidate welding area is associated with the node and recorded as a suspicious welding point.

[0109] If the distance projection error between the spatial centroid of a candidate welding area and a certain edge is less than the spatial radius threshold, the corresponding candidate welding area is associated with the edge and recorded as a suspicious welding point.

[0110] Perform a topology-preserving graph embedding operation on the steel mesh structure diagram to construct a low-dimensional embedding space and output the node embedding representation and connectivity confidence graph.

[0111] In this embodiment, the output node embedding representation and connectivity confidence graph include:

[0112] A topology-preserving graph embedding method is performed on the rebar mesh structure diagram. Based on the set of nodes and edges in the rebar mesh structure diagram, an embedding method based on graph neural networks is used to map each node to an embedding vector in a low-dimensional embedding space. The local structural features of each node's embedding representation are constrained to be consistent with the degree of the actual node in the rebar mesh structure diagram by a node degree constraint regularization term. The loop structure consistency regularization term is constrained to ensure that the embedding representations of all nodes in the same loop form a closed or continuous trajectory in space. After training, each node obtains a unique embedding vector. The embedding vectors of all nodes together form the node embedding representation. Combined with connectivity analysis in the embedding space, the connectivity confidence graph of each node and edge is output.

[0113] During the embedding process, the adjacency-maintaining regularization term ensures that the embedding distances of connected nodes in the space are as close as possible. The space explicitly maintains the consistency of node adjacency relationships, node degree constraints, and loop structure. For each pair of nodes connected by an edge, the distance between the embedding vectors is forcibly shortened. Nodes with abnormal degrees are penalized, and nodes with loop interruptions caused by breakpoints or defects are subject to distinguishing constraints.

[0114] The reference image and the steel mesh structure diagram are divided into several image blocks and diagram structure blocks.

[0115] In this embodiment, the reference image and the steel mesh structure diagram are jointly divided into several image blocks and diagram structure blocks, including:

[0116] The standardized reference image is divided into several image blocks, and the spatial boundary range of the image blocks is extracted.

[0117] Each image block represents a rectangular sub-region in the reference image corresponding to a certain section of steel strip or cell region. The number of image blocks is equal to the number of all rectangular sub-regions obtained from the division.

[0118] The spatial boundary is uniquely determined by the pixel positions of the top-left and bottom-right pixel coordinates in the standard reference plane coordinate system.

[0119] Perform spatial intersection analysis between the spatial boundary of each image block and the graph elements in the steel mesh structure diagram, and extract all graph structure blocks that intersect with the image block in space.

[0120] Each graph structure block includes a subset of nodes whose spatial coordinates fall within the spatial boundary of the image block, a subset of edges whose endpoints are all within the subset of nodes, and a subset of candidate welding points whose spatial centroids fall within the spatial boundary of the image block.

[0121] A one-to-one mapping relationship is established between image blocks and graph structure blocks. Each image block corresponds to a graph structure block whose spatial boundary range is completely consistent with it. This is used to associate the area covered by each image block in the image domain with the corresponding graph structure block in the graph structure domain.

[0122] A block sparse attention mechanism is applied to image patches and their corresponding graph structure blocks for feature inference, and the output is a fused attention feature map that includes local details and cross-cell continuity features.

[0123] In this embodiment, the output is a fused attention feature map that includes local details and cross-cell continuity features, including:

[0124] For each graph structure block, construct a block sparse attention graph structure, which includes local intra-block self-attention modules and remote stripe attention modules;

[0125] The node features in each graph structure block are represented as a feature matrix. The number of rows in the feature matrix is ​​the number of nodes in the graph structure block, and the number of columns is the feature dimension of each node.

[0126] Within each graph block, a local self-attention module is applied. An attention matrix is ​​established based on the relative spatial relationships between nodes, and the local attention response values ​​between all nodes within each graph block are calculated.

[0127] In Example 1, spatial weights are assigned based on the spatial positional relationship between each node and other nodes and the similarity of node features. The node feature vector is then weighted with the feature vectors of adjacent nodes, and the weighted result is normalized to obtain the attention weights between node pairs. The attention weights between all node pairs form an attention matrix. The local attention response value is obtained by multiplying the node feature vector with the corresponding attention weight one by one and accumulating the results. This value is used to measure the structural correlation between all nodes within the graph structure block.

[0128] For each graph structure block, determine the current graph structure block and its remote graph structure blocks that may have structural connections along the reinforcement direction based on the structural connection confidence value in the connectivity confidence graph.

[0129] Each remote graph structure block is determined by node pairs whose connectivity confidence values ​​exceed the connectivity confidence activation threshold.

[0130] A remote striped attention module is applied to graph block pairs that meet the remote connection conditions. Based on the node feature matrix of the current graph block and the node feature matrix of the remote graph block, a remote attention response value is constructed.

[0131] In Example 1, a remote striped attention module is applied to graph block pairs that meet the remote connection conditions. Based on the node feature matrix of the current graph block and the node feature matrix of the remote graph block, the spatial orientation consistency weight of each pair of nodes is calculated. The spatial orientation consistency weight is obtained by comparing the angle between the line direction connecting the current node and the remote node in the spatial coordinate system and the direction of the main axis of the reinforcing bar. The smaller the angle, the higher the weight. The spatial orientation consistency weight is multiplied by the node feature similarity weight to obtain the remote attention response value between each pair of nodes. All remote attention response values ​​are combined into a remote attention response matrix. The node feature similarity is obtained by calculating the cosine similarity between the feature vector of the current node and the feature vector of the remote node. The remote attention response matrix is ​​used to measure the continuity of directional features between the current graph block and the remote block.

[0132] The local attention response values ​​are fused with all remote attention response values ​​to construct a fused attention feature map;

[0133] The fusion attention feature map assigns fusion weights to the feature representations obtained by each node under the local attention response value and the feature representations obtained under the remote attention response value, and then sums them up to obtain the final fusion feature representation of each node. The fusion feature representations of all nodes form the fusion attention feature map, which is used to integrate the local details in the current graph structure block with the directional continuity features between cross graph structure blocks.

[0134] Based on the fusion of attention feature map, steel mesh structure map and welding area candidate set, and combined with the spatial topology and connectivity features of nodes, edges and candidate welding points in each image block and graph structure block, the system determines whether the connectivity of each candidate welding point is abnormal, locates the welding breakpoint and outputs the defect location and breakpoint type, and verifies the consistency of the judgment results.

[0135] In this embodiment, locating the weld break point and outputting the defect location, break point type, and confidence level includes:

[0136] Extract the fusion attention feature map and the steel mesh structure map, and for each welding area candidate set, calculate the node degree, number of associated edges, loop closure and cross-graph structural block continuity score of the candidate welding points in the spatial location;

[0137] In Example 1, the node degree is obtained by counting the number of edges connected to the candidate welding point as a node in the steel mesh structure diagram. The node degree reflects the number of physical connections between the corresponding position and the surrounding steel strips. The number of associated edges is obtained by counting the total number of edges directly connected to the candidate welding point in the steel mesh structure diagram. The number of associated edges determines whether the welding point is connected by the steel strip. The loop closure is obtained by detecting whether the smallest loop structure including the candidate welding point is closed. A loop closure of 1 indicates that the welding point is on a complete loop, and a loop closure of 0 indicates that the loop is interrupted or broken. The continuity score across the graph structure block is obtained by analyzing the similarity of node features between the graph structure block where the candidate welding point is located and the adjacent graph structure blocks and the attention response intensity along the steel strip direction. The higher the continuity score, the better the continuity of the steel strip. The lower the continuity score, the more likely there is a structural interruption.

[0138] Based on the connectivity anomaly determination rules, determine whether the connectivity of each candidate welding point is abnormal, where:

[0139] If a node has a degree of 0 and no associated edges, or a loop closure of 0, it is considered a complete break.

[0140] If a node has a degree of 1 and an associated edge, but its loop closure is less than the specified threshold, it is determined to be a cold solder joint or a half solder joint.

[0141] If the node degree is normal and there are associated edges, but the spatial position is offset or the loop structure is damaged, it is judged as misaligned welding or missing welding.

[0142] All other cases are considered to have normal connectivity;

[0143] For each candidate weld point with connectivity anomalies, output the following:

[0144] Defect location: The horizontal and vertical coordinates of the spatial centroid of the weld point in the standard reference plane coordinate system;

[0145] Breakpoint types include complete breakpoints, cold or partial welds, misaligned welds, and missing welds.

[0146] In this embodiment, consistency verification of the judgment result includes:

[0147] The pixel domain and structural domain breakpoint detection results of each image block and graph structure block are bidirectionally aligned. If a high uncertainty region is found, a re-inspection prompt is triggered by combining the local structural features of the corresponding image block and graph structure block.

[0148] Example 2:

[0149] During the production of steel mesh, the production line's automatic inspection system automatically photographs each mesh sheet, with a single image resolution of 10000×5000 pixels. In one day, the production line collected a total of 1800 steel mesh images. The system selected a batch of 200 sample mesh sheets for breakpoint detection experiments.

[0150] In a conventional production line environment, in a certain batch of images, the system automatically completed the illumination equalization and reflection suppression of the images. Due to the different lighting angles at the construction site, some sample images showed strong reflective spots on the surface of the steel bars, and some weld areas were even partially covered by steel bar dust and welding slag.

[0151] After the detection process is started, the system segments the rebar region for each standardized reference image and extracts the strips and intersections. In Example 2, in the image numbered "1001234", 59 rebar strips and 367 intersections were automatically extracted. In the skeleton analysis stage, the system automatically found 4 strip fracture structures, 2 of which fell exactly near the intersection of two rebars and were initially marked as candidate weld break points.

[0152] In the further structural modeling process, the system established a set of nodes (intersections) and a set of edges (strips) based on the skeleton relationships, and associated the 34 automatically segmented candidate welding areas with nodes and edges through spatial centroid analysis. At this point, the mesh structure diagram numbered "1001234" has been completely established by the system.

[0153] Subsequently, the system divides the image into 12 rebar strip blocks and 30 cell blocks, with each image block mapped to a structural block. In Example 2, image block number 5 corresponds to the structural sub-image of the 45th transverse rebar area and the 36th longitudinal rebar area of ​​the mesh. Attention weight relationships are established within each block and between distant blocks along the rebar direction.

[0154] During the block sparse attention inference stage, the system detected a significant decrease in the continuity score of block 9, a weakening of the local self-attention response, and a break in the cross-block long-range stripe attention chain in this region. The node degree of one candidate solder joint in this block became 0, and the loop closure was 0. The system determined it to be a complete breakpoint with a breakpoint confidence level of 0.97. The output detection result is: breakpoint location (pixel coordinates 4211, 2347), type "complete breakpoint", confidence level 0.97.

[0155] Another sample mesh, numbered "1001251", has a solder joint covered by mortar in its lower right corner cell area. The pixel domain features are not obvious, but the self-attention chain within the structural block is not broken. The cross-block continuity score is 0.81, the node degree is 1, and the loop closure is 0.2. Based on the connectivity anomaly rule, the system determines it to be a "cold solder joint or half solder joint". The breakpoint confidence is 0.78, and the output detection result is: breakpoint location (pixel coordinates 9534, 4860), type "cold solder joint or half solder joint", confidence 0.78.

[0156] On the mesh with test sample number "1001276", there is a solder joint where all indicators in the structural domain are normal, but there is a significant pixel offset between the spatial centroid and the node, and the corresponding loop structure is partially obscured. The system analysis shows a cross-block continuity score of 0.64, a node degree of 2, and a number of associated edges of 2, but the spatial centroid offset of this solder joint reaches 36 pixels. According to the judgment rules, the system classifies it as "misaligned welding or missing welding", with a breakpoint confidence level of 0.69, and outputs the detection result as: breakpoint location (pixel coordinates 1547, 2992), type "misaligned welding or missing welding", confidence level 0.69.

[0157] All test results are automatically generated into a defect distribution report. The production line MES system receives the coordinates, type, and confidence level of the welding breakpoints for each mesh piece. When the breakpoint confidence level is below 0.8, it is automatically marked as a high uncertainty sample, triggering a re-inspection process.

[0158] Comparative experiment with traditional methods: For the same batch of 200 sample net images, 364 actual breakpoints were manually labeled. The method of this invention detected 358 breakpoints, with 2 false positives and 6 false negatives. The traditional method based on pixel segmentation and edge detection detected 329 breakpoints, with 11 false positives and 35 false negatives. The statistical data is shown in Table 1 below:

[0159] Table 1. Comparative experimental data of the method of the present invention and the traditional method.

[0160] method Detection accuracy False positive rate False negative rate Average inspection time per sheet (seconds) Average positioning error (pixels) Method of the present invention 98.3% 0.6% 1.7% 1.2 5.1 Traditional segmentation + edge detection method 90.3% 3.0% 9.6% 3.6 13.7

[0161] In 20 difficult samples with complex strong reflections, obstructions and rust contamination, the breakpoint recall rate of the method of the present invention was 95.5%, while that of the traditional method was only 78.4%.

[0162] In Example 2, in the night shift sample number "1001299", three actual breakpoints were manually marked. The present invention successfully detected all of them, while the traditional method only detected one and falsely reported two solder joints as breakpoints. Similarly, in sample number "1001244", two solder joints were falsely identified due to fixture obstruction. The present invention supplemented the continuity information through a cross-block attention mechanism, thus avoiding false detections, while the traditional method produced false alarms.

[0163] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A steel bar mesh welding defect identification method based on a graph neural network, characterized in that, The method comprises the following steps: Collecting large-format image data of the reinforcement mesh and performing preprocessing to generate a standardized reference image; Performing coarse segmentation of the reinforcement area on the reference image to form a candidate set of welding areas; Extracting a reinforcement mesh structure graph, modeling the intersection of the reinforcement mesh as a graph node, modeling the reinforcement strip as a graph edge, and spatially correlating the candidate set of welding areas with the nodes and edges to mark the positions of suspected welding points and form a reinforcement mesh structure graph; Performing a topological preserving graph embedding operation on the reinforcement mesh structure graph to construct a low-dimensional embedding space and output node embedding representations and a connectedness confidence map; Dividing the reference image and the reinforcement mesh structure graph into a plurality of image blocks and graph structure blocks; Applying a block sparse attention mechanism to the image blocks and their corresponding graph structure blocks to perform feature reasoning and output a fusion attention feature map including local details and cross-cell continuity features; Constructing a block sparse attention graph structure for each graph structure block, which includes a local intra-block self-attention module and a long-range striped attention module; Applying the local intra-block self-attention module within each graph structure block to establish an attention matrix based on the relative spatial position relationship between the nodes and calculate the local attention response values between all nodes within each graph structure block; For each graph structure block, determining the long-range graph structure blocks that may have structural connections with the current graph structure block along the direction of the reinforcement based on the structural connection confidence values in the connectedness confidence map; Applying the long-range striped attention module to the graph structure block pairs that meet the long-range connection condition, constructing long-range attention response values based on the node feature matrix of the current graph structure block and the node feature matrix of the long-range graph structure block; Fusing the local attention response values and all long-range attention response values to construct a fusion attention feature map; The fusion attention feature map assigns fusion weights to the feature representation obtained by each node under the action of the local attention response value and the feature representation obtained by each node under the action of the long-range attention response value, and then performs weighted summation to obtain the final fusion feature representation of each node. The fusion feature representation of all nodes constitutes a fusion attention feature map, which is used to integrate the local detail information in the current graph structure block and the directional continuity features between the graph structure blocks; Based on the fusion attention feature map, the reinforcement mesh structure graph, and the candidate set of welding areas, and in combination with the spatial topology and connectivity features of the nodes, edges, and candidate welding points in each image block and graph structure block, it is determined whether the connectivity of each candidate welding point is abnormal, the welding breakpoint is located, and the defect position and breakpoint type are output, and the consistency of the judgment result is verified.

2. The steel bar mesh welding defect identification method based on a graph neural network according to claim 1, characterized in that, The method for generating a standardized reference image comprises the following steps: Using an industrial vision acquisition system to acquire images of the reinforcement mesh to be inspected, setting the imaging time and spatial resolution during acquisition, and obtaining a set of original image data; Performing illumination equalization processing on each original image in the set of original image data to obtain an image after illumination equalization; Performing reflection suppression processing on the image after illumination equalization to obtain an image after reflection suppression; Performing perspective geometric correction on the image after reflection suppression based on the intrinsic matrix and extrinsic matrix pre-calibrated during the image acquisition process to obtain a standardized reference image; The perspective geometry correction maps each original image coordinate to a reference coordinate according to the intrinsic matrix and the extrinsic matrix, and obtains pixel coordinates on a standard reference plane after correction, and the perspective geometry correction is used to ensure the geometric consistency of the steel bar strips in the images.

3. The steel bar mesh welding defect identification method based on a graph neural network according to claim 2, characterized in that, The forming of the candidate set of welding regions comprises: For each reference image in the set of normalized reference images, a multi-scale convolution feature extraction network is used to calculate a channel-enhanced semantic feature value of each pixel in all semantic feature dimensions, to obtain a channel-enhanced semantic feature map; Based on the channel-enhanced semantic feature map of each reference image, a probability value of each pixel belonging to the steel bar strip is calculated, to obtain a steel bar strip probability map; A threshold segmentation strategy is used for the steel bar strip probability map, and the probability value of each pixel of the steel bar strip is compared with a steel bar strip segmentation threshold value, if the probability value of the steel bar strip is greater than or equal to the steel bar strip segmentation threshold value, the pixel belongs to the steel bar strip region, otherwise, it is a non-steel bar strip region, to form a steel bar strip region mask; Geometric skeleton extraction and endpoint analysis are performed on the steel bar strip region mask, to obtain a steel bar skeleton map; Based on the steel bar skeleton map, all centerline pixel points intersecting in different directions are detected, to obtain a candidate set of steel intersection points; Taking each intersection pixel position in the candidate set of steel intersection points as the center, a neighborhood window with a fixed radius is constructed, and a candidate welding region is extracted in each neighborhood window, and all the candidate welding regions are combined to form a candidate set of welding regions.

4. The steel bar mesh welding defect identification method based on a graph neural network according to claim 3, characterized in that, The steel mesh structure graph comprising a node set, an edge set and a candidate welding point label is formed, comprising: Based on the steel bar skeleton map, a set of all steel centerline pixel positions in each reference image is extracted; A node set of the steel mesh graph is constructed according to the topological connection relationship in the set of steel centerline pixel positions; If there is a path between any two nodes in the set of centerline pixels that only contains the continuous pixel points between the two nodes and does not pass through other nodes, a edge is constructed to represent the steel bar strip segment on the path, and all steel bar strip segments together form an edge set, and each edge in the edge set represents a steel bar strip connecting two intersection points; Based on the node set, the edge set and the candidate set of welding regions, the spatial barycenter position of each candidate welding region is calculated; All candidate welding regions meeting the spatial correlation condition are uniformly labeled as a candidate set of welding points according to the spatial barycenter position, and a steel mesh structure graph corresponding to each reference image is constructed.

5. The steel bar mesh welding defect identification method based on a graph neural network according to claim 4, characterized in that, The spatial correlation condition comprises: If the spatial barycenter position of the candidate welding region and the Euclidean distance of a node are less than or equal to the spatial radius threshold, the corresponding candidate welding region is associated with the node and recorded as a suspicious welding point label; If the spatial barycenter position of the candidate welding region and the distance projection error of an edge are less than the spatial radius threshold, the corresponding candidate welding region is associated with the edge and recorded as a suspicious welding point label.

6. The steel bar mesh welding defect identification method based on a graph neural network according to claim 5, characterized in that, The output node embedding representation and connectivity confidence map comprises: The topology preserving graph embedding is performed on the steel bar mesh structure graph, based on the node set and edge set in the steel bar mesh structure graph, each node is mapped to an embedding vector in a low-dimensional embedding space by using a graph neural network-based embedding method, the local structure features of each node embedding representation are constrained to be consistent with the degree of the actual node in the steel bar mesh structure graph by a node degree constraint regular term, all node embedding representations in the same loop form a closed or continuous track in the space by a loop structure consistency regular term, each node obtains a unique embedding vector after training, and the embedding vectors of all nodes collectively form a node embedding representation, and a connected confidence map of each node and edge is output by combining the connectivity analysis in the embedding space.

7. The steel bar mesh welding defect identification method based on a graph neural network according to claim 6, characterized in that, The reference image and the steel bar mesh structure graph are divided into a plurality of image blocks and graph structure blocks, including: The reference image is divided into a plurality of image blocks, and the spatial boundary range of each image block is extracted; The spatial boundary range of each image block is analyzed with the graph elements in the steel bar mesh structure graph, and all graph structure blocks intersecting with the image block in space are extracted; A one-to-one mapping relationship is established between the image block and the graph structure block, each image block corresponds to a graph structure block whose spatial boundary range is completely consistent, and the region covered by each image block in the image domain is associated with the corresponding graph structure block in the graph structure domain.

8. The steel bar mesh welding defect identification method based on a graph neural network according to claim 7, characterized in that, The welding breakpoint is positioned and the defect position, breakpoint type and confidence are output, including: The fusion attention feature map and the steel bar mesh structure graph are extracted, and for each candidate welding point in the candidate welding area set, the node degree, the number of associated edges, the loop closure and the cross-graph structure block continuity score of the spatial position are calculated; According to the connectivity anomaly judgment rule, whether the connectivity of each candidate welding point is abnormal is judged, wherein: If the node degree is 0 and there is no associated edge, or the loop closure is 0, it is determined as a complete breakpoint; If the node degree is 1 and there is an associated edge, but the loop closure is less than a specified threshold, it is determined as a virtual weld or a half weld; If the node degree is normal and there is an associated edge, but the spatial position is offset or the loop structure is damaged, it is determined as a mispositioned weld or a missed weld; The rest is determined as normal connectivity; For each candidate welding point with abnormal connectivity, the following contents are output: Defect position: the horizontal coordinate and the vertical coordinate of the spatial center of gravity of the welding point in the standard reference plane coordinate system; Breakpoint type: including complete breakpoint, virtual weld or half weld, mispositioned weld or missed weld.

9. The steel bar mesh welding defect identification method based on a graph neural network according to claim 7, characterized in that, The consistency of the judgment result is verified, including: The pixel domain and the structure domain breakpoint detection results of each image block and graph structure block are aligned bidirectionally, if a high-uncertainty area is found, the local structure features of the corresponding image block and graph structure block are combined to trigger a re-examination prompt.

Images