Reinforcing mesh welding defect identification method based on graph neural network

Abstract

The invention discloses a reinforcing mesh welding defect identification method based on a graph neural network. The method comprises the following steps: generating a standardized reference image; forming a welding area candidate set; extracting a reinforcing mesh structure diagram, and marking positions of suspicious welding points to form the reinforcing mesh structure diagram; performing topology preserving graph embedding operation on the reinforcing mesh structure graph, constructing a low-dimensional embedding space, and outputting node embedding representation and a communication confidence graph; jointly dividing the reference image and the reinforcing mesh structure diagram into a plurality of image blocks and diagram structure blocks; outputting a fusion attention feature map including local details and cross-cell continuity features; and judging whether the connectivity of each candidate welding spot is abnormal or not, positioning a welding breakpoint, outputting a defect position and a breakpoint type, and performing consistency verification on a judgment result. According to the method, the structural consistency and the anti-artifact capability of welding breakpoint detection in a complex scene are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of steel bar mesh, and in particular to a steel bar mesh welding defect identification method based on a graph neural network. BACKGROUND

[0002] With the advancement of intelligent construction of civil engineering and automatic production of steel bar mesh, the demand for non-destructive testing of steel bar mesh welding quality is increasingly prominent. Traditional steel bar mesh welding breakpoint detection methods mainly rely on manual inspection or machine vision algorithms based on pixel segmentation, edge detection and key point extraction.

[0003] In actual construction environments such as complex backgrounds, strong reflections and local occlusions, traditional methods are easily disturbed by visual artifacts such as rust spots, construction reflections and mortar coverage, and it is difficult to stably distinguish between real welding breakpoints and visual false breakpoints, resulting in high breakpoint false detection rate and missed detection rate.

[0004] Existing deep learning methods based on global self-attention consume a large amount of computing resources when facing large-format high-resolution images, and real-time performance and deployability are severely limited. Existing pixel domain methods cannot effectively express the "intersection-bar-loop" topology of steel bar mesh, and cannot accurately distinguish the continuity of complex structures, resulting in insufficient stability of welding defect positioning and type judgment. SUMMARY

[0005] One object of the present application is to provide a steel bar mesh welding defect identification method based on a graph neural network, which significantly improves the structural consistency and anti-artifact capability of welding breakpoint detection in complex scenarios.

[0006] According to the steel bar mesh welding defect identification method based on the graph neural network of the present application, the method comprises: Collecting steel bar mesh large-format image data and performing preprocessing to generate standardized reference images; Performing rough segmentation of the steel bar region on the reference image to form a welding region candidate set; Extracting the steel bar mesh structure graph, modeling the steel bar intersection as a graph node, modeling the steel bar strip as a graph edge, and simultaneously associating the welding region candidate set with the nodes and edges in space to mark the suspicious welding point position, forming a steel bar mesh structure graph; Performing a topological preserving graph embedding operation on the steel bar mesh structure graph to construct a low-dimensional embedding space and output node embedding representation and connectivity confidence map; Dividing the reference image and the steel bar mesh structure graph into a plurality of image blocks and graph structure blocks; Applying a block sparse attention mechanism to the image block and its corresponding graph structure block to perform feature reasoning and output a fusion attention feature map including local details and cross-cell continuity features; Based on the fusion attention feature map, the reinforcement mesh structure map and the welding area candidate set, and combined with the spatial topological and connectivity features of each image block, edge and candidate welding point in the graph structure block, whether the connectivity of each candidate welding point is abnormal is judged, the welding breakpoint is located, the defect position and the breakpoint type are output, and the consistency of the judgment result is verified.

[0007] Optionally, the generating the standardized reference image comprises: An industrial vision acquisition system is used to acquire images of the steel reinforcement mesh to be inspected, imaging time and spatial resolution are set during acquisition, and a set of original image data is obtained; Each original image in the set of original image data is subjected to illumination equalization processing to obtain an image after illumination equalization; The image after illumination equalization is subjected to reflection suppression processing to obtain an image after reflection suppression; The image after reflection suppression is subjected to perspective geometric correction based on the intrinsic matrix and extrinsic matrix pre-calibrated during the image acquisition process to obtain a standardized reference image; The perspective geometric correction maps each original image coordinate to a reference coordinate according to the intrinsic matrix and extrinsic matrix, assuming that the depth plane height is a constant, the pixel coordinates under the standard reference plane are obtained after correction, and the perspective geometric correction is used to ensure the geometric consistency of the steel reinforcement strip in the image.

[0008] Optionally, the forming of the welding area candidate set comprises: For each reference image in the set of standardized reference images, a multi-scale convolution feature extraction network is used to calculate the channel enhanced semantic feature value of each pixel in all semantic feature dimensions to obtain a channel enhanced semantic feature map; Based on the channel enhanced semantic feature map of each reference image, the probability value of each pixel belonging to the steel reinforcement strip is calculated to obtain a steel reinforcement strip probability map; A threshold segmentation strategy is used for the steel reinforcement strip probability map, the probability value of each pixel of the steel reinforcement strip is compared with the steel reinforcement strip segmentation threshold, if the probability value of the steel reinforcement strip is greater than or equal to the steel reinforcement strip segmentation threshold, the pixel belongs to the steel reinforcement strip region, otherwise it is a non-steel reinforcement strip region, forming a steel reinforcement strip region mask; The steel reinforcement strip region mask is subjected to geometric skeleton extraction and endpoint analysis to obtain a steel reinforcement strip skeleton map; Based on the steel reinforcement strip skeleton map, all centerline pixel points intersecting in different directions are detected to obtain a steel intersection point candidate set; Taking each intersection pixel position in the steel intersection point candidate set as the center, a neighborhood window with a fixed radius is constructed, and a welding area candidate region is extracted in each neighborhood window, and all welding area candidate regions are combined to form a welding area candidate set.

[0009] Optionally, the forming the reinforcement mesh structure graph comprising a node set, an edge set and a candidate weld point label comprises: Based on the reinforcement bar strip skeleton graph, a set of all reinforcement center line pixel positions in each reference image is extracted; According to the topological connection relationship in the set of reinforcement center line pixel positions, a node set of the reinforcement mesh graph is constructed; If there is a path between any two nodes in the center line pixel set that only contains the continuous pixel points between the two nodes and does not pass through other nodes, a edge is constructed to represent the reinforcement bar strip segment on the path, and all reinforcement bar strip segments together form an edge set, each edge in the edge set represents a reinforcement bar strip connecting two intersection points; Based on the node set, the edge set and the candidate weld region set, the spatial barycenter position of each candidate weld region is calculated; According to the spatial barycenter position, all weld region candidate regions that satisfy the spatial correlation condition are uniformly labeled as a candidate weld point set, and a reinforcement mesh structure graph corresponding to each reference image is constructed.

[0010] Optionally, the spatial correlation condition comprises: If the spatial barycenter position of the weld region candidate region and the Euclidean distance of a certain node are less than or equal to the spatial radius threshold, the corresponding weld region candidate region is associated with the node and recorded as a suspicious weld point label; If the spatial barycenter position of the weld region candidate region and the distance projection error of a certain edge are less than the spatial radius threshold, the corresponding weld region candidate region is associated with the edge and recorded as a suspicious weld point label.

[0011] Optionally, the output node embedding representation and connectivity confidence map comprises: Topological preserving graph embedding is performed on the reinforcement mesh structure graph, based on the node set and the edge set in the reinforcement mesh structure graph, each node is mapped to an embedding vector in a low-dimensional embedding space using a graph neural network-based embedding method, the local structure features of each node embedding representation are constrained to be consistent with the degree of the actual node in the reinforcement mesh structure graph through a node degree constraint regular term, all node embedding representations within the same loop form a closed or continuous track in space through a loop structure consistency regular term, each node obtains a unique embedding vector after training, and the embedding vectors of all nodes together form a node embedding representation, and a connectivity confidence map of each node and edge is output in combination with the connectivity analysis in the embedding space.

[0012] The adjacency preserving regularizer in the embedding process ensures that the embedding distance of connected nodes in the space is as close as possible, and the explicit maintenance of node adjacency relationship, node degree constraint and loop structure consistency in the space shortens the distance between embedding vectors for each pair of nodes connected by an edge, and imposes penalties on nodes with abnormal degrees and distinguishes nodes with broken loops caused by breakpoints or defects.

[0013] Optionally, the dividing the reference image and the reinforcement mesh structure diagram into a plurality of image blocks and graph structure blocks comprises: Dividing the standardized reference image into a plurality of image blocks, and extracting the spatial boundary range of the image blocks; Performing spatial intersection analysis on the spatial boundary range of each image block and the graph elements in the reinforcement mesh structure diagram to extract all graph structure blocks intersecting with the image blocks in space; Establishing a one-to-one mapping relationship between the image blocks and the graph structure blocks, each image block corresponding to a graph structure block whose spatial boundary range is completely consistent with that of the image block, for associating the area covered by each image block in the image domain with the corresponding graph structure block in the graph structure domain.

[0014] Optionally, the output includes a fusion attention feature map of local details and cross-cell continuity features, comprising: Constructing a block sparse attention graph structure for each graph structure block, the block sparse attention graph structure including a local intra-block self-attention module and a remote striped attention module; Applying the local intra-block self-attention module within each graph structure block to establish an attention matrix based on the relative spatial position relationship between nodes and calculate the local attention response value between all nodes within each graph structure block; For each graph structure block, determine the remote graph structure blocks that may have structural connections with the current graph structure block along the reinforcement direction according to the structural connection confidence values in the connectivity confidence map; Apply the remote striped attention module to the graph structure block pairs that meet the remote connection condition, and construct a remote attention response value based on the node feature matrix of the current graph structure block and the node feature matrix of the remote graph structure block; Fuse the local attention response value and all remote attention response values to construct a fusion attention feature map; The fusion attention feature map assigns a fusion weight to the feature representation obtained by each node under the action of the local attention response value and the feature representation obtained by each node under the action of the remote attention response value, and performs weighted summation to obtain the final fusion feature representation of each node. The fusion feature representations of all nodes constitute the fusion attention feature map, which is used to integrate the local detail information in the current graph structure block and the directional continuity features between the graph structure blocks.

[0015] Optionally, the positioning welding breakpoint and outputting the defect position, breakpoint type and confidence level comprise: The fusion attention feature map and the reinforcement mesh structure map are extracted, and for each candidate welding point in the candidate welding area set, the node degree, the associated edge number, the loop closure and the cross-graph structure block continuity score of the spatial position are calculated. According to the connectivity abnormality judgment rule, whether the connectivity of each candidate welding point is abnormal is judged, wherein: If the node degree = 0 and there is no associated edge, or the loop closure = 0, it is determined as a complete breakpoint; If the node degree = 1 and there is an associated edge, but the loop closure is less than a specified threshold, it is determined as a virtual welding or a half welding; If the node degree is normal and there is an associated edge, but the spatial position appears to be offset or the loop structure is damaged, it is determined as a mispositioned welding or a missing welding; The rest is determined as normal connectivity; For each candidate welding point with abnormal connectivity, the following contents are output: Defect position: the horizontal coordinate and the vertical coordinate of the spatial barycenter of the welding point in the standard reference plane coordinate system; Breakpoint type: including complete breakpoint, virtual welding or half welding, mispositioned welding or missing welding.

[0016] Optionally, the consistency verification of the judgment result comprises: The pixel domain and the structure domain breakpoint detection results of each image block and the graph structure block are bidirectionally aligned, and if a high-uncertainty area is found, the local structure features of the corresponding image block and the graph structure block are combined to trigger a re-examination prompt.

[0017] The beneficial effects of the present application are:

[0018] The present application organically combines the topological preserving graph embedding and the block sparse attention mechanism in the reinforcement mesh welding breakpoint detection process, through skeletonization and graph modeling of the reinforcement intersection, strip and other structural elements in the image, the topological preserving graph embedding algorithm is used to explicitly maintain the node adjacency relationship, node degree constraint and loop consistency constraint in the low-dimensional space, so that the welding point continuity / discontinuity problem is converted into a structure connectivity and topological closure discrimination problem, and the reference image and the structure graph are divided into natural blocks, combined with the block sparse attention mechanism to realize the block internal fine-grained self-attention and the long-range strip attention along the reinforcement direction, effectively supporting the long-distance continuity judgment across the cells, even if there is strong reflection and shielding site interference, the robust distinction between the real breakpoint and the visual false breakpoint can still be realized, and the structure consistency and the anti-artifact ability of the welding breakpoint detection in the complex scene are significantly improved.

[0019] The application proposes a block sparse attention mechanism under structural prior dynamic guidance, takes the natural block of the reinforcement mesh as the attention unit, dynamically schedules the sparse attention path through the connected confidence map, activates the long-range block pair that is continuous and reliable on the structure, only retains the directional dependence across the cells, realizes the sparse attention calculation of O(N·k) level, effectively breaks through the O(N²) calculation bottleneck of the traditional dense self-attention under large-scale high-resolution images, balances the real-time performance and engineering deployability of the detection algorithm, and provides a technical basis for the reinforcement mesh production line and the intelligent detection of the mobile terminal on site. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings are included to provide a further understanding of the application, and constitute a part of the specification, illustrate the application together with the embodiments of the application, and do not constitute a limitation on the application. In the drawings: Figure 1 A flowchart of a reinforcement mesh welding defect recognition method based on a graph neural network proposed by the application. DETAILED DESCRIPTION

[0021] Embodiment 1

[0022] Reference Figure 1 A reinforcement mesh welding defect recognition method based on a graph neural network, comprising: Collecting large-scale image data of the reinforcement mesh and performing preprocessing to generate standardized reference images; In this embodiment, the generation of the standardized reference images comprises: Using an industrial vision acquisition system to acquire images of the reinforcement mesh to be detected, setting the imaging time and spatial resolution during acquisition, and obtaining a set of original image data; The set of original image data is composed of multiple original images acquired at different imaging times and different spatial resolutions, and each original image includes the pixel value of each pixel point in each channel.

[0023] Performing illumination equalization processing on each original image in the set of original image data to obtain an image after illumination equalization; The illumination equalization processing converts the brightness value of each pixel into an illumination-equalized brightness value after uniform processing through histogram matching and brightness redistribution strategy. The illumination-equalized brightness value is obtained by subtracting the mean value of the original image brightness from the original image brightness, dividing by the standard deviation of the original image brightness, multiplying the standard deviation of the target brightness distribution, and finally adding the mean value of the target brightness distribution. The illumination-equalized brightness value is used to weaken the influence of the difference between shadows and strong light in the reinforcement area.

[0024] Performing reflection suppression processing on the image after illumination equalization to obtain an image after reflection suppression; The reflection suppression processing calculates a reflection response value for each pixel based on color saturation, brightness gradient and local contrast, the reflection response value being the color saturation plus the modulus length of the brightness gradient plus the local contrast, and if the reflection response value exceeds a preset reflection threshold, performing polarization filtering and region smoothing processing on the pixel region, for suppressing the influence of reflection interference on the texture structure of the steel bar image.

[0025] The perspective geometry correction is performed on the image after reflection suppression based on the internal parameter matrix and the external parameter matrix pre-calibrated in the image acquisition process, to obtain a standardized reference image; The perspective geometry correction maps each original image coordinate to a reference coordinate according to the internal parameter matrix and the external parameter matrix, assuming that the depth plane height is constant, to obtain the pixel coordinates under the standard reference plane after correction, and the perspective geometry correction is used to ensure the geometric consistency of the steel bar strip in the image.

[0026] The standardized reference image set is composed of a plurality of reference images in the standard reference plane coordinate system, and each reference image includes the pixel value of each pixel in each channel.

[0027] The reference image is subjected to coarse segmentation processing of the steel bar region to form a welding region candidate set; In the embodiment, the formation of the welding region candidate set includes: For each reference image in the standardized reference image set, a multi-scale convolution feature extraction network is used to calculate the channel-enhanced semantic feature value of each pixel in all semantic feature dimensions, to obtain a channel-enhanced semantic feature map; For each reference image in the standardized reference image set, a multi-scale convolution feature extraction network is used to perform convolution operation on all pixels of the reference image, to extract the spatial neighborhood information of each pixel under different receptive fields, and to complete channel enhancement of each pixel in all semantic feature dimensions, and to combine the channel-enhanced semantic feature values of each pixel in all semantic feature dimensions, to obtain a channel-enhanced semantic feature map.

[0028] Based on the channel-enhanced semantic feature map of each reference image, the probability value of each pixel belonging to the steel bar strip is calculated to obtain a steel bar strip probability map; The probability value of the steel bar strip is obtained by inputting the inner product of the channel-enhanced semantic feature value of each pixel position and the steel bar strip classification weight vector to the Sigmoid activation function after adding a bias, and the probability value of the steel bar strip is used to describe the possibility of each pixel as a steel bar strip region; ; wherein, is the steel bar strip classification weight vector, is the bias, sigmoid activation function, channel augmented semantic feature map.

[0029] A threshold segmentation strategy is adopted for the steel bar strip probability map. The probability value of each pixel of the steel bar strip is compared with a steel bar strip segmentation threshold. If the probability value of the steel bar strip is greater than or equal to the steel bar strip segmentation threshold, the pixel belongs to the steel bar strip region, otherwise it belongs to the non-steel bar strip region, forming a steel bar strip region mask. The steel bar strip region mask is used to identify the spatial distribution range of all steel bar strips in the reference image.

[0030] Geometric skeleton extraction and endpoint analysis are performed on the steel bar strip region mask to obtain a steel bar strip skeleton map. In embodiment 1, the geometric skeleton extraction and endpoint analysis specifically include gradually thinning the steel bar strip region to a single-pixel width to retain the main structure, obtaining a centerline pixel position set of each steel bar strip, performing endpoint analysis on the centerline pixel position set, identifying endpoint pixels and bifurcation point pixels in the centerline pixel position set by detecting the number of neighborhood pixels of each centerline pixel, and all centerline pixels and endpoint and bifurcation point pixel information together constitute a steel bar strip skeleton map. The steel bar strip skeleton map represents the main line structure of the steel bar strip, and simultaneously analyzes the endpoint attribute of each pixel point in the centerline pixel position set.

[0031] Based on the steel bar strip skeleton map, all centerline pixel points intersecting in different directions are detected to obtain a steel intersection candidate set; Each intersection pixel position in the steel intersection candidate set represents the intersection of two centerline pixel sets. Only when the intersection angle of the two centerlines is within the legal intersection angle range, the corresponding intersection pixel position is included in the steel intersection candidate set.

[0032] A neighborhood window of a fixed radius is constructed around each intersection pixel position in the steel intersection candidate set, and a welding region candidate region is extracted in each neighborhood window. All welding region candidate regions form a welding region candidate set.

[0033] The neighborhood window is composed of all pixels with a distance from the intersection pixel position less than or equal to the fixed radius. A welding region candidate region is extracted in each neighborhood window, and the welding region candidate set is used to mark all spatial regions that may contain steel welding points.

[0034] A steel mesh structure map is extracted, the steel intersection is modeled as a graph node, the steel bar strip is modeled as a graph edge, and the welding region candidate set is spatially associated with the nodes and edges to mark suspicious welding point positions, forming a steel mesh structure map. In the embodiment, the steel bar mesh structure graph including the node set, the edge set and the candidate welding point mark is formed, including: Based on the steel bar strip skeleton graph, a set of all steel bar center line pixel positions in each reference image is extracted; Each center line pixel position in the set of steel bar center line pixel positions is used to represent the position of the steel bar trunk after skeleton thinning in the image.

[0035] The node set of the steel bar mesh graph is constructed according to the topological connection relationship in the set of steel bar center line pixel positions; Each node in the node set of the steel bar mesh graph corresponds to the intersection pixel position with the intersection relationship in the steel bar strip skeleton graph, and the construction of the node depends on the intersection angle relationship of the center line pixel in the local neighborhood. If the intersection angle falls within the legal intersection angle range, it is determined as a node.

[0036] If there is a path between any two nodes in the set of center line pixels, which only contains the continuous pixel points between the two nodes and does not pass through other nodes, a steel bar strip segment on the path is represented by an edge. All steel bar strip segments together form an edge set, and each edge in the edge set represents a steel bar strip connecting two intersections; Based on the node set, the edge set and the candidate set of welding regions, the spatial barycenter position of each welding region candidate is calculated; The spatial barycenter position is obtained by summing the horizontal coordinates and vertical coordinates of all pixel points in the welding region candidate region and dividing by the total number of pixel points, respectively. The spatial barycenter position is used to represent the geometric center of the welding region candidate region in the standard reference plane coordinate system.

[0037] All welding region candidate regions that satisfy the spatial correlation condition are uniformly marked as a candidate welding point set according to the spatial barycenter position, and a steel bar mesh structure graph corresponding to each reference image is constructed.

[0038] Each candidate welding point region in the candidate welding point set is a suspicious welding point region spatially covered by a node or an edge. The steel bar mesh structure graph is composed of three parts: the node set, the edge set and the candidate welding point set.

[0039] In the embodiment, the spatial correlation condition includes: If the spatial barycenter position of the welding region candidate region and the Euclidean distance of a certain node are less than or equal to the spatial radius threshold, the corresponding welding region candidate region is associated with the node and recorded as a suspicious welding point mark; If the spatial barycenter position of the welding region candidate region and the distance projection error of a certain edge are less than the spatial radius threshold, the corresponding welding region candidate region is associated with the edge and recorded as a suspicious welding point mark.

[0040] performing a topology-preserving graph embedding operation on the reinforcement mesh structure graph, constructing a low-dimensional embedding space, and outputting a node embedding representation and a connectivity confidence map; In this embodiment, the output node embedding representation and the connectivity confidence map include: The topology-preserving graph embedding operation is performed on the reinforcement mesh structure graph, and based on the node set and the edge set in the reinforcement mesh structure graph, each node is mapped to an embedding vector in a low-dimensional embedding space by using a graph neural network-based embedding method. The local structure features of each node embedding representation are constrained to be consistent with the degree of the actual node in the reinforcement mesh structure graph by a node degree constraint regular term. All node embedding representations within the same loop form a closed or continuous track in the space by a loop structure consistency regular term. After training, each node obtains a unique embedding vector, and the embedding vectors of all nodes collectively form a node embedding representation. In combination with connectivity analysis in the embedding space, a connectivity confidence map of each node and edge is output.

[0041] In the embedding process, the adjacency preserving regular term ensures that the embedding distances of connected nodes in the space are as close as possible. The node adjacency relationship is explicitly maintained in the space, and the node degree constraint and the loop structure consistency are applied to each pair of nodes connected by an edge. The distance between the embedding vectors is forced to be shortened, and the nodes with abnormal degrees are penalized. The nodes that cause the interruption of the loop due to breakpoints or defects are distinguished and constrained.

[0042] The reference image and the reinforcement mesh structure graph are divided into a plurality of image blocks and graph structure blocks; In this embodiment, the reference image and the reinforcement mesh structure graph are divided into a plurality of image blocks and graph structure blocks, including: The standardized reference image is regionally divided, the reference image is divided into a plurality of image blocks, and the spatial boundary range of the image block is extracted. Each image block represents a rectangular sub-region in the reference image corresponding to a certain reinforcement strip area or a unit cell area. The number of image blocks is equal to the number of all rectangular sub-regions obtained by division.

[0043] The spatial boundary range is uniquely determined by the pixel position of the top-left corner pixel coordinate and the bottom-right corner pixel coordinate in the standard reference plane coordinate system.

[0044] The spatial boundary range of each image block is analyzed for spatial intersection with the graph elements in the reinforcement mesh structure graph, and all graph structure blocks intersecting with the image block in space are extracted; Each graph structure block includes a node sub-set with all spatial coordinates falling within the spatial boundary range of the image block, an edge sub-set with both end points within the node sub-set, and a candidate welding point sub-set with all spatial centers of gravity falling within the spatial boundary range of the image block.

[0045] A one-to-one mapping relationship is established between the image blocks and the graph structure blocks, and each image block corresponds to a graph structure block whose spatial boundary range is completely consistent with that of the image block, so as to associate the area covered by each image block in the image domain with the corresponding graph structure block in the graph structure domain.

[0046] A block sparse attention mechanism is applied to the image blocks and the corresponding graph structure blocks to perform feature inference, and a fusion attention feature map including local details and cross-cell continuity features is output. In this embodiment, the fusion attention feature map including local details and cross-cell continuity features includes: A block sparse attention graph structure is constructed for each graph structure block, and the block sparse attention graph structure includes a local intra-block self-attention module and a remote striped attention module. The node feature in each graph structure block is represented as a feature matrix, the number of rows of the feature matrix is the number of nodes in the graph structure block, and the number of columns is the feature dimension of each node.

[0047] A local intra-block self-attention module is applied inside each graph structure block, an attention matrix is established through the relative spatial position relationship between nodes, and a local attention response value between all nodes inside each graph structure block is calculated. In embodiment 1, the spatial position relationship and the similarity of node features between each node and other nodes are calculated, a spatial weight is given according to the relative spatial position relationship between nodes, a weighted inner product operation is performed on the node feature vector and the adjacent node feature vector, the weighted result is normalized to obtain the attention weight between the node pair, and the attention weights between all node pairs form an attention matrix. The local attention response value is obtained by multiplying the node feature vector and the corresponding attention weight one by one and accumulating, which is used to measure the structural correlation between all nodes inside the graph structure block.

[0048] For each graph structure block, the structural connection confidence value in the connected confidence map is used to determine the remote graph structure block that may have a structural connection with the current graph structure block along the reinforcement direction. Each remote graph structure block is determined by the node pair relationship whose connected confidence value exceeds the connected confidence activation threshold.

[0049] A remote striped attention module is applied to the graph structure block pair that meets the remote connection condition, and a remote attention response value is constructed based on the node feature matrix of the current graph structure block and the node feature matrix of the remote graph structure block. In Example 1, the remote striped attention module is applied on the graph structure block pairs meeting the remote connection condition, the spatial direction consistency weight between each pair of nodes is calculated based on the node feature matrix of the current graph structure block and the node feature matrix of the remote graph structure block, the spatial direction consistency weight is obtained by comparing the included angle between the connecting line direction of the current node and the remote node in the spatial coordinate system and the direction of the reinforcing bar main shaft, the smaller the included angle, the higher the weight; the remote attention response value between each pair of nodes is obtained by multiplying the spatial direction consistency weight and the node feature similarity weight, all remote attention response values are combined into a remote attention response matrix, the node feature similarity is obtained by calculating the cosine similarity between the current node feature vector and the remote node feature vector, and the remote attention response matrix is used to measure the directional feature continuity between the current graph structure block and the remote structure block.

[0050] The local attention response value and all remote attention response values are fused to construct a fusion attention feature map. The fusion attention feature map is obtained by assigning a fusion weight to the feature representation obtained under the action of the local attention response value and the feature representation obtained under the action of the remote attention response value for each node, and weighted summing to obtain the final fusion feature representation of each node, and the fusion feature representation of all nodes constitutes the fusion attention feature map, which is used to integrate the local detail information in the current graph structure block and the directional continuity features between the graph structure blocks.

[0051] Based on the fusion attention feature map, the reinforcing mesh structure graph and the candidate welding area set, and combined with the spatial topology and connectivity features of the nodes, edges and candidate welding points in each image block and graph structure block, it is judged whether the connectivity of each candidate welding point is abnormal, the welding breakpoint is located, and the defect position and breakpoint type are output, and the consistency of the judgment result is verified.

[0052] In the embodiment, the welding breakpoint is located and the defect position, breakpoint type and confidence are output, including: The fusion attention feature map and the reinforcing mesh structure graph are extracted, and for each candidate welding point in each candidate welding area set, the node degree, associated edge number, loop closure and cross-graph structure block continuity score of the spatial position are calculated. In Example 1, the node degree is obtained by counting the number of edges connected to the candidate welding point as a node in the reinforcement mesh structure graph, which reflects the number of physical connections of the corresponding position with the surrounding reinforcement strips; the number of associated edges is obtained by counting the total number of edges directly connected to the candidate welding point in the reinforcement mesh structure graph, which determines whether the welding point is penetrated by the reinforcement strip; the loop closure is obtained by detecting whether the smallest loop structure including the candidate welding point is closed, and the loop closure is 1 indicating that the welding point is on a complete loop, and the loop closure is 0 indicating that the loop is interrupted or broken; the cross-graph structure block continuity score is obtained by analyzing the similarity of node features between the graph structure block where the candidate welding point is located and the adjacent graph structure block and the attention response strength along the reinforcement direction, and the higher the continuity score, the better the continuity of the reinforcement strip, and the lower the continuity score, the more likely there is a structural interruption.

[0053] According to the connectivity anomaly determination rule, it is judged whether the connectivity of each candidate welding point is abnormal, wherein: If the node degree = 0 and there is no associated edge, or the loop closure = 0, it is determined to be a complete breakpoint; If the node degree = 1 and there is an associated edge, but the loop closure < a specified threshold, it is determined to be a virtual weld or a half weld; If the node degree is normal and there is an associated edge, but the spatial position is offset or the loop structure is damaged, it is determined to be a mispositioned weld or a missed weld; In other cases, it is determined to be normal connectivity; For each candidate welding point with abnormal connectivity, the following content is output: Defect position: the horizontal and vertical coordinates of the spatial center of gravity of the welding point in the standard reference plane coordinate system; Breakpoint type: including complete breakpoint, virtual weld or half weld, mispositioned weld or missed weld.

[0054] In this embodiment, consistency verification is performed on the judgment result, including: Bidirectional alignment is performed on the pixel domain and structure domain breakpoint detection results of each image block and graph structure block, and if a high-uncertainty region is found, the local structure features of the corresponding image block and graph structure block are combined to trigger a re-examination prompt.

[0055] Example 2:

[0056] In the production process of reinforcement mesh, an automatic detection system on the production line takes a picture of each mesh, and the resolution of a single image is 10000x5000 pixels. Within one day, the production line accumulatively collects 1800 reinforcement mesh images. The system selects a batch of 200 sample meshes for breakpoint detection experiments.

[0057] In a conventional production line environment, the system automatically completes the image illumination equalization and reflection suppression in a certain batch of images. Due to different angles of light irradiation on the construction site, strong reflection spots appear on the surface of the steel bars, and some of the welding point areas are even partially covered by steel dust and welding slag.

[0058] After the detection process is started, the system performs steel bar region segmentation on each standardized reference image, extracts strips and intersection points. In example 2, in the image numbered “1001234”, 59 steel bar strips and 367 intersection points are automatically extracted. In the skeleton analysis link, the system automatically discovers 4 strip fracture structures, 2 of which are exactly near the intersection of two steel bars, which are preliminarily marked as candidate welding breakpoints.

[0059] In the further structure modeling process, the system establishes a node set (intersection) and an edge set (strip) according to the skeleton relationship, and associates the 34 automatically segmented welding area candidate blocks with the nodes and edges through spatial center of gravity analysis. At this time, the mesh structure diagram numbered “1001234” has been completely established by the system.

[0060] Subsequently, the system divides the image into 12 steel bar strip blocks and 30 cell blocks, and each image block is mapped to a structure block. In example 2, the 5th image block corresponds to the structure subgraph of the 45th horizontal steel bar region and the 36th vertical steel bar region of the mesh. Attention weight relationships are established between each block and the remote blocks along the steel bar direction.

[0061] In the block sparse attention reasoning stage, the system detects that the continuity score of the 9th block has decreased significantly, the local self-attention response has weakened, and the cross-block long-range strip attention chain in this area has also been broken. The node degree of a candidate welding point in this block becomes 0, and the loop closure is 0. The system determines that it is a complete breakpoint, with a breakpoint confidence of 0.97. The output detection result is: breakpoint position (pixel coordinates 4211, 2347), type “complete breakpoint”, confidence 0.97.

[0062] Another sample numbered “1001251” has a welding point covered by mortar in the lower right corner cell area, which is not obvious in the pixel domain, but the self-attention chain in the structure block is not broken, the cross-block continuity score is 0.81, the node degree is 1, and the loop closure is 0.2. According to the connectivity anomaly rule, the system determines that it is a “virtual or half-welding” breakpoint, with a breakpoint confidence of 0.78. The output detection result is: breakpoint position (pixel coordinates 9534, 4860), type “virtual or half-welding”, confidence 0.78.

[0063] On the mesh of test sample No. "1001276", there is a welding spot where all domain indicators are normal, but the spatial center of gravity and the node have obvious pixel deviation and the corresponding loop structure is partially blocked, the system analyzes the cross-block continuity score as 0.64, the node degree is 2, the number of associated edges is 2, but the spatial center of gravity deviation of the welding spot reaches 36 pixels. According to the judgment rule, the system judges it as "misplaced welding or missed welding", the breakpoint confidence is 0.69, and the output detection result is: breakpoint position (pixel coordinates 1547, 2992), type "misplaced welding or missed welding", confidence 0.69.

[0064] All detection results automatically generate a defect distribution report, and the MES system of the production line receives the welding breakpoint coordinates, type and confidence of each mesh, and when the breakpoint confidence is lower than 0.8, it is automatically marked as a high-uncertainty sample, triggering the re-inspection process.

[0065] Comparison experiment with traditional method: For the same batch of 200 sample meshes, 364 real breakpoints are manually labeled. The method of the application detects 358 breakpoints in total, with 2 false positives and 6 missed detections. The traditional method based on pixel segmentation + edge detection detects 329 breakpoints, with 11 false positives and 35 missed detections. The statistical data is shown in Table 1 below: Table 1 Comparison experimental data of the method of the application and the traditional method Method Detection accuracy False detection rate Missed detection rate Average detection time per sheet (seconds) Average positioning error (pixels) The method of the invention 98.3% 0.6% 1.7% 1.2 5.1 Traditional segmentation + edge detection method 90.3% 3.0% 9.6% 3.6 13.7 In 20 difficult example samples with complex strong light, shielding and rust pollution, the breakpoint recall rate of the method of the application is 95.5%, while that of the traditional method is only 78.4%.

[0066] In Example 2, in the night shift sample No. "1001299", 3 real breakpoints are manually labeled, which are successfully detected by the application, while the traditional method only detects 1 and falsely reports 2 welding spots as breakpoints. For example, sample No. "1001244" is misjudged due to fixture shielding, the application completes the continuity information through the cross-block attention mechanism and does not produce false positives, while the traditional method produces false positives.

[0067] The above is only a preferred specific embodiment of the application, but the protection scope of the application is not limited thereto, and any person skilled in the art can make equivalent substitutions or changes within the technical scope disclosed by the application and according to the technical solutions and inventive concepts of the application, which should be covered within the protection scope of the application.

Claims

1. A method for identifying welding defects in reinforcing mesh based on graph neural networks, characterized in that, include: Collect large-format image data of the steel mesh and preprocess it to generate standardized reference images; The reference image is coarsely segmented to form a candidate set of welding areas; Extract the steel mesh structure diagram, model the steel bar intersections as graph nodes and the steel bar strips as graph edges, and spatially associate the candidate set of welding areas with nodes and edges, mark the positions of suspicious welding points, and form the steel mesh structure diagram. Perform a topology-preserving graph embedding operation on the steel mesh structure diagram to construct a low-dimensional embedding space and output the node embedding representation and connectivity confidence graph. The reference image and the steel mesh structure diagram are divided into several image blocks and diagram structure blocks. A block sparse attention mechanism is applied to image patches and their corresponding graph structure blocks for feature inference, and the output is a fused attention feature map that includes local details and cross-cell continuity features. Based on the fusion of attention feature map, steel mesh structure map and welding area candidate set, and combined with the spatial topology and connectivity features of nodes, edges and candidate welding points in each image block and graph structure block, the system determines whether the connectivity of each candidate welding point is abnormal, locates the welding breakpoint and outputs the defect location and breakpoint type, and verifies the consistency of the judgment results.

2. The method for identifying welding defects in reinforcing mesh based on graph neural networks according to claim 1, characterized in that, The generation of the standardized reference image includes: An industrial vision acquisition system is used to acquire images of the steel mesh to be inspected. During acquisition, the imaging time and spatial resolution are set to obtain the original image data set. Perform illumination equalization processing on each original image in the original image dataset to obtain an illumination equalized image; Perform reflection suppression processing on the image after illumination equalization to obtain the image after reflection suppression; The image after reflection suppression is subjected to perspective geometric correction based on the intrinsic and extrinsic parameter matrices pre-calibrated during image acquisition to obtain a standardized reference image; Perspective geometric correction maps each original image coordinate to reference coordinates based on intrinsic and extrinsic parameter matrices. Assuming the depth plane height is constant, the correction yields pixel coordinates under the standard reference plane. Perspective geometric correction is used to ensure the geometric consistency of the reinforcing bar strips in the image.

3. The method for identifying welding defects in reinforcing mesh based on graph neural networks according to claim 2, characterized in that, The formation of the candidate set of welding areas includes: For each reference image in the standardized reference image set, a multi-scale convolutional feature extraction network is used to calculate the channel-enhanced semantic feature value of each pixel in all semantic feature dimensions to obtain a channel-enhanced semantic feature map. Based on the semantic feature map enhanced by each reference image channel, the probability value of each pixel belonging to the steel strip is calculated to obtain the steel strip probability map; A threshold segmentation strategy is adopted for the probability map of steel strips. The probability value of the steel strip of each pixel is compared with the steel strip segmentation threshold. If the probability value of the steel strip is greater than or equal to the steel strip segmentation threshold, the pixel belongs to the steel strip region; otherwise, it is a non-steel strip region, thus forming a steel strip region mask. Geometric skeleton extraction and endpoint analysis are performed on the mask of the rebar strip area to obtain the rebar strip skeleton map; Based on the steel strip skeleton map, all centerline pixels that intersect in different directions are detected to obtain a candidate set of steel strip intersections; Using the pixel position of each intersection point in the candidate set of rebar intersections as the center, a neighborhood window with a fixed radius is constructed. Within each neighborhood window, candidate welding areas are extracted, and all candidate welding areas are combined into a candidate welding area set.

4. The method for identifying welding defects in reinforcing mesh based on graph neural networks according to claim 3, characterized in that, The process of forming a steel mesh structure diagram that includes a set of nodes, a set of edges, and candidate weld point markers includes: Based on the steel bar skeleton map, extract the set of pixel positions of all steel bar center lines in each reference image; The node set of the rebar mesh is constructed based on the topological connection relationship in the pixel position set of the rebar centerline; If there exists a path in the centerline pixel set between any two nodes that contains only the continuous pixels between the two nodes and does not pass through other nodes, then an edge is constructed to represent the steel strip segment on the path. All steel strip segments together form the edge set, and each edge in the edge set represents the steel strip connecting two intersection points. Based on the node set, edge set, and welding region candidate set, calculate the spatial centroid position for each welding region candidate region; Based on the location of the spatial centroid, all candidate welding areas that meet the spatial correlation conditions are uniformly marked as a set of candidate welding points, and a steel mesh structure diagram corresponding to each reference image is constructed.

5. The method for identifying welding defects in reinforcing mesh based on graph neural networks according to claim 1, characterized in that, The spatial association conditions include: If the Euclidean distance between the spatial centroid of a candidate welding area and a node is less than or equal to the spatial radius threshold, the corresponding candidate welding area is associated with the node and recorded as a suspicious welding point. If the distance projection error between the spatial centroid of a candidate welding area and a certain edge is less than the spatial radius threshold, the corresponding candidate welding area is associated with the edge and recorded as a suspicious welding point.

6. The method for identifying welding defects in reinforcing mesh based on graph neural networks according to claim 5, characterized in that, The output node embedding representation and connectivity confidence graph include: A topology-preserving graph embedding method is performed on the rebar mesh structure diagram. Based on the set of nodes and edges in the rebar mesh structure diagram, an embedding method based on graph neural networks is used to map each node to an embedding vector in a low-dimensional embedding space. The local structural features of each node's embedding representation are constrained to be consistent with the degree of the actual node in the rebar mesh structure diagram by a node degree constraint regularization term. The loop structure consistency regularization term is constrained to ensure that the embedding representations of all nodes in the same loop form a closed or continuous trajectory in space. After training, each node obtains a unique embedding vector. The embedding vectors of all nodes together form the node embedding representation. Combined with connectivity analysis in the embedding space, the connectivity confidence graph of each node and edge is output.

7. The method for identifying welding defects in reinforcing mesh based on graph neural networks according to claim 6, characterized in that, The process of dividing the reference image and the steel mesh structure diagram into several image blocks and structure blocks includes: The standardized reference image is divided into several image blocks, and the spatial boundary range of the image blocks is extracted. Perform spatial intersection analysis between the spatial boundary of each image block and the graph elements in the steel mesh structure diagram, and extract all graph structure blocks that intersect with the image block in space. A one-to-one mapping relationship is established between image blocks and graph structure blocks. Each image block corresponds to a graph structure block whose spatial boundary range is completely consistent with it. This is used to associate the area covered by each image block in the image domain with the corresponding graph structure block in the graph structure domain.

8. The method for identifying welding defects in reinforcing mesh based on graph neural networks according to claim 7, characterized in that, The output includes a fused attention feature map of local details and cross-cell continuity features, including: For each graph structure block, construct a block sparse attention graph structure, which includes local intra-block self-attention modules and remote stripe attention modules; Within each graph block, a local self-attention module is applied. An attention matrix is ​​established based on the relative spatial relationships between nodes, and the local attention response values ​​between all nodes within each graph block are calculated. For each graph structure block, determine the current graph structure block and its remote graph structure blocks that may have structural connections along the reinforcement direction based on the structural connection confidence value in the connectivity confidence graph. A remote striped attention module is applied to graph block pairs that meet the remote connection conditions. Based on the node feature matrix of the current graph block and the node feature matrix of the remote graph block, a remote attention response value is constructed. The local attention response values ​​are fused with all remote attention response values ​​to construct a fused attention feature map; The fusion attention feature map assigns fusion weights to the feature representations obtained by each node under the local attention response value and the feature representations obtained under the remote attention response value, and then sums them up to obtain the final fusion feature representation of each node. The fusion feature representations of all nodes form the fusion attention feature map, which is used to integrate the local details in the current graph structure block with the directional continuity features between cross graph structure blocks.

9. The method for identifying welding defects in reinforcing mesh based on graph neural networks according to claim 7, characterized in that, The method for locating weld breakpoints and outputting defect location, breakpoint type, and confidence level includes: Extract the fusion attention feature map and the steel mesh structure map, and for each welding area candidate set, calculate the node degree, number of associated edges, loop closure and cross-graph structural block continuity score of the candidate welding points in the spatial location; Based on the connectivity anomaly determination rules, determine whether the connectivity of each candidate welding point is abnormal, where: If a node has a degree of 0 and no associated edges, or a loop closure of 0, it is considered a complete break. If a node has a degree of 1 and an associated edge, but its loop closure is less than the specified threshold, it is determined to be a cold solder joint or a half solder joint. If the node degree is normal and there are associated edges, but the spatial position is offset or the loop structure is damaged, it is judged as misaligned welding or missing welding. All other cases are considered to have normal connectivity; For each candidate weld point with connectivity anomalies, output the following: Defect location: The horizontal and vertical coordinates of the spatial centroid of the weld point in the standard reference plane coordinate system; Breakpoint types include complete breakpoints, cold or partial welds, misaligned welds, and missing welds.

10. The method for identifying welding defects in reinforcing mesh based on graph neural networks according to claim 7, characterized in that, The consistency verification of the judgment result includes: The pixel domain and structural domain breakpoint detection results of each image block and graph structure block are bidirectionally aligned. If a high uncertainty region is found, a re-inspection prompt is triggered by combining the local structural features of the corresponding image block and graph structure block.

Images