Fabricated T-beam upper wing plate connecting steel bar quality intelligent detection method

Abstract

The invention discloses an intelligent detection method for the quality of a connecting steel bar of an upper wing plate of a fabricated T-beam, a segmentation model adopted by the detection method adopts a VIM model as an image encoder, and the image processing efficiency is improved. And then a prompt encoder is utilized to generate a target token, a prompt token and a high-quality target token, so that the segmentation precision and details are remarkably improved, and the zero sample generalization ability of the SAM is kept. Besides, the segmentation model adopts a mask decoder formed by a plurality of CBAM attention mechanism layers, a plurality of KAN networks and convolution operation, so that the calculation complexity and the calculation amount are reduced, but the network can better capture key information in the image, and the efficiency and the accuracy of the network are improved. Besides, according to the detection method, a semantic segmentation algorithm is applied to intelligent quality detection of the fabricated T-beam upper wing plate connecting steel bar, the function of fine recognition of image information is achieved, steel bar quality recognition can be quantified, construction efficiency and quality are improved, and dependence of manual monitoring is reduced.

Description

Technical Field

[0001] The present invention relates to the technical fields of image semantic segmentation and intelligent construction, and in particular to an intelligent detection method for the quality of steel bars connected to wing plates on assembled T-beams. Background Art

[0002] Prefabricated T-beam bridge is a type of bridge structure widely used in medium and small span beam bridges. Its superstructure consists of main beam, transverse beam (plate), bridge deck structure and other parts. The main beam is composed of beam ribs and upper wing plates. The upper wing plates not only constitute part of the main beam, but also form the bridge deck together with the beam ribs to bear the load together. Compared with integral T-beams, prefabricated T-beam bridges have the advantages of easy construction quality assurance, factory-based and standardized construction. At present, they have gradually replaced integral T-beams and become the mainstream choice in the design of new bridges.

[0003] In terms of steel bar structure, the main beam ribs and upper wing plates of the T-beam need to be equipped with different types of steel bars, including longitudinal force-bearing main bars, erection bars, inclined bars, stirrups, distribution bars, etc., to ensure the stability and bearing capacity of the structure. The connection structure is an important part of the prefabricated T-beam bridge, and its connection method must have sufficient strength and rigidity to ensure the integrity and durability of the structure. The connection part is the concentrated force transmission part of the overall structure. Insufficient steel bar length in this part may result in the failure of the connection to effectively transfer stress, resulting in failure of the connection between the T-beams, and even the breakage or falling of the T-beam. Therefore, it is crucial that the length of the steel bars at the connection part of the T-beam meet the design requirements and need to be accurately measured.

[0004] At present, the connection steel bar measurement of T-beams is done manually on site. The installation height of T-beams may reach tens or even hundreds of meters. Manual measurement requires workers to work at high altitudes, which increases the risk of falling. It is a high-altitude dangerous operation with high risk. In addition, there may be unstable factors at the construction site, such as unstable brackets, sliding objects, etc., which may pose a threat to workers working at high altitudes. Moreover, when measuring steel bars at high altitudes, the workers' line of sight may be limited, making it difficult to accurately observe and measure the position and size of the steel bars, and real-time monitoring cannot be achieved. It is a post-operation.

[0005] Image segmentation technology is widely used in computer vision, natural language processing and other fields. Applying it to the detection of the quality of the steel bars connected to the wing plates on T-beams is an ideal intelligent detection method that can reduce safety risks. It can realize quality inspection during the construction process. However, although the traditional quality inspection network can accurately locate the uploaded image, the accuracy of the positioning result is low, and the guiding significance for the actual construction process in the project is limited. Therefore, how to use high-precision semantic segmentation algorithms to monitor and measure the quality of the steel bars connected to the wing plates of prefabricated T-beams is an urgent need in the field. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the above technologies and provide an intelligent detection method for the quality of steel bars connected to the flange of a prefabricated T-beam. Compared with the prior art, the present invention creatively uses image segmentation technology to achieve the connection of the flange of the prefabricated T-beam, which can accurately and efficiently detect the quality of the steel bars in the connection process, providing an effective means for ensuring the quality of the connected steel bars.

[0007] The technical solution of the present invention to solve the technical problem is to design an intelligent detection method for the quality of steel bars connected to the flange of an assembled T-beam, characterized in that the detection method comprises the following steps: Step 1: Obtain no less than 1,000 images of the steel bars connected to the flange plates of prefabricated T-beams, first pre-process them, then label each image, and then divide them according to the number of images, with no less than 60% of them divided into training sets and the rest divided into test sets; Step 2: Construct an efficient network model for segmenting steel bars on the flange of prefabricated T-beams The efficient segmentation network model is an improved SAM network model, which mainly includes three backbone modules, namely, an image encoder module, a prompt encoder module, and a mask decoder module; The improved SAM network model uses the VIM model as an image encoder to extract the feature representation of the input image; the specific workflow of the VIM model is as follows: First, the input image F is divided into multiple non-overlapping image blocks of fixed size, and then the position information of each image block is embedded into its corresponding pixel vector representation information, and the pixel vector representation information of each image block is flattened and arranged into a sequence to obtain the original pixel sequence information and the corresponding position information of each image block; the original pixel sequence information of each image block is linearly transformed through a linear layer to adjust its data dimension, and the combined embedding features of the primary pixel sequence information and the corresponding position information of each image block are obtained; then, the primary pixel sequence information of the image block is normalized through a normalization layer, and batch normalization is used; thereafter, the results of the primary pixel sequence information Y of the image block after the normalization layer are processed using two different projection layers, and are projected into a sequence X and a sequence Z of dimension E respectively; Then, the sequence X is processed in both forward and backward directions to capture the local dependencies in the vector; for forward processing, the sequence X is firstly processed by applying a one-dimensional forward convolution operation and a SILU function to obtain , and then Linear projection is applied to three different linear layers to obtain the sequences , and , where the sequence Use the soft-add activation function to ensure that it is positive; then the sequence Apply the Hadamard product to obtain the transformation parameters and , that is, the sequence with sequence Perform Hadamard product to get parameters ,sequence The elements in are trainable parameters, the sequence with sequence Perform Hadamard product to get parameters ; Last used , , and input Through the spatial state model, the forward processing result of sequence X is calculated ; use , , and input The forward processing result of sequence X is calculated by the spatial state model The specific process is expressed by the formula: h(X)=A+B×X = ×h(X)+ × = × In the above formula, A and B are trainable parameter matrices, and X is the sequence X; The result of backward processing of sequence X The obtained process is the same as the forward processing result of sequence X The process of obtaining is similar, except that the one-dimensional forward convolution operation in the forward processing is replaced by a one-dimensional backward convolution operation; The sequence X will be , The results obtained after the activation function processing of sequence Z are added separately, and then the two results are added together. The obtained results are processed by the projection layer and combined with the primary pixel sequence information of the image block. Perform residual connection and then generate sequence ,sequence It will be sent to MLP for processing to obtain the coded pixel sequence information; the obtained coded pixel sequence information is combined with the position information of the corresponding image block to obtain the embedded feature information of the image block; the embedded feature information of the image block is the output of the image encoder; The prompt encoder module is used to convert the input prompt into a vector sequence that the model can understand; the prompt encoder module inputs a mask prompt, which is a label image of image F; the prompt encoder module processes the label image through three different convolutional layers, two of which output a target token and a prompt token respectively; at the same time, the output result of the third convolutional layer of the prompt encoder module is weighted to output a high-quality target token; the weight is a learnable parameter; the outputs of the three convolutional layers are all one-dimensional vectors with the same number of elements; The specific workflow of the mask decoder module is as follows: First, the target token, prompt token, and high-quality target token output by the prompt encoder module are used as Query, Key, and Value to combine into a three-dimensional matrix of original features F0, which is used as the input of the first CBAM attention mechanism layer. The first CBAM attention mechanism layer outputs a one-dimensional first output information F1; Then, the first output information F1 is input as Query to the second CBAM attention mechanism layer. At the same time, the encoded pixel sequence information in the embedded feature information of the image block output by the image encoder module is input as Key and Value to the second CBAM attention mechanism layer. The above Query, Key and Value are combined into a three-dimensional matrix feature information. The feature information is processed by the second CBAM attention mechanism layer to obtain a one-dimensional second output information F2. The second output information F2 is input into the first KAN network, and after being processed by the first KAN network, the third output information F3 is obtained; wherein the part of the third output information F3 corresponding to the first output information F1 is a mask prediction token related to the image segmentation task, and the part corresponding to the encoded pixel sequence information in the embedded feature information of the image block is a specific image feature representation for predicting the segmentation mask; Then, the specific image feature representation in the third output information F3 is input as a query to the third CBAM attention mechanism layer, and the encoded pixel sequence information in the embedded feature information of the image block output by the image encoder module is input as the key and value to the third CBAM attention mechanism layer. After processing and updating the information, a one-dimensional fourth output information F4 is obtained; The fourth output information F4 is used as the key and value, and the mask prediction token in the third output information F3 is used as the query, and is combined into a three-dimensional matrix and input into the fourth CBAM attention mechanism layer. After processing, the mask of each target point is output to obtain the one-dimensional fifth output information F5; At the same time, the fourth output information F4 undergoes two convolution operations to obtain an output feature F6 with the same size as the original image F; The part of the fifth output information F5 corresponding to the mask prediction token is input to the second KAN network, and a binary image with the probability of each predicted target mask is output; the binary image is multiplied by the output feature F6 to obtain the final segmentation image, which is also a binary image; the mask decoder finally outputs the segmentation image of the original image F; The first CBAM attention mechanism layer, the second CBAM attention mechanism layer, the third CBAM attention mechanism layer, and the fourth CBAM attention mechanism layer have the same basic structure, but the parameters are not shared; The first KAN network and the second KAN network have the same basic structure, but the parameters are not shared; the first KAN network is a structure including two activation function layers, wherein the input of the first layer is two identical features, and the input of the second layer is five; the two inputs of the first layer are processed using the ten activation functions of the first layer, and each input obtains five clones; the first clone of the first input and the first clone of the second input are added together, and the result is used as the first input of the second layer, and the remaining four inputs of the second layer are obtained in the same manner; the five inputs of the second layer are processed by the five activation functions of the second layer, and then the five results are added together to obtain the output of the first KAN network; the formula of each activation function in the two activation function layers is the same, but the parameters are not shared; The two second output information F2 are used as the input of the first KAN network, and the input of the second KAN network is analogous to that; Step 3: Train the network model for efficient segmentation of prefabricated T-beam connection steel bars in step 2 First, the trainable parameters of the efficient segmentation network model in step 2 are randomly initialized, and the fixed parameters of the network model are set. Then, an image in the training set and the corresponding label image are input into the initialized efficient segmentation network model. The efficient segmentation network model outputs a segmented image of a binary image. Then, combined with the corresponding label image, the mean square error loss function is used to calculate the pixel-level training loss, and then the trainable parameters of the efficient segmentation network model are updated once through the back propagation of the training loss. All images are iteratively trained in turn. After training all images in the training set, a round of training is completed. The trainable parameters of the network that completed the previous round are used as the initial trainable parameters of the next round. When the training gradient value is within ±1e －５When the training iterations reach 30,000, stop training and extract the trainable parameters with the smallest training loss to obtain an efficient segmentation network model that has completed training. Then, the images in the test set and the corresponding annotated images are input into the trained efficient segmentation network model, and the accuracy of the segmentation results is calculated. When the pixel accuracy is 95%-100%, it is proved that the trained efficient segmentation network model is the optimal segmentation model. Otherwise, the initialization method of the trainable parameters of the efficient segmentation network model is adjusted, and the values ​​of the fixed parameters of the network model are adjusted. The training process and the testing process of the efficient segmentation network model are repeated until the optimal segmentation model is obtained. Step 4: Inspection of the connecting steel bars of the upper flange of the assembled T-beam Use a drone to take an image of the steel bars connected to the wing plate of the prefabricated T-beam to be inspected, and then input it into the optimal segmentation network model in step three. The network model outputs a segmented image of the binary image of the image; the lengths of various pixel points are counted, and the white pixels with a length greater than 2 are the pixels of the steel bars; when the length of the steel bars is detected to be between 580 and 620 pixels, it is qualified. If the length is not in this range, the steel bars connected to the wing plate of the prefabricated T-beam at this location are judged to be abnormal, thereby realizing the quality detection of the steel bars connected to the wing plate of the prefabricated T-beam.

[0008] Compared with the prior art, the beneficial effects of the present invention are as follows: the segmentation model of the present invention adopts the VIM (vision mamba) model as an image encoder to improve the efficiency of image processing. The design of VIM allows it to be more easily extended to higher resolution images, which is particularly important for the analysis of large-scale visual data sets. Then, the prompt encoder is used to generate target tokens (output tokens) and prompt tokens (Prompt tokens) and high-quality target tokens (HQ-Output-tokens), which significantly improves the accuracy and details of the segmentation, maintains the zero-sample generalization ability of SAM, enables it to effectively segment unseen data, and helps the model to generate more robust segmentation results when facing challenging scenes (such as occlusion, illumination changes, etc.). In addition, the segmentation model adopts a mask decoder composed of multiple CBAM attention mechanism layers, multiple KAN networks and convolution operations, which reduces the computational complexity and amount of calculation, avoids complex attention weight calculations and high-dimensional matrix multiplication operations, but enables the network to better capture key information in the image and improve the efficiency and accuracy of the network.

[0009] The detection method of the present invention applies the semantic segmentation algorithm to the intelligent detection of the quality of the steel bars connected to the wing plates of the assembled T-beams. It has the function of finely identifying image information and can quantify the steel bar quality identification. It can overcome the shortcomings of the quality detection of the steel bars connected to the wing plates of the assembled T-beams, improve construction efficiency and quality, reduce the reliance on manual monitoring, and provide a more reliable and safe solution for the quality detection of the steel bars connected to the wing plates of the assembled T-beams. The technical solution of the present invention provides the possibility of realizing the quality detection of the steel bars connected to the wing plates of the assembled T-beams at a larger scale at the building level. It has great significance and broad application prospects. It also provides strong support and guarantee for the development of the wing plate connection technology of the assembled T-beams. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a schematic diagram of the structure and principle of an efficient segmentation network model of an embodiment of an intelligent detection method for the quality of steel bars connected to the flange plate of an assembled T-beam of the present invention.

[0011] Figure 2 It is a schematic diagram of the structure and principle of a VIM model of an efficient segmentation network model of an embodiment of an intelligent detection method for the quality of steel bars connected to the flange plate of an assembled T-beam of the present invention.

[0012] Figure 3 It is a schematic diagram of the structure and principle of the CBAM attention mechanism layer of the mask decoder module of the efficient segmentation network model of an embodiment of the intelligent detection method for the quality of the wing plate connection steel bars of the assembled T-beam of the present invention.

[0013] Figure 4 It is a schematic diagram of the structure and principle of a KAN network of a mask decoder module of an efficient segmentation network model of an embodiment of an intelligent detection method for the quality of steel bars connected to the flange plate of an assembled T-beam of the present invention.

[0014] Figure 5 It is a schematic diagram of the operation flow of an embodiment of an intelligent detection method for the quality of steel bars connected to the flange plate of an assembled T-beam of the present invention. DETAILED DESCRIPTION

[0015] In order to more clearly describe the technical problems, technical solutions and advantages of the present invention, the following will be described in detail with reference to the drawings and embodiments. It should be noted that these embodiments are only used to illustrate the principles and application scope of the present invention and should not be regarded as limiting the present invention.

[0016] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0017] The present invention provides an intelligent detection method for the quality of steel bars connected to the upper flange of an assembled T-beam, the detection method comprising the following steps: Step 1: Obtain no less than 1,000 images of the steel bars connected to the flange plates of prefabricated T-beams, first pre-process them, then label each image, and then divide them according to the number of images, with no less than 60% of them divided into training sets and the rest divided into test sets; Use a drone to collect images (side shot, able to capture the entire assembled T-beam upper wing plate connecting steel bars) with a resolution of 1280×960 pixels to ensure image clarity. During image acquisition, ensure that the drone is not affected by light and wind to avoid blur and ensure that the captured images are clear enough. From the collected images, select an image that contains the entire assembled T-beam upper wing plate connecting steel bars and is in the middle of the image to create the training set and test set.

[0018] In this embodiment, the number of images of the wing plate connecting steel bars of the assembled T-beam is 2000, of which 1500 are collected by drones, and 500 are conventional images of the wing plate connecting steel bars of the assembled T-beam from various angles that are publicly available on the Internet. The conventional images of the wing plate connecting steel bars of the assembled T-beam are added to improve the accuracy and robustness of the training, so that the model can more accurately identify the images of the wing plate connecting steel bars of the assembled T-beam being tested.

[0019] The preprocessing method specifically uses the MSR algorithm to perform the following operations on the image: Step 1.1: Perform log transformation (i.e. logarithmic transformation) on the image to obtain a log image; Step 1.2: Gaussian blur the log image (i.e., obtain the low-frequency component) to obtain the blurred log image; Step 1.3: Difference the original image and the blurred log image to get the image.

[0020] Perform steps 1.1, 1.2, and 1.3 on the original image in the three dimensions of R, G, and B (i.e., three color channels), respectively. The difference is that when performing the Gaussian blur in step 1.2, the sigmoid parameters are set to 15, 80, and 200, respectively; perform steps 1.1, 1.2, and 1.3 on each dimension of an image, then weightedly average the results of the three steps, and finally merge the three dimensions and scale the data to 0-255 to obtain the preprocessed image R. The data set R* is obtained from all the preprocessed images.

[0021] The method for labeling each image is as follows: using the binary segmentation mask of the steel bars connected to the flange of the prefabricated T-beam, the area to be identified in the preprocessed image is annotated at the pixel level using the Photoshop tool, and each pixel is assigned a corresponding semantic label (whether it is a steel bar or not), generating a 712×712 binary image label; in the label, the black pixel value is 0, representing the background, and the white pixel value is 255, where the width is 1 to 2 (the width of a single line) pixels is the white annotation line, and the length is 580 to 620 pixels representing the target steel bar.

[0022] Step 2: Construct an efficient network model for segmenting steel bars on the flange of prefabricated T-beams The efficient segmentation network model is an improved SAM network model. The overall network architecture of the improved SAM network model is as follows: Figure 1 The improved SAM network model mainly includes three main modules, namely, an image encoder module, a prompt encoder module, and a mask decoder module.

[0023] The specific process of the improved SAM network model is as follows: First, the input image F enters the image encoder module. The image encoder module uses the VIM model to divide the input image into blocks and extract feature vectors for each block. The image encoder finally outputs an image embedding that captures the high-level representation of all important visual features in the image. This image embedding will be used by the subsequent prompt encoder module and mask decoder module. Then, the prompt encoder module converts the input prompt into a vector sequence that the model can understand. The input is the mask prompt, and the prompt encoder module uses convolution as the encoder output vector sequence. The prompt encoder module finally outputs the target tokens (output tokens), prompt tokens, and high-quality target tokens (HQ-Output-tokens). Finally, the mask decoder module combines the image embedding output by the image encoder and the target tokens (output tokens), prompt tokens, and high-quality target tokens (HQ-Output-tokens) output by the prompt encoder into a three-dimensional matrix as input. Through the CBAM attention mechanism, KAN network, and convolution operations, a mask map and IOU scores segmented according to the prompts are finally output, and the final segmented image is obtained from the two.

[0024] The improved SAM network model uses the VIM (vision mamba) model as an image encoder to extract the feature representation of the input image. The VIM (vision mamba) model structure is as follows: Figure 2 As shown, the specific workflow is as follows: First, the input image F is divided into multiple non-overlapping image patches of fixed size, and then the position information of each image patch is embedded into its corresponding pixel vector representation information, and the pixel vector representation information of each image patch is flattened and arranged into a sequence to obtain the original pixel sequence information and the corresponding position information of each image patch. The original pixel sequence information of each image patch is linearly transformed through a linear layer to adjust its data dimension to obtain the combined embedding features of the primary pixel sequence information and the corresponding position information of each image patch. The above linear transformation is defined by a weight matrix and a bias term, where the input vector is multiplied by the weight matrix and then the bias term is added to obtain the output of the linear transformation. The linear transformation can be expressed as Y=TW Y +b Y, where T is the original pixel sequence information of the image block, W Y is the weight matrix, b Y is the bias vector, and Y is the primary pixel sequence information obtained after transformation.

[0025] Then, the primary pixel sequence information of the image block is normalized through the normalization layer, and batch normalization is used. After that, the primary pixel sequence information Y (B, M, D) of the image block is processed by the normalization layer and the results are processed by two different projection layers, and are projected into sequences X (B, M, E) and Z (B, M, E) with dimensions E (extended state dimension, set to 384), where B is the batch size, M is the sequence length, and D is the original feature dimension.

[0026] The two different projection layers are two different linear layers (or fully connected layers). The linear layers perform linear transformation operations. The two linear layers project the input sequences to generate sequences X and Z. The linear layer consists of a weight matrix and a bias term. For sequences X and Z, there are weight matrices Wx and Wz, and bias vectors bx and bz, respectively. Multiply the input sequence with the weight matrix Wx and add the bias bx to obtain the sequence X, X=Wx×Input+bx, Input is the input sequence, Wx is a D×E matrix, E is the extended state dimension, N is the hidden state dimension, and D is the original feature dimension. The method for obtaining the sequence Z is similar. The dimensions of the weight matrix and bias vector of the corresponding linear layer are the same as those of the sequence X, but the element values ​​are different.

[0027] Then, the sequence X is processed in both forward and backward directions to capture local dependencies in the vector. For the forward processing, the sequence X is firstly processed by applying a one-dimensional forward convolution operation and a SILU function to obtain , and then Linear projection is applied to three different linear layers to obtain the sequences (B, M, N), (B, M, N) and (B, M, E), where the sequence Use the softplus activation function to ensure it is positive. Then convert the sequence Apply the Hadamard product (element-wise multiplication) to get the transformation parameters and , that is, the sequence with sequence (E, N) performs Hadamard product to obtain parameters ,sequence The elements in are trainable parameters, the sequence with sequence (B, M, N) performs Hadamard product to obtain parameters , and finally use , , and input Through the spatial state model, the forward processing result of sequence X is calculated .

[0028] A spatial state model, SSM, is a method that transforms a one-dimensional function or sequence x(t) (x(t)∈R) through a hidden state h(t) (h(t)∈ ) is mapped to y(t) (y(t)∈R). The system uses the evolution parameter A (A∈ ), projection parameter B (B∈ )、C(C∈ ), A is the evolution matrix of the hidden state, B is the influence matrix of the input signal x(t) on the hidden state, and C is the output matrix.

[0029] use , , and input The forward processing result of sequence X is calculated by the spatial state model The specific process is expressed by the formula: h(X)=A+B×X = ×h(X)+ × = × In the above formula, A and B are trainable parameter matrices, and X is the sequence X.

[0030] The result of backward processing of sequence X The obtained process is the same as the forward processing result of sequence X The acquisition process is similar, the difference is that the one-dimensional forward convolution operation in the forward processing is replaced by a one-dimensional backward convolution operation.

[0031] The sequence X will be , The results obtained after the activation function processing of sequence Z are added separately, and then the two results are added together. The obtained results are processed by the projection layer and combined with the primary pixel sequence information of the image block. Perform residual connection (addition) and then generate a sequence ,sequence It will be sent to MLP (multi-layer perceptron) for processing to obtain the encoded pixel sequence information; the encoded pixel sequence information is combined with the position information of the corresponding image block to obtain the embedded feature information of the image block; the embedded feature information of the image block (referred to as image embedding) is the output of the image encoder.

[0032] The prompt encoder module is used to convert the input prompt into a vector sequence that the model can understand. Prompts include the following forms: points, boxes, texts, and masks. If the input is a prompt of a point or box, the prompt encoder module will output a position vector through a position encoder. If the input is a text prompt, the prompt encoder module will output a vector as an encoder through the Clip model. If the input is a mask prompt, the prompt encoder module will output a vector sequence as an encoder through convolution. In this embodiment, the prompt encoder module inputs a mask prompt, which is a label image of image F; the prompt encoder module processes the label image through three different convolutional layers, two of which output target tokens (output tokens) and prompt tokens (Prompttokens) respectively; at the same time, the output result of the third convolutional layer of the prompt encoder module is weighted to output high-quality target tokens (HQ-Output-tokens), where the weight is a learnable parameter. The outputs of the three convolutional layers are all one-dimensional vectors with the same number of elements.

[0033] The high-quality target token (HQ-Output-tokens) described above is a learnable parameter vector that obtains a representation of the high-quality segmentation features of the input image through the learning process. It will be combined with the target tokens (Output tokens) and prompt tokens output by the prompt encoder module and input into the mask decoder module. First, the high-quality target tokens (HQ-Output -tokens) will use the global semantic context and local detail features to enhance the model's understanding of the image content by fusing the target tokens (Output tokens) and prompt tokens (Prompttokens). Then, the high-quality target tokens (HQ-Output -tokens) can focus on different areas of the input image through the attention mechanism of the mask decoder module, thereby better understanding the image structure and content. Finally, in an iterative manner, the high-quality target tokens (HQ-Output -tokens) update their representation after each decoder layer, gradually optimizing the mask prediction.

[0034] The Mask decoder module uses the output of the image encoder and the output tokens, prompt tokens, and high-quality target tokens (HQ-Output-tokens) output by the prompt encoder to finally output the predicted segmentation mask. The specific workflow is as follows: First, the target tokens (output tokens), prompt tokens (Prompttokens), and high-quality target tokens (HQ-Output-tokens) output by the prompt encoder module are used as Query, Key, and Value to combine into a three-dimensional matrix of original features F0, which is used as the input of the first CBAM attention mechanism layer. The first CBAM attention mechanism layer outputs a one-dimensional first output information F1.

[0035] The CBAM attention mechanism updates the representations of these tokens internally, and each token is updated based on the information of other tokens. For each token, its attention score with all other tokens in the set (including itself) is calculated. This is achieved by calculating the dot product between tokens. The attention scores are normalized using the softmax function of the channel attention module in the CBAM attention to ensure that the sum of all scores is 1. This step ensures that the model can weigh the relative importance of different tokens. The normalized attention scores are used as weights to perform a weighted summation of the representations of other tokens. In this way, the updated representation of each token not only contains its own information, but also incorporates the information of all other tokens. The updated token representation is added to the original token representation through a residual connection to enhance the learning ability of the model and avoid the gradient vanishing problem in deep networks. Next, the updated token representation is normalized using layer normalization to obtain the one-dimensional first output information F1; Then, the first output information F1 is input as Query to the second CBAM attention mechanism layer. At the same time, the encoded pixel sequence information in the embedded feature information of the image block output by the image encoder module is input as Key and Value to the second CBAM attention mechanism layer. The above Query, Key and Value are combined into a three-dimensional matrix feature information. The feature information is processed by the second CBAM attention mechanism layer to obtain a one-dimensional second output information F2. The second output information F2 is input into the first KAN network, and after being processed by the first KAN network, the third output information F3 is obtained; wherein, the part of the third output information F3 corresponding to the first output information F1 is the mask prediction token related to the image segmentation task, and the part corresponding to the encoded pixel sequence information in the embedded feature information of the image block is the specific image feature representation used to predict the segmentation mask.

[0036] Then, the specific image feature representation in the third output information F3 is input as a query to the third CBAM attention mechanism layer, and the encoded pixel sequence information in the embedded feature information of the image block output by the image encoder module is input as the key and value to the third CBAM attention mechanism layer. After processing and updating the information, the one-dimensional fourth output information F4 is obtained; The fourth output information F4 is used as the key and value, and the mask prediction token in the third output information F3 is used as the query, and is combined into a three-dimensional matrix and input into the fourth CBAM attention mechanism layer. After processing, the mask of each target point is output to obtain the one-dimensional fifth output information F5; At the same time, the fourth output information F4 undergoes two convolution operations to obtain an output feature F6 with the same size as the original image F; The part of the fifth output information F5 corresponding to the mask prediction token is input to the second KAN network, and a binary image with the probability of each predicted target mask is output; the binary image is multiplied by the output feature F6 to obtain the final segmented image, which is also a binary image. The mask decoder finally outputs the segmented image of the original image F.

[0037] The first CBAM attention mechanism layer, the second CBAM attention mechanism layer, the third CBAM attention mechanism layer, and the fourth CBAM attention mechanism layer have the same basic structure, but the parameters are not shared. Take the first CBAM attention mechanism layer as an example to explain the structure and principle: its structure is as follows Figure 3 As shown, first, the first CBAM attention mechanism layer receives Query, Key, and Value as input, where Query, Key, and Value correspond to target tokens (output tokens), prompt tokens (Prompttokens), and high-quality target tokens (HQ-Output-tokens), respectively. The three are arranged into a matrix (i.e., features) to form the original feature F0.

[0038] The original feature F0 is first input into the Channel Attention Module, which first compresses its features in the spatial dimension using average pooling and maximum pooling to generate two different spatial context descriptors. and The two descriptors are then input into a shared multi-layer perceptron (MLP) to generate channel attention features , the formula is as follows: = ))+ ( ( ))) In the above formula, refers to the sigmoid function, , R is the hidden activation size, r is the reduction rate, C is the number of channels, , is the trainable parameter matrix.

[0039] Then the channel attention feature The refined feature F' is obtained by element-wise multiplication with the original feature F0. The feature F' is used as the input of the Spatial Attention Module, which applies average pooling and maximum pooling in the channel dimension to generate two 2D maps and The two 2D maps are then concatenated in the channel dimension and passed through a convolutional layer to generate spatial attention features. The formula is as follows: = ; ) In the above formula, refers to the sigmoid function, Represents the convolution operation of the filter 7*7.

[0040] Finally, the spatial attention feature Multiply it with F' to get the final refined one-dimensional feature F1. F1 will be used as the input of the subsequent network layer to achieve adaptive refinement of features.

[0041] The basic structures of the first KAN network and the second KAN network are the same, but the parameters are not shared. The first KAN network is a structure including two activation function layers, wherein the input of the first layer is two identical features, and the input of the second layer is five; for the two inputs of the first layer, the ten activation functions of the first layer are used for processing, and each input obtains five clones; the first clone of the first input and the first clone of the second input are added together, and the result is used as the first input of the second layer, and the remaining four inputs of the second layer are obtained in the same way. The five inputs of the second layer are processed by the five activation functions of the second layer, and then the five results are added together to obtain the output of the first KAN network; the formula of each activation function in the two activation function layers is the same, but the parameters are not shared.

[0042] Take the first KAN network as an example to explain the structure and principle: the KAN network structure is as follows: Figure 4 As shown, when the first layer input of the KAN network has n, the output is 2n+1, the second layer input is 2n+1, and the final output is 1. In this embodiment, there are two inputs, so the second layer has 2×2+1=5. is the activation function on each edge, that is, the nonlinear transformation, is the layer number, =0 means the first layer, =1 indicates the second layer. Each input has 5 clones, which are then combined separately. is used to mark the nodes of the current layer, and Used to mark the nodes of the next layer. Each node The output of the activation function After processing, contribute to all the next layers The activation function is the basis function b( ) and the sum of the spline functions: ( ) = w (b ( )+spline( ) The b ( )=silu( )= , spline( ) is a spline function; spline( )= ,in is a trainable parameter, and w is a preset constant parameter used to control the activation function The range of output values. silu is a neural network activation function, also known as the swish function. is the input feature, the two input All the same.

[0043] The activation functions in the first KAN network and the second KAN network are the same, but the parameters are not shared.

[0044] The two second output information F2 are used as inputs of the first KAN network, and the inputs of the second KAN network are analogous to those of the first KAN network.

[0045] The KAN network will fit the target function as much as possible. During the training process, the sparse method will be used to improve the generalization ability of the model and reduce the risk of overfitting. After sparseness, unimportant connections and neurons are further removed through pruning technology. After adjusting the activation function, the remaining parameters in the model are retrained to optimize these parameters to best fit the data. Finally, the KAN network will output a well-fitted symbolic formula, which is the feature representation required for prediction.

[0046] Step 3: Train the network model for efficient segmentation of prefabricated T-beam connection steel bars in step 2 First, the trainable parameters of the efficient segmentation network model in step 2 are randomly initialized, and the fixed parameters of the network model are set. Then, an image in the training set and the corresponding label image are input into the initialized efficient segmentation network model. The efficient segmentation network model outputs a segmented image of a binary image, and then combined with the corresponding label image, the pixel-level training loss is calculated using the mean square error loss function, and the trainable parameters of the efficient segmentation network model are updated once through the training loss back propagation. All images are iterated and trained in sequence. After training all images in the training set, a round of training is completed. The trainable parameters of the network that completed the previous round are used as the initial trainable parameters of the next round. When the training gradient value is within ±1e －５ When the training iterations reach 30,000 or the training is stopped, the trainable parameters with the smallest training loss are extracted to obtain a trained efficient segmentation network model.

[0047] Then the images in the test set and the corresponding annotated images are input into the trained efficient segmentation network model, and the accuracy of the segmentation results is calculated. When the pixel accuracy is 95%-100%, it is proved that the trained efficient segmentation network model is the optimal segmentation model. Otherwise, the initialization method of the trainable parameters of the efficient segmentation network model is adjusted, and the values ​​of the fixed parameters of the network model are adjusted. The training and testing processes of the efficient segmentation network model are repeated until the optimal segmentation model is obtained.

[0048] The training was done on the same training set and the performance was tested on the same test set. The performance of different network models is shown in Table 1.

[0049]

[0050] Mean Pixel Accuracy (mPA) is a commonly used evaluation metric in image segmentation tasks. It measures the consistency between the mask predicted by the segmentation network model and the true mask. Pixel accuracy is the correct proportion of each pixel point prediction, which is an intuitive indicator of image segmentation quality. Width measurement accuracy usually refers to the accuracy of measuring the width of a specific object in image analysis or computer vision tasks. Average FPS is the number of images that can be processed per second.

[0051] Step 4: Inspection of the connecting steel bars of the upper flange of the assembled T-beam The image of the steel bar connected to the flange of the assembled T-beam to be inspected is taken by a drone, and then input into the optimal segmentation network model in step 3. The network model outputs the segmentation image of the binary image of the image; the length of each type of pixel is counted, and the white pixel with a length greater than 2 is the pixel of the steel bar. When the length of the steel bar is detected to be between 580-620 pixels, it is qualified. If the length is not in this range, the steel bar connected to the flange of the assembled T-beam at this location is judged to be abnormal, and the quality of the steel bar connected to the flange of the assembled T-beam is detected.

[0052] Since the model training uses a label image of 712×712 pixels, the actual ratio of 1:20 is used for conversion, that is, 1 pixel is 0.05 cm, and the actual length of the steel bar can be obtained.

[0053] The calculation method of the pixel area in the image is as follows: the binary mask of the steel bar is obtained by the image segmentation algorithm, that is, the steel bar part is marked as the foreground and the non-steel part is marked as the background. The number of pixels in the steel bar part, that is, the number of foreground pixels, is calculated using the pixel counting method, which is represented by N. The number of pixels N is multiplied by the square of the pixel size to obtain the area S of the steel bar part, in pixels squared.

[0054] Any matters not described in the present invention are applicable to the prior art.

Claims

1. An intelligent detection method for the quality of steel bars connected to the flange of an assembled T-beam, characterized in that: The detection method comprises the following steps: Step 1: Obtain no less than 1,000 images of the steel bars connected to the flange plates of prefabricated T-beams, first pre-process them, then label each image, and then divide them according to the number of images, with no less than 60% of them divided into training sets and the rest divided into test sets; Step 2: Construct an efficient network model for segmenting steel bars on the flange of prefabricated T-beams The efficient segmentation network model is an improved SAM network model, which mainly includes three backbone modules, namely, an image encoder module, a prompt encoder module, and a mask decoder module; The improved SAM network model uses the VIM model as an image encoder to extract the feature representation of the input image; the specific workflow of the VIM model is as follows: First, the input image F is divided into multiple non-overlapping image blocks of fixed size, and then the position information of each image block is embedded into its corresponding pixel vector representation information, and the pixel vector representation information of each image block is flattened and arranged into a sequence to obtain the original pixel sequence information and the corresponding position information of each image block; the original pixel sequence information of each image block is linearly transformed through a linear layer to adjust its data dimension, and the combined embedding features of the primary pixel sequence information and the corresponding position information of each image block are obtained; then, the primary pixel sequence information of the image block is normalized through a normalization layer, and batch normalization is used; thereafter, the results of the primary pixel sequence information Y of the image block after the normalization layer are processed using two different projection layers, and are projected into a sequence X and a sequence Z of dimension E respectively; Then, the sequence X is processed in both forward and backward directions to capture the local dependencies in the vector; for forward processing, the sequence X is firstly processed by applying a one-dimensional forward convolution operation and a SILU function to obtain , and then Linear projection is applied to three different linear layers to obtain the sequences , and , where the sequence Use the soft-add activation function to ensure that it is positive; then the sequence Apply the Hadamard product to obtain the transformation parameters and , that is, the sequence with sequence Perform Hadamard product to get parameters ,sequence The elements in are trainable parameters, the sequence with sequence Perform Hadamard product to get parameters ; Last used , , and input Through the spatial state model, the forward processing result of sequence X is calculated ; use , , and input The forward processing result of sequence X is calculated by the spatial state model The specific process is expressed by the formula: h(X)=A+B×X = ×h(X)+ × = × In the above formula, A and B are trainable parameter matrices, and X is the sequence X; The result of backward processing of sequence X The obtained process is the same as the forward processing result of sequence X The process of obtaining is similar, except that the one-dimensional forward convolution operation in the forward processing is replaced by a one-dimensional backward convolution operation; The sequence X will be , The results obtained after the activation function processing of sequence Z are added separately, and then the two results are added together. The obtained results are processed by the projection layer and combined with the primary pixel sequence information of the image block. Perform residual connection and then generate sequence ,sequence It will be sent to MLP for processing to obtain the coded pixel sequence information; the obtained coded pixel sequence information is combined with the position information of the corresponding image block to obtain the embedded feature information of the image block; the embedded feature information of the image block is the output of the image encoder; The prompt encoder module is used to convert the input prompt into a vector sequence that the model can understand; the prompt encoder module inputs a mask prompt, which is a label image of image F; the prompt encoder module processes the label image through three different convolutional layers, two of which output a target token and a prompt token respectively; at the same time, the output result of the third convolutional layer of the prompt encoder module is weighted to output a high-quality target token; the weight is a learnable parameter; the outputs of the three convolutional layers are all one-dimensional vectors with the same number of elements; The specific workflow of the mask decoder module is as follows: First, the target token, prompt token, and high-quality target token output by the prompt encoder module are used as Query, Key, and Value to combine into a three-dimensional matrix of original features F0, which is used as the input of the first CBAM attention mechanism layer. The first CBAM attention mechanism layer outputs a one-dimensional first output information F1; Then, the first output information F1 is input as Query to the second CBAM attention mechanism layer. At the same time, the encoded pixel sequence information in the embedded feature information of the image block output by the image encoder module is input as Key and Value to the second CBAM attention mechanism layer. The above Query, Key and Value are combined into a three-dimensional matrix feature information. The feature information is processed by the second CBAM attention mechanism layer to obtain a one-dimensional second output information F2. The second output information F2 is input into the first KAN network, and after being processed by the first KAN network, the third output information F3 is obtained; wherein the part of the third output information F3 corresponding to the first output information F1 is a mask prediction token related to the image segmentation task, and the part corresponding to the encoded pixel sequence information in the embedded feature information of the image block is a specific image feature representation for predicting the segmentation mask; Then, the specific image feature representation in the third output information F3 is input as a query to the third CBAM attention mechanism layer, and the encoded pixel sequence information in the embedded feature information of the image block output by the image encoder module is input as the key and value to the third CBAM attention mechanism layer. After processing and updating the information, a one-dimensional fourth output information F4 is obtained; The fourth output information F4 is used as the key and value, and the mask prediction token in the third output information F3 is used as the query, and is combined into a three-dimensional matrix and input into the fourth CBAM attention mechanism layer. After processing, the mask of each target point is output to obtain the one-dimensional fifth output information F5; At the same time, the fourth output information F4 undergoes two convolution operations to obtain an output feature F6 with the same size as the original image F; The part of the fifth output information F5 corresponding to the mask prediction token is input to the second KAN network, and a binary image with the probability of each predicted target mask is output; the binary image is multiplied by the output feature F6 to obtain the final segmentation image, which is also a binary image; the mask decoder finally outputs the segmentation image of the original image F; The first CBAM attention mechanism layer, the second CBAM attention mechanism layer, the third CBAM attention mechanism layer, and the fourth CBAM attention mechanism layer have the same basic structure, but the parameters are not shared; The first KAN network and the second KAN network have the same basic structure, but the parameters are not shared; the first KAN network is a structure including two activation function layers, wherein the input of the first layer is two identical features, and the input of the second layer is five; the two inputs of the first layer are processed using the ten activation functions of the first layer, and each input obtains five clones; the first clone of the first input and the first clone of the second input are added together, and the result is used as the first input of the second layer, and the remaining four inputs of the second layer are obtained in the same manner; the five inputs of the second layer are processed by the five activation functions of the second layer, and then the five results are added together to obtain the output of the first KAN network; the formula of each activation function in the two activation function layers is the same, but the parameters are not shared; The two second output information F2 are used as the input of the first KAN network, and the input of the second KAN network is analogous to that; Step 3: Train the network model for efficient segmentation of steel bars connected to the flange of the assembled T-beam in step 2 First, the trainable parameters of the efficient segmentation network model in step 2 are randomly initialized, and the fixed parameters of the network model are set. Then, an image in the training set and the corresponding label image are input into the initialized efficient segmentation network model. The efficient segmentation network model outputs a segmented image of a binary image. Then, combined with the corresponding label image, the mean square error loss function is used to calculate the pixel-level training loss, and then the trainable parameters of the efficient segmentation network model are updated once through the back propagation of the training loss. All images are iteratively trained in sequence. After training all images in the training set, a round of training is completed. The trainable parameters of the network that completed the previous round are used as the initial trainable parameters of the next round. When the training gradient value is within ±1e －５ When the training iterations reach 30,000, stop training and extract the trainable parameters with the smallest training loss to obtain an efficient segmentation network model that has completed training. Then, the images in the test set and the corresponding annotated images are input into the trained efficient segmentation network model, and the accuracy of the segmentation results is calculated. When the pixel accuracy is 95%-100%, it is proved that the trained efficient segmentation network model is the optimal segmentation model. Otherwise, the initialization method of the trainable parameters of the efficient segmentation network model is adjusted, and the values ​​of the fixed parameters of the network model are adjusted. The training process and the testing process of the efficient segmentation network model are repeated until the optimal segmentation model is obtained. Step 4: Inspection of the connecting steel bars of the upper flange of the assembled T-beam Use a drone to take an image of the steel bars connected to the wing plate of the prefabricated T-beam to be inspected, and then input it into the optimal segmentation network model in step three. The network model outputs a segmented image of the binary image of the image; the lengths of various pixel points are counted, and the white pixels with a length greater than 2 are the pixels of the steel bars; when the length of the steel bars is detected to be between 580 and 620 pixels, it is qualified. If the length is not in this range, the steel bars connected to the wing plate of the prefabricated T-beam at this location are judged to be abnormal, thereby realizing the quality detection of the steel bars connected to the wing plate of the prefabricated T-beam.

2. According to claim 1, a method for intelligently detecting the quality of steel bars connected to the flange of an assembled T-beam is characterized in that: In step 1, 2000 images of the connecting steel bars of the flange plate of the assembled T-beam are obtained.

3. The intelligent detection method for the quality of steel bars connected to the flange of an assembled T-beam according to claim 2 is characterized in that: 1500 of the 2000 images were collected by drones, and 500 were images of steel bars connecting the wing plates of prefabricated T-beams that were publicly available on the Internet.

4. The intelligent detection method for the quality of steel bars connected to the flange of an assembled T-beam according to claim 1 is characterized in that: In step 1, the preprocessing method specifically uses the MSR algorithm to perform the following operations on the image: Step 1.1: Perform log transformation (i.e. logarithmic transformation) on the image to obtain a log image; Step 1.2: Gaussian blur the log image (i.e., obtain the low-frequency component) to obtain the blurred log image; Step 1.3: Use the original image and the blurred log image to make a difference to get the image; The original image is subjected to the operations of step 1.1, step 1.2, and step 1.3 in the three dimensions of R, G, and B, respectively. The difference is that when performing the Gaussian blur in step 1.2, the sigmoid parameters are set to 15, 80, and 200, respectively; each dimension of an image is subjected to the operations of step 1.1, step 1.2, and step 1.3, and then the results of the three steps are weightedly averaged. Finally, the three dimensions are merged and the data is scaled to 0-255 to obtain the preprocessed image R. The data set R* is obtained from all the preprocessed images.

5. The intelligent detection method for the quality of steel bars connected to the flange of an assembled T-beam according to claim 1 is characterized in that: In step 1, the method for labeling each image is as follows: using the binary segmentation mask of the flange connection steel bars of the prefabricated T-beam, the area to be identified in the preprocessed image is annotated at the pixel level using the Photoshop tool, and a corresponding semantic label is assigned to each pixel to generate a 712×712 binary image label; in the label, the black pixel value is 0, representing the background, and the white pixel value is 255, among which the width of 1 to 2 pixels is the white annotation line, and the length of 580 to 620 pixels represents the target steel bar.

6. The intelligent detection method for the quality of steel bars connected to the flange of an assembled T-beam according to claim 1 is characterized in that: In step 2, the linear transformation is defined by a weight matrix and a bias term, where the input vector is multiplied by the weight matrix and then the bias term is added to obtain the output of the linear transformation.

7. The intelligent detection method for the quality of steel bars connected to the flange of an assembled T-beam according to claim 1 is characterized in that: In step 2, the two different projection layers are two different linear layers.

8. The intelligent detection method for the quality of steel bars connected to the flange of an assembled T-beam according to claim 1 is characterized in that: In step 2, the first CBAM attention mechanism layer receives Query, Key, and Value as input. Here, Query, Key, and Value correspond to the target token, prompt token, and high-quality target token respectively. The three are arranged into a matrix to form the original feature F0; The original feature F0 is first input into the channel attention module, which first compresses its features in the spatial dimension using average pooling and maximum pooling respectively. Generate two different spatial context descriptors and ; The two descriptors are then input into a shared multi-layer perceptron to generate channel attention features , the formula is as follows: = ））+ （ （ ））） In the above formula, refers to the sigmoid function, , is the trainable parameter matrix; Then the channel attention feature By multiplying the original feature F0 element by element, we get the refined feature F'; The feature F' is used as the input of the spatial attention module, which applies average pooling and max pooling in the channel dimension to generate two 2D maps and ; The two 2D maps are then concatenated in the channel dimension and passed through a convolutional layer to generate spatial attention features. ; The formula is as follows: = ； ）) In the above formula, refers to the sigmoid function, Represents the convolution operation of filter 7*7; Finally, the spatial attention feature Multiply it with F' to get the final refined one-dimensional feature F1.

9. The intelligent detection method for the quality of steel bars connected to the flange of an assembled T-beam according to claim 1 is characterized in that: In step 2, the activation functions in the first KAN network and the second KAN network are the same, but the parameters are not shared.

10. The intelligent detection method for the quality of steel bars connected to the flange of an assembled T-beam according to claim 9 is characterized in that: The activation function is the basis function b( ) and the sum of the spline functions: ( )=w(b( )+spline( )) The b ( )=silu( )= , spline( ) is a spline function, w is a preset constant parameter used to control the activation function The range of output values.

Images