A tower foundation crack detection method based on pixel focused attention

Abstract

A tower foundation crack detection method based on pixel-focused attention relates to the field of crack detection. This invention addresses the low accuracy of existing tower foundation crack detection methods. The method comprises: utilizing a tower image dataset to obtain a training set, and utilizing the training set to train a tower foundation crack detection network; inputting the tower foundation crack image to be tested into a trained ResNet network to obtain a feature map of the tower foundation crack image to be tested; and constructing a query sequence to be tested using the predicted crack type and crack target prediction frame position of the tower image data in the training set. The query sequence to be tested and the feature map of the tower foundation crack image to be tested are input into the trained tower foundation crack detection network to obtain the predicted crack type and corresponding probability in the tower foundation crack image to be tested, as well as the predicted crack target detection frame position of the tower foundation crack to be tested. This method is used to detect tower foundation crack types.

Description

Technical Field

[0001] The present invention relates to the field of crack detection, and in particular to a tower foundation crack detection method based on pixel focused attention. Background Art

[0002] With the advancement of science and technology, people's demand for networks has expanded beyond basic communication functions. The pursuit of a smooth and efficient user experience has become a common belief. To meet this growing demand for communication, my country is continuously expanding the number and scale of its towers. China Tower, the world's largest provider of communications tower infrastructure services, operated and managed 2.046 million tower sites by the end of 2023. However, due to factors such as weather changes, material usage, and construction quality, tower foundations are prone to cracking over time, forming various types of foundation cracks. These cracks can affect the overall stability of the tower foundation, reducing its ability to withstand wind loads and equipment loads, and even causing it to tilt or deform. If not addressed promptly, they can lead to more serious safety issues, such as tower collapse or other structural failures, posing a serious threat to the safety of surrounding personnel and equipment.

[0003] Traditional methods for detecting cracks in tower foundations rely on manual inspection, ultrasonic testing, and penetrant testing. These methods are subject to significant subjectivity, high cost, and complex processing. To address these issues and efficiently detect and provide early warnings for tower foundation cracks, the field of artificial intelligence has proposed a tower foundation crack detection method based on a convolutional neural network (CNN). CNNs use fixed-size convolution kernels to extract features, but this method has limitations when dealing with cracks that exhibit diverse shapes and complex backgrounds. Cracks often exhibit diverse shapes, and CNN training primarily extracts local features. However, crack location and size are often dependent on global context, and CNNs are limited in their ability to process this information. This limitation results in existing methods being unable to effectively identify cracks of certain shapes or sizes, resulting in low accuracy in tower foundation crack detection. Summary of the Invention

[0004] The purpose of the present invention is to solve the problem of low accuracy in existing tower foundation crack detection, and to propose a tower foundation crack detection method based on pixel focused attention.

[0005] A tower foundation crack detection method based on pixel focused attention, the specific process is as follows:

[0006] S1. Using the tower image data set to obtain a training set, using the training set to train the tower foundation crack detection network, obtaining the trained tower foundation crack detection network and the crack prediction type and crack target prediction frame position of the tower image data in the training set;

[0007] S2. Obtain an image of cracks in the foundation of the iron tower to be tested, input the image of cracks in the foundation of the iron tower to be tested into the trained ResNet network, and obtain a feature map of the image of cracks in the foundation of the iron tower to be tested;

[0008] S3, using the crack prediction type and crack target prediction frame position of the tower image data in the training set to form a query sequence to be tested, inputting the query sequence to be tested and the crack image feature map of the tower foundation to be tested into the trained tower foundation crack detection network, and obtaining the crack prediction type corresponding to the maximum probability in the tower foundation crack image to be tested and the crack target detection frame prediction position of the tower foundation to be tested;

[0009] The types of cracks in the image of the iron tower to be tested include: surface cracks, deep cracks, and through cracks.

[0010] Furthermore, the step of obtaining a training set using the tower image dataset in S1, and training the tower foundation crack detection network using the training set to obtain a trained tower foundation crack detection network is performed in the following manner:

[0011] Step 1: Obtain the tower image dataset;

[0012] The tower image data set includes: tower image data and target detection frame position labels and crack type labels;

[0013] The target detection frame position includes: the coordinates of the upper left corner and the lower right corner of the crack target detection frame;

[0014] Step 2: Use the tower image dataset to obtain a training set, and use the training set to train the tower foundation crack detection network to obtain a trained tower foundation crack detection network, specifically:

[0015] Step 21: Input the tower image data into the pre-trained ResNet network to obtain the tower image feature map, randomly initialize the tower foundation crack target detection frame position and the tower foundation crack type, and use the tower image feature map, the initialized tower foundation crack target detection frame position, the initialized tower foundation crack type, the target detection frame position label and the crack type label of the tower image data to form a training set;

[0016] Step 22: Use the training set to train the tower foundation crack detection network to obtain a trained tower foundation crack detection network.

[0017] Furthermore, the tower foundation crack detection network includes: an input module, an encoding module, a decoding module, and an output module;

[0018] The input module uses the tower image feature map, the crack target detection frame position and the tower foundation crack type to obtain the final input sequence and target sequence;

[0019] The input module includes: an input embedding layer, a first position encoding layer, an output embedding layer, and a second position encoding layer;

[0020] The input embedding layer is used to process the tower image feature map to obtain an input sequence;

[0021] The first position encoding layer is used to add position codes to the input sequence to obtain a final input sequence X, and the final input sequence X is input into the encoding module;

[0022] The output embedding layer is used to process the crack target detection frame position and the corresponding crack type label to obtain the target query sequence;

[0023] The second position encoding layer is used to add position encoding to the target query sequence to obtain a target sequence, and the target sequence is input into the multi-head attention layer of each decoder in the decoding module;

[0024] The encoding module is used to encode the final input sequence X to obtain the encoding information matrix C;

[0025] The encoding module is composed of multiple encoders connected in series;

[0026] Each encoder includes: a multi-head attention mechanism layer, a first Add&Norm layer, a first feedforward control layer, and a second Add&Norm layer;

[0027] The first Add&Norm layer and the second Add&Norm layer both include: a residual connection layer and a layer normalization layer; the output of the residual connection layer is the input of the layer normalization layer;

[0028] The input of the first encoder is the final input sequence X, the input of the n'th encoder is the output of the n'-1th encoder, and the output of the N'th encoder is the output of the encoding module, that is, the encoding information matrix C, n'∈[2,N'], N' is the total number of encoders in the encoding module;

[0029] The decoding module uses the coding information matrix C and the target sequence to obtain a prediction score of whether the cracks in the tower image belong to a certain type;

[0030] The decoding module is composed of a plurality of decoders connected in series;

[0031] Each decoder includes: a masked multi-head attention layer, a third Add&Norm layer, a multi-head attention layer, a fourth Add&Norm layer, a second feedforward control layer, and a fifth Add&Norm layer;

[0032] The third Add&Norm layer, the fourth Add&Norm layer, and the fifth Add&Norm layer have the same structure as the first Add&Norm layer;

[0033] The input of the first decoder is the encoding information matrix C and the target sequence. The input of the m'th decoder is the output of the m'-1th decoder and the encoding information matrix C. The output of the M'th decoder is the output of the decoding module, that is, the prediction score of whether the crack in the tower image belongs to a certain type. m'∈[2,M'], where M' is the total number of decoders.

[0034] The output module uses the prediction score of the crack in the tower image belonging to a certain category to output the probability that the crack in the current tower image belongs to a certain category and the predicted position of the crack target detection frame;

[0035] The output module includes: a linear layer and a softmax layer in sequence.

[0036] Furthermore, the input module uses the tower image feature map, the crack target detection frame position and the tower foundation crack type label to obtain the final input sequence and target sequence, specifically:

[0037] A1. Preprocess the tower image feature map to obtain the input sequence:

[0038] A1-1. Divide the tower image feature map into blocks according to a preset size to obtain multiple feature blocks;

[0039] A1-2. Flatten the feature blocks and input the flattened feature blocks into the fully connected layer to obtain the feature vector.

[0040] A1-3. Combine all feature vectors to obtain a feature sequence, and add the class token to the feature sequence as the starting feature vector of the feature sequence to obtain the input sequence.

[0041] A2. Add position codes to the input sequence to obtain the final input sequence.

[0042] A3. Process the crack target detection frame position and tower foundation crack type to obtain the target query sequence:

[0043] The crack target detection box position corresponding to a crack in the tower image feature map and the type of crack in the tower foundation are combined into a target query vector. All target query vectors form a target query sequence.

[0044] A4. Add a position code to the target query sequence to obtain the target sequence.

[0045] Furthermore, the position code is added to the input sequence in A2 to obtain the final input sequence, specifically:

[0046] A2-1. Obtain the position code of each feature vector in the input sequence, specifically:

[0047] The position encoding of the feature vector is composed by using the position encoding of each element in the feature vector;

[0048] The position encoding of each element in the feature vector is obtained by the following formula:

[0049]

[0050] Among them, pos is the feature vector number in the input sequence, d is the current feature vector dimension, i' is the dimension index in the current feature vector, PE (pos,2i') is the position code of the 2i'th element in the posth eigenvector, PE (pos,2i'+1) is the position code of the 2i'+1th element in the posth eigenvector, i'∈[0,d], d is an integer;

[0051] A2-2. Each eigenvector and the corresponding eigenvector position code are added together to form a sequence to obtain the final input sequence.

[0052] Furthermore, the step A4 of adding a position code to the target query sequence to obtain the target sequence is as follows:

[0053] A4-1. Obtain the position encoding of each target query vector in the target query sequence:

[0054] Composing a positional encoding of the target query vector using the positional encoding of each element in the target query vector;

[0055] The method for obtaining the position encoding of each element in the target query vector is the same as the method for obtaining the position encoding of each element in the feature vector;

[0056] A4-2. Each target query vector and the corresponding position code of the target query vector are added together to form a sequence to obtain a target sequence.

[0057] Furthermore, the encoding module is used to encode the final input sequence X to obtain the encoding information matrix C, which is specifically:

[0058] B1. Use the final input sequence X to obtain the local range attention score matrix of the pixel and the global context attention score matrix of the pixel:

[0059]

[0060] ||ρ(i,j)||=k 2

[0061] ||σ(X)||=HpWp

[0062] Where (i, j) is the coordinate of any pixel position, ρ(i, j) is a set of pixels in a sliding window of size k×k centered at (i, j), k is a constant, S (i,j)～ρ(i,j) is the local range attention score of the pixel at position (i, j), S (i,j)～σ(X) is the global context attention score of the pixel at position (i, j), Q (i,j) is the pixel Query value at position (i, j), K ρ(i,j) is the Key value of ρ(i,j), σ(X) represents the feature map obtained after X passes through a single-layer neural network, pooling, and layer normalization, K σ(X) is the Key value of σ(X), It's K ρ(i,j) The weight matrix, It's K σ(X) The weight matrix of , Hp is the height of the feature map obtained after X passes through a single-layer neural network and pooling, and Wp is the width of the feature map obtained after X passes through a single-layer neural network and pooling;

[0063] B2. Pixel attention weights using the pixel’s local range attention score matrix and the pixel’s global context attention score matrix:

[0064]

[0065] Among them, A (i,j) is the pixel attention weight at position (i, j), Concat() is the concatenation function, is the scaling factor, B(i,j) is the bias term;

[0066] B3. Use pixel attention weights to obtain the final input sequence attention value;

[0067] B4. Use PFA(X) to obtain the output of the multi-head attention mechanism layer in the encoder;

[0068] B5. Input the output of the multi-head attention mechanism layer in the current encoder and the final input sequence into the first Add&Norm layer for residual connection and normalization. Then input the output Y of the first Add&Norm layer into the first feedforward control layer. Then input the output of the first feedforward control layer and Y into the second Add&Norm layer for residual connection and layer normalization to obtain the output sequence of the current encoder.

[0069] B6. Use the output sequence of the current encoder as the input sequence of the next encoder and repeat B1-B4 until the last encoder outputs the encoding information matrix C.

[0070] Furthermore, the pixel attention weight is used in B3 to obtain the final input sequence attention value, specifically:

[0071]

[0072] PFA(X (i,j) )=A (i,j)～ρ(i,j) V ρ(i,j) +A (i,j)～σ(X) V σ(X)

[0073]

[0074] Among them, X (i,j) is the corresponding eigenvalue of position (i, j) in X, i∈[1,M], j∈[1,N], M is the maximum value of the pixel horizontal coordinate, N is the maximum value of the pixel vertical coordinate, PFA(X) is the final input sequence attention value, PFA(X) (i,j) ) is X (i,j) The corresponding attention value, A (i,j)～ρ(i,j) It's A (i,j) The relationship value with ρ(i,j), A (i,j)～σ(X) It's A (i,j) Relationship value with σ(X), V ρ(i,j) is the Value associated with ρ(i,j), V σ(X) A represents the Value value associated with σ(X). (m,n) is the pixel attention weight at position (m,n), d k It is the dimension of Query and key, It is V ρ(i,j) The weight matrix, It is V σ(X) The weight matrix of .

[0075] Furthermore, the output of the multi-head attention mechanism layer in the encoder is obtained by using PFA(X) in B4, specifically:

[0076] MultiHead(Q,K,V)=Concat(head1,…,head h )W O

[0077] where head i* =Attention(QW i * Q ,KWi* K ,VW i* V )

[0078] Q=PFA(X)W Q

[0079] K=PFA(X)W K

[0080] V=PFA(X)W V

[0081] Where i*∈[1,h], i* is the number of attention heads in the multi-head attention mechanism layer in the encoder, h is the total number of attention heads in the multi-head attention mechanism layer in the encoder, and head i* is the output of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the query projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Key projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Value projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, W O is the output projection matrix, Q is the Query value of the current multi-head attention mechanism layer, K is the Key value of the current multi-head attention mechanism layer, V is the Value value of the current multi-head attention mechanism layer, and W Q is the weight matrix of Q, W K is the weight matrix of K, W V is the weight matrix of V.

[0082] Furthermore, the decoding module uses the coding information matrix C and the target sequence to obtain a prediction score of whether the cracks in the tower image belong to a certain type, specifically:

[0083] C1. Input the target sequence M into the masked multi-head attention layer to obtain the masked attention output;

[0084] C2. Use the masked attention output and the target sequence M for residual connection and normalization to obtain the updated target sequence Z;

[0085] C3, input the encoded information matrix C and the updated target sequence Z into the multi-head attention layer of the decoder to obtain the multi-head attention layer output matrix of the decoder;

[0086] MultiHead(Q',K',V')=Concat(head1,head2,....,head h' )W' O

[0087]

[0088] K'=CW' K'

[0089] V'=CW' V'

[0090] Q'=ZW' Q'

[0091] Where i'∈[1,h'], i' is the number of attention heads in the multi-head attention layer in the decoder, h' is the total number of attention heads in the multi-head attention layer in the decoder, head i” is the output of the ith attention head, is the Query projection matrix of the i'th attention head, is the Key projection matrix of the i'th attention head, is the Value projection matrix of the i'th attention head, W' O is the output projection matrix, W' K' is the weight matrix of K', W' V' is the weight matrix of V', W' Q' is the weight matrix of Q', Q' is the Query value of the current multi-head attention layer, K' is the Key value of the current multi-head attention layer, and V' is the Value value of the current multi-head attention layer;

[0092] C4, use the multi-head attention layer output matrix of the decoder obtained in C3 and the matrix Z to perform residual connection and normalization to obtain the matrix Z', then input the matrix Z' into the second feedforward control layer, input the output matrix of the second feedforward control layer and the matrix Z' into the fifth Add&Norm layer to obtain the output matrix Z* of the current decoder;

[0093] C5. Use the output matrix Z* of the current decoder as the input matrix of the next decoder and repeat C1-C4 until the last decoder outputs the prediction score of whether the cracks in the tower image belong to a certain type.

[0094] The beneficial effects of the present invention are:

[0095] Based on the Transformer network architecture, the present invention improves the attention mechanism of the encoder and applies it to the task of detecting and classifying cracks in tower foundations. The present invention classifies cracks in tower foundations through real data sets and divides them into three types: surface cracks, deep cracks, and through cracks, thereby reducing the labor cost of identifying cracks in tower foundations and improving the speed of detecting cracks in tower foundations. The encoding module of the present invention adopts a pixel-focused attention mechanism to combine coarse-grained information with fine-grained information, thereby improving the overall understanding of the tower foundation crack image, and the fine-grained analysis improves the ability to identify cracks in the tower foundation, thereby improving the accuracy of tower foundation crack detection. The encoder of the present invention adopts a Dense Query architecture, which can create a comprehensive vector containing more information by splicing multiple query vectors. Therefore, the present invention can still achieve accurate feature extraction in images with complex backgrounds, thereby learning more complex features and greatly improving the accuracy of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 This is the Tansformer architecture diagram;

[0097] Figure 2(a) is a physical picture of the cracks on the surface of the tower foundation;

[0098] Figure 2(b) is a physical picture of the deep cracks in the tower foundation;

[0099] Figure 2(c) is a physical picture of the through-crack in the tower foundation;

[0100] Figure 3 Flowchart of the pixel-focused attention mechanism;

[0101] Figure 4 This is a multi-head attention model diagram;

[0102] Figure 5(a) is the Basic Query architecture diagram;

[0103] Figure 5(b) is the Dense Query architecture diagram. DETAILED DESCRIPTION

[0104] Specific implementation method 1: This implementation method is a method for detecting cracks in a tower foundation based on pixel focus. The specific process is as follows:

[0105] S1. Using the tower image data set to obtain a training set, using the training set to train the tower foundation crack detection network, obtaining the trained tower foundation crack detection network and the crack prediction type and crack target prediction frame position of the tower image data in the training set;

[0106] The trained tower foundation crack detection network is obtained by:

[0107] Step 1: Obtain the tower image dataset;

[0108] The tower image data set includes: tower image data and target detection frame position labels and crack type labels;

[0109] The position of the crack target detection frame of the tower foundation includes: the coordinates of the upper left corner and the lower right corner of the crack target detection frame;

[0110] The tower image dataset uses real tower data. The dataset includes 1,500 towers in Heilongjiang Province, totaling more than 30,000 images, of which about 10,000 are images of tower foundation cracks. All tower foundation cracks in the dataset are divided into three types: 1) surface cracks 2) deep cracks 3) through cracks, such as Figure 2(a)-Figure 2(c) shown.

[0111] Among them, surface cracks are shallow and mostly confined to the outer surface or surface layer, and have not penetrated deep into the main structure, which will not have a significant impact on the overall tower structure; deep cracks have begun to affect the main structure of the tower, and such cracks will reduce the overall stability of the structure; through cracks penetrate all layers of the tower foundation structure, seriously affecting the stability of the structure. The three types of cracks have different degrees of severity, and their warning levels and corresponding repair treatments should be effectively classified. Therefore, when the model identifies surface cracks, the impact on the tower structure is small, and the crack warning level is set as mild, requiring timely repair. When the model identifies deep cracks, there is a certain impact on the tower structure, and the crack warning level is set as moderate, requiring professional evaluation to determine the repair plan. When the model identifies through cracks, it seriously endangers the stability of the tower, and the crack warning level is set as severe, requiring emergency treatment and reconstruction or strengthening of the structure to avoid the risk of collapse. These three types of tower foundation crack example pictures are as follows Figure 2(a)-Figure 2(c) shown.

[0112] Step 2: Use the tower image dataset to obtain a training set, and use the training set to train the tower foundation crack detection network to obtain a trained tower foundation crack detection network, specifically:

[0113] Step 21: Input the tower image data into the pre-trained ResNet network to obtain the tower image feature map, randomly initialize the tower foundation crack target detection frame position and the tower foundation crack type, and use the tower image feature map, the initialized tower foundation crack target detection frame position, the initialized tower foundation crack type, the target detection frame position label and the crack type label of the tower image data to form a training set;

[0114] Step 22: Use the training set to train the tower foundation crack detection network to obtain a trained tower foundation crack detection network;

[0115] like Figure 1 As shown, the tower foundation crack detection network is an improved Transformer network architecture, including: input module, encoding module, decoding module, and output module;

[0116] The input module uses the tower image feature map, the crack target detection frame position and the tower foundation crack type to obtain the final input sequence and target sequence;

[0117] The input module includes: an input embedding layer Input Embedding, a first position coding layer PositionalEncoding, an output embedding layer Output Embedding, and a second position coding layer Positional Encoding;

[0118] The input embedding layer is used to process the tower image feature map to obtain an input sequence;

[0119] The first position encoding layer is used to add position codes to the input sequence to obtain a final input sequence X, and the final input sequence is input into the encoding module;

[0120] The output embedding layer is used to process the position of the crack target detection frame and the type of cracks in the tower foundation to obtain a target query sequence;

[0121] The second position encoding layer is used to add position encoding to the target query sequence to obtain a target sequence, and the target sequence is input into the multi-head attention layer of each decoder in the decoding module;

[0122] The encoding module is used to encode the final input sequence X to obtain the encoded final input sequence, namely the encoding information matrix C;

[0123] The encoding module is composed of multiple encoders connected in series; each encoder includes: a multi-head attention mechanism layer, a first Add&Norm layer, a first feedforward control layer, and a second Add&Norm layer;

[0124] The Add&Norm layer includes: Residual Connection layer and LayerNorm layer; the output of the residual connection layer is the input of the layer normalization layer;

[0125] The input of the first encoder is the final input sequence X, the input of the n'th encoder is the output of the n'-1th encoder, and the output of the last encoder is the output of the encoding module, that is, the code information matrix C, n'∈[2,N'], where N' is the total number of encoders in the encoding module.

[0126] The decoding module uses the coding information matrix C and the target sequence to obtain a prediction score of whether the cracks in the tower image belong to a certain type;

[0127] The decoding module is composed of multiple decoders connected in series; each decoder includes: a masked multi-head attention layer, a third Add&Norm layer, a multi-head attention layer, a fourth Add&Norm layer, a second feedforward control layer, and a fifth Add&Norm layer;

[0128] The input of the first decoder is the encoding information matrix C and the target sequence. The input of the m'th decoder is the output of the m'-1th decoder and the encoding information matrix C. The output of the last decoder in the decoding module (the M'th decoder) is the output of the entire decoding module, that is, the prediction score of whether the crack in the tower image belongs to a certain type. m'∈[2,M'], where M' is the total number of decoders.

[0129] The output module is used to output the probability that the cracks in the current tower image belong to a certain type and the predicted position of the crack target detection frame;

[0130] The output module includes: a linear layer and a softmax layer;

[0131] In this step, the multi-head attention mechanism layer, multi-head attention layer, and FeedForward layer in the encoding module and the decoding module are not the same layer, but different layers in their respective modules.

[0132] S2. Obtain an image of cracks in the foundation of the iron tower to be tested, input the image of cracks in the foundation of the iron tower to be tested into the trained ResNet network, and obtain a feature map of the image of cracks in the foundation of the iron tower to be tested;

[0133] S3, using the crack prediction type and crack target prediction frame position of the tower image data in the training set to form a query vector to be tested, inputting the query vector to be tested and the crack image feature map of the tower foundation to be tested into the trained tower foundation crack detection network, and obtaining the crack prediction type corresponding to the maximum probability in the tower foundation crack image to be tested and the crack target detection frame prediction position of the tower foundation to be tested, specifically:

[0134] First, the crack types in the tower image to be tested are randomly initialized, and the position of the crack target detection frame to be tested is randomly initialized;

[0135] Then, the crack prediction types and crack target prediction box positions of the tower image data in the training set are used to form a query sequence to be tested;

[0136] Finally, the feature map of the tower foundation crack image to be tested and the query sequence to be tested are input into the trained tower foundation crack detection network to obtain the crack category corresponding to the maximum probability in the tower foundation crack image to be tested and the predicted position of the crack target detection frame to be tested;

[0137] Specific implementation method 2: The input module uses the tower image feature map, the crack target detection frame position and the tower foundation crack type to obtain the final input sequence and target sequence, specifically:

[0138] A1. The input embedding layer preprocesses the tower image feature map to obtain the input sequence:

[0139] A1-1. Divide the tower image feature map into blocks according to a preset size (16*16) to obtain multiple feature blocks;

[0140] A1-2. Flatten the feature image block and input the flattened feature image block into the fully connected layer for linear mapping to obtain the feature vector;

[0141] A1-3. Combine all feature vectors to obtain a feature sequence, and add the class token to the feature sequence as the starting feature vector of the feature sequence to obtain the input sequence.

[0142] A2. The first position encoding layer adds position encoding to the input sequence to obtain the final input sequence:

[0143] A2-1. Get the position code of each feature vector in the input sequence:

[0144] The position encoding of the feature vector is composed by using the position encoding of each element in the feature vector;

[0145] The position encoding of each element in the feature vector is obtained by the following formula:

[0146]

[0147] Among them, pos is the feature vector number in the input sequence, d is the current feature vector dimension, i' is the dimension index in the current feature vector, PE (pos,2i') is the position code of the 2i'th element in the posth eigenvector, PE (pos,2i'+1) is the position code of the 2i'+1th element in the posth eigenvector, i'∈[0,d], d is an integer;

[0148] A2-2. Each eigenvector and the corresponding eigenvector position code are added together to form a sequence to obtain the final input sequence.

[0149] A3. The output embedding layer processes the crack target detection frame position and the tower foundation crack type label to obtain the target query sequence:

[0150] The crack target detection frame position corresponding to a crack in the tower image feature map and the type of crack in the tower foundation are combined into a target query vector;

[0151] A4. The second position encoding layer adds position encoding to the target query sequence to obtain the target sequence:

[0152] A4-1. Obtain the position encoding of each target query vector in the target query sequence:

[0153] Composing a positional encoding of the target query vector using the positional encoding of each element in the target query vector;

[0154] The method for obtaining the position encoding of each element in the target query vector is the same as the method for obtaining the position encoding of each element in the feature vector;

[0155] A4-2. Add each target query vector and the corresponding position code of the target query vector, and combine the summed results to obtain a target sequence.

[0156] Specific implementation method three: Figure 3 As shown, the encoding module is used to encode the final input sequence X to obtain the encoded final input sequence, namely the encoding information matrix C, which is specifically:

[0157] B1. Use the final input sequence X to obtain the local range attention score matrix of the pixel and the global context attention score matrix of the pixel:

[0158]

[0159] Where (i, j) is the coordinate of any pixel position, ρ(i, j) is a set of pixels in a sliding window of size k×k centered at (i, j), and ||ρ(i, j)||=k 2 , k is a constant, σ(X) represents the feature map obtained by X after passing through a single-layer neural network, pooling, and layer normalization in sequence (a single-layer neural network is used for projection and activation before feature map pooling, and layer normalization is used to normalize the output after pooling), ||σ(X)||=HpWp, Hp is the height of the feature map obtained by X after passing through a single-layer neural network, pooling, and layer normalization in sequence, Wp is the width of the feature map obtained by X after passing through a single-layer neural network, pooling, and layer normalization in sequence, S (i,j)～ρ(i,j) is the local range attention score of the pixel at position (i, j), S (i,j)～σ(X) is the global context attention score of the pixel at position (i, j), Q (i,j)is the pixel Query value at position (i, j), K ρ(i,j) is the Key value of ρ(i,j), It's K ρ(i,j) The weight matrix, K σ(X) is the Key value of σ(X), It's K σ(X) The weight matrix of

[0160] In this step, the scaled dot product attention method is used to calculate the similarity between the query and each key. The similarity score reflects the matching degree between the query and each key. T The similarity between the query and each key is calculated.

[0161] B2. Concatenate and normalize the pixel’s local range attention score matrix and the pixel’s global context attention score matrix to obtain the pixel attention weight:

[0162]

[0163] Among them, A (i,j) is the pixel attention weight at position (i, j), Concat() is the concatenation function, is a scaling factor, and B(i,j) is a bias term, which is usually a learnable parameter used to adjust the attention score.

[0164] In this step, after obtaining the attention score, the two attention score matrices are concatenated together through (scaling factor) is used to normalize the concatenated scores, and the Softmax function is applied to the concatenated attention score matrix to convert the similarity into a probability distribution.

[0165] B3. Use pixel attention weights to obtain the final input sequence attention value:

[0166]

[0167] PFA(X (i,j) )=A (i,j)～ρ(i,j) V ρ(i,j) +A (i,j)～ρ(X) V σ(X)

[0168]

[0169] Among them, i∈[1,M], j∈[1,N], M is the maximum value of the pixel horizontal coordinate, N is the maximum value of the pixel vertical coordinate, A (m,n) is the pixel attention weight at position (m,n), X (i,j)is the eigenvalue of position (i, j) in X, d k is the dimension of Query and key, PFA(X) is the final input sequence attention value (given by PFA(X) (i,j) ) composition), PFA(X (i,j) ) is X (i,j) The corresponding attention value, V ρ(i,j) is the Value associated with ρ(i,j), A (i,j)～ρ(i,j) Represents the attention weight A at position (i, j) (i,j) The relationship value with the pixel set ρ(i,j), A (i,j)～σ(X) Represents the attention weight A at position (i, j) (i,j) Relationship value with σ(X), V σ(X) Represents the Value value associated with σ(X), It is V ρ(i,j) The weight matrix, It is V σ(X) The weight matrix of .

[0170] B4. Use PFA(X) to obtain the output of the multi-head attention mechanism layer in the encoder:

[0171] MultiHead(Q,K,V)=Concat(head1,…,head h )W O

[0172] where head i* =Attention(QW i* Q ,KW i* K ,VW i* V )

[0173] Q=PFA(X)W Q

[0174] K=PFA(X)W K

[0175] V=PFA(X)W V

[0176] Where i*∈[1,h], i* is the number of attention heads in the multi-head attention mechanism layer in the encoder, h is the total number of attention heads in the multi-head attention mechanism layer in the encoder, is the query projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Key projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Value projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, W O is the output projection matrix, Q is the Query value of the current multi-head attention mechanism layer, K is the Key value of the current multi-head attention mechanism layer, V is the Value value of the current multi-head attention mechanism layer, and W Q is the weight matrix of Q, W K is the weight matrix of K, W V is the weight matrix of V;

[0177] B5. Input the output of the multi-head attention mechanism layer in the current encoder and the final input sequence into the first Add&Norm layer for residual connection and normalization. Then, input the output Y of the first Add&Norm layer into the first feedforward control layer, perform effective feature transformation at each position, input the transformed Y and Y into the second Add&Norm layer for residual connection and layer normalization to obtain the output sequence of the current encoder.

[0178] B6. Use the output sequence of the current encoder as the input sequence of the next encoder, and repeat B1-B4 until the last encoder outputs the encoded final input sequence, namely the encoding information matrix C.

[0179] In this step, most models use a mixture of high-level information and low-level information to achieve the purpose of accurately extracting image features. However, due to the depth degradation effect, many efficient models cannot form sufficient information mixing through stacking. Because through deep stacking, the network will lose key information when processing data layer by layer. And because each layer of data will form different features after processing, it is easy for the features generated by the previous layers to conflict with those generated by the next layers, or the captured edge information is not fine enough. The pixel-focused attention mechanism proposed in this step can effectively solve the above problems. The idea of ​​this attention mechanism is to have fine-grained perception near each query while maintaining coarse-grained awareness of global information, which can better extract the characteristics of cracks. Therefore, the encoder sequence of the present invention adopts a pixel-focused attention mechanism, which adopts a dual-path design, including fine-grained sliding window attention centered on the query and coarse-grained pooling attention. In addition, in order to ensure the effective mixing of fine-grained information and coarse-grained information, the mechanism uses the same Softmax to calculate the query-key similarity results of the two paths. This leads to competition between fine-grained and coarse-grained features, transforming the pixel-focused attention into a multi-scale attention mechanism.

[0180] Specific implementation method four: Figure 4As shown, the decoding module uses the coding information matrix C and the target sequence M to obtain the prediction score of the crack in the tower image belonging to a certain type, specifically:

[0181] C1. Input the target sequence M into the masked multi-head attention layer to obtain the masked attention output;

[0182] C2. Use the masked attention output and the target sequence M for residual connection and normalization to obtain the updated target sequence Z;

[0183] C3, input the encoded information matrix C and the updated target sequence Z into the multi-head attention layer of the decoder to obtain the multi-head attention layer output matrix of the decoder;

[0184] MultiHead(Q',K',V')=Concat(head1,head2,....,head h' )W' O

[0185]

[0186] K'=CW' K'

[0187] V'=CW' V'

[0188] Q'=ZW' Q'

[0189] Where i'∈[1,h'], i' is the number of attention heads in the multi-head attention layer in the decoder, h' is the total number of attention heads in the multi-head attention layer in the decoder, head i” is the output of the ith attention head, is the Query projection matrix of the i'th attention head, is the Key projection matrix of the i'th attention head, is the Value projection matrix of the i'th attention head, W' O is the output projection matrix, W' K' is the weight matrix of K', W' V' is the weight matrix of V', W' Q' is the weight matrix of Q', Q' is the Query value of the current multi-head attention layer, K' is the Key value of the current multi-head attention layer, and V' is the Value value of the current multi-head attention layer;

[0190] C4, use the multi-head attention layer output matrix of the decoder obtained in C3 and the matrix Z to perform residual connection and normalization to obtain the matrix Z', then input the matrix Z' into the second feedforward control layer, input the output matrix of the second feedforward control layer and the matrix Z' into the fifth Add&Norm layer to obtain the output matrix Z* of the current decoder;

[0191] C5. Use the output matrix Z* of the current decoder as the input matrix of the next decoder, and repeat C1-C4 until the last decoder outputs the prediction score of whether the cracks in the tower image belong to a certain type.

[0192] In the present invention, the processing flow of the decoder is similar to that of the encoder. The main difference is that the encoder's multi-head attention mechanism is a pixel-focused attention mechanism, while the decoder's multi-head attention is a self-attention mechanism. In the Transformer model's multi-head attention mechanism, the model processes all elements in the sequence at the same time, which may cause the model to obtain information about subsequent elements in the target sequence when processing a certain element, thereby obtaining future information. Once future information is used during model training, the performance of the evaluation model will be reduced in practical applications. Because the model allows the acquisition of future information, it will violate its proper autoregressive properties, causing it to directly obtain prediction results without going through a complete detection process, and will also cause overfitting, reduced generalization ability, and other problems, making it impossible to achieve the purpose of training the model. In order to solve this problem, the present invention uses two multi-head attention modules, including a masked multi-head attention module on the first layer and a multi-head attention module on the second layer. The masked multi-head attention module on the first layer uses a mask to prevent the decoder from seeing future information. That is, for a sequence, at the moment Time_step is t, the decoded output can only depend on the output before t, but cannot depend on the output after t.

[0193] The image of the crack in the tower foundation needs to obtain as many image features as possible due to the complex background or blurred target, so that the query vector evolves from the initial rough representation to a fine representation containing more contextual information and target features. The query operation performed by Basic Query is simple and direct, and does not involve complex joins, aggregations or filtering operations. This operation has a weak ability to capture image details and is not suitable for image detection with many details such as cracks. However, when facing images with complex backgrounds or blurred targets, Dense Query creates a comprehensive vector containing more information by splicing multiple query vectors. It can capture the details of the image more accurately, which helps to improve the model's feature extraction capabilities, so that it can still achieve accurate detection results even in complex backgrounds. Therefore, the Dense Query architecture is used for crack images. This architecture splices the queries generated by all the encoders that have passed through before with the query at this time in the feature dimension to generate a new query, and continues to repeat this operation until the end. The two architectures are compared. Figure 5(a)-Figure 5(b) shown.

[0194] As shown in Figure 5(a), Basic Query starts with an initial query q 0 At the beginning, the initial context information is C 0 , through a series of transformations or operations, which may include feature extraction, feature mapping or attention mechanism, the initial query is passed to D 1 ,,D 2 , D 3 , D 4 At each layer, the query is updated based on the information extracted from the data, and the context information is also updated accordingly. After processing through all layers, the final query is Contains information extracted from all levels. As shown in Figure 5(b), we also use an initial query q 0 At the beginning, the initial context information is C 0 , enter D 1 , D 2 , D 3 , D 4 , and output respectively At the same time, directly enter D 2 , D 3 , D 4 layer, respectively After D 1 The query vector obtained by layer processing and the initial query q 0 Enter D as input 2 , and directly enter D 3 , D 4 layer, respectively That is, the query obtained in each layer is input to the next few layers at the same time. After multi-layer query update and cross-layer fusion, the query is finally obtained. All queries are concatenated to obtain the final q. During this process, the context information is continuously updated.

[0195] The multi-head attention mechanism improves the expressiveness and performance of the model by executing multiple attention functions in parallel. In essence, multiple attention heads process the input in parallel and then splice the output together. The spliced ​​output and matrix W O Multiply to get the final output. The multi-head attention mechanism process is as follows:

[0196] Initialization: First, initialize the necessary parameters, including the weights of the query, key, and value matrices, and the number of heads in the multi-head attention. These weights will be used in subsequent linear transformations.

[0197] Linear transformation: linear transformation of the input query, key and value matrices. These linear transformations are achieved by multiplying with the corresponding weight matrices. The transformed matrices will be used in subsequent multi-head attention calculations:

[0198] Q=XW Q

[0199] K=XW K

[0200] V=XW V

[0201] Split and Project: Split the linearly transformed query, key, and value matrices into multiple heads. Each head has its own query, key, and value matrices. Then, the attention score is calculated independently in each head.

[0202] Scaling and Softmax: The attention scores of each head are scaled to avoid the problem of gradient disappearance or explosion. Then, the Softmax function is applied to normalize the attention scores so that the sum of the weights at each position is 1.

[0203] For each head, calculate the attention score:

[0204]

[0205] Apply the softmax function to obtain the normalized attention weights:

[0206] A=softmax(Attention_Scores)

[0207] The value matrix is ​​weighted summed using the normalized attention weights to obtain the output matrix of each head:

[0208] output_head i =A×V i

[0209] Splicing and linear transformation: Splice the output matrices of all heads together to form a large output matrix:

[0210] concat_heads=[output_head1,output_head2,…,output_head h ]

[0211] Then, perform a linear transformation on this output matrix to obtain the final output:

[0212] output = concat_heads_W O

[0213] Through the above process, the multi-head attention mechanism can take into account different subspaces when processing data, thereby obtaining richer information.

Claims

1. A tower foundation crack detection method based on pixel focused attention, characterized in that The specific process of the method is: S1. Using the tower image data set to obtain a training set, using the training set to train the tower foundation crack detection network, obtaining the trained tower foundation crack detection network and the crack prediction type and crack target prediction frame position of the tower image data in the training set; The tower foundation crack detection network includes: an input module, an encoding module, a decoding module and an output module; The encoding module is used to encode the final input sequence X to obtain the encoding information matrix C, which is specifically: B1. Use the final input sequence X to obtain the local range attention score matrix of the pixel and the global context attention score matrix of the pixel: ||ρ(i,j)||=k 2 ||σ(X)||=HpWp Where (i, j) is the coordinate of any pixel position, ρ(i, j) is a set of pixels in a sliding window of size k×k centered at (i, j), k is a constant, S (i,j)～ρ(i,j) is the local range attention score of the pixel at position (i, j), S (i,j)～σ(X) is the global context attention score of the pixel at position (i, j), Q (i,j) is the pixel Query value at position (i, j), K ρ(i,j) is the Key value of ρ(i,j), σ(X) represents the feature map obtained after X passes through a single-layer neural network, pooling, and layer normalization, K σ(X) is the Key value of σ(X), It's K ρ(i,j) The weight matrix, It's K σ(X) The weight matrix of , Hp is the height of the feature map obtained after X passes through a single-layer neural network and pooling, and Wp is the width of the feature map obtained after X passes through a single-layer neural network and pooling; B2. Pixel attention weights using the pixel’s local range attention score matrix and the pixel’s global context attention score matrix: Among them, A (i,j) is the pixel attention weight at position (i, j), Concat() is the concatenation function, is the scaling factor, B(i,j) is the bias term; B3. Use pixel attention weights to obtain the final input sequence attention value; B4. Use PFA(X) to obtain the output of the multi-head attention mechanism layer in the encoder; B5. Input the output of the multi-head attention mechanism layer in the current encoder and the final input sequence into the first Add&Norm layer for residual connection and normalization. Then input the output Y of the first Add&Norm layer into the first feedforward control layer. Then input the output of the first feedforward control layer and Y into the second Add&Norm layer for residual connection and layer normalization to obtain the output sequence of the current encoder. B6. Use the output sequence of the current encoder as the input sequence of the next encoder and repeat B1-B4 until the last encoder outputs the encoding information matrix C; S2. Obtain an image of cracks in the foundation of the iron tower to be tested, input the image of cracks in the foundation of the iron tower to be tested into the trained ResNet network, and obtain a feature map of the image of cracks in the foundation of the iron tower to be tested; S3, using the crack prediction type and crack target prediction frame position of the tower image data in the training set to form a query sequence to be tested, inputting the query sequence to be tested and the crack image feature map of the tower foundation to be tested into the trained tower foundation crack detection network, and obtaining the crack prediction type corresponding to the maximum probability in the tower foundation crack image to be tested and the crack target detection frame prediction position of the tower foundation to be tested; The types of cracks in the tower image to be tested include: surface cracks, deep cracks, and through cracks.

2. The tower foundation crack detection method based on pixel focused attention according to claim 1, characterized in that: In the step S1, a training set is obtained by using the tower image data set, and a tower foundation crack detection network is trained by using the training set to obtain a trained tower foundation crack detection network, which is obtained by the following method: Step 1: Obtain the tower image dataset; The tower image data set includes: tower image data and target detection frame position labels and crack type labels; The target detection frame position includes: the coordinates of the upper left corner and the lower right corner of the crack target detection frame; Step 2: Use the tower image dataset to obtain a training set, and use the training set to train the tower foundation crack detection network to obtain a trained tower foundation crack detection network, specifically: Step 21: Input the tower image data into the pre-trained ResNet network to obtain the tower image feature map, randomly initialize the tower foundation crack target detection frame position and the tower foundation crack type, and use the tower image feature map, the initialized tower foundation crack target detection frame position, the initialized tower foundation crack type, the target detection frame position label and the crack type label of the tower image data to form a training set; Step 22: Use the training set to train the tower foundation crack detection network to obtain a trained tower foundation crack detection network.

3. The method for detecting cracks in a tower foundation based on pixel-focused attention according to claim 2, wherein: The tower foundation crack detection network includes: an input module, an encoding module, a decoding module and an output module; The input module uses the tower image feature map, the crack target detection frame position and the tower foundation crack type to obtain the final input sequence and target sequence; The input module includes: an input embedding layer, a first position encoding layer, an output embedding layer, and a second position encoding layer; The input embedding layer is used to process the tower image feature map to obtain an input sequence; The first position encoding layer is used to add position codes to the input sequence to obtain a final input sequence X, and the final input sequence X is input into the encoding module; The output embedding layer is used to process the crack target detection frame position and the corresponding crack type label to obtain the target query sequence; The second position encoding layer is used to add position encoding to the target query sequence to obtain a target sequence, and the target sequence is input into the multi-head attention layer of each decoder in the decoding module; The encoding module is used to encode the final input sequence X to obtain the encoding information matrix C; The encoding module is composed of multiple encoders connected in series; Each encoder includes: a multi-head attention mechanism layer, the first Add&Norm layer, the first feedforward control layer, and the second Add&Norm layer; The first Add&Norm layer and the second Add&Norm layer both include: a residual connection layer and a layer normalization layer; the output of the residual connection layer is the input of the layer normalization layer; The input of the first encoder is the final input sequence X, the input of the n'th encoder is the output of the n'-1th encoder, and the output of the N'th encoder is the output of the encoding module, that is, the encoding information matrix C, n'∈[2,N'], N' is the total number of encoders in the encoding module; The decoding module uses the coding information matrix C and the target sequence to obtain a prediction score of whether the cracks in the tower image belong to a certain type; The decoding module is composed of a plurality of decoders connected in series; Each decoder includes: a masked multi-head attention layer, a third Add&Norm layer, a multi-head attention layer, a fourth Add&Norm layer, a second feedforward control layer, and a fifth Add&Norm layer; The third Add&Norm layer, the fourth Add&Norm layer, and the fifth Add&Norm layer have the same structure as the first Add&Norm layer; The input of the first decoder is the encoding information matrix C and the target sequence. The input of the m'th decoder is the output of the m'-1th decoder and the encoding information matrix C. The output of the M'th decoder is the output of the decoding module, that is, the prediction score of whether the crack in the tower image belongs to a certain type. m'∈[2,M'], where M' is the total number of decoders. The output module uses the prediction score of the crack in the tower image belonging to a certain category to output the probability that the crack in the current tower image belongs to a certain category and the predicted position of the crack target detection frame; The output module includes: a linear layer and a softmax layer in sequence.

4. The method for detecting cracks in a tower foundation based on pixel-focused attention according to claim 3, wherein: The input module uses the tower image feature map, the crack target detection frame position and the tower foundation crack type label to obtain the final input sequence and target sequence, specifically: A1. Preprocess the tower image feature map to obtain the input sequence: A1-1. Divide the tower image feature map into blocks according to a preset size to obtain multiple feature blocks; A1-2. Flatten the feature blocks and input the flattened feature blocks into the fully connected layer to obtain the feature vector. A1-3. Combine all feature vectors to obtain a feature sequence, and add the class token to the feature sequence as the starting feature vector of the feature sequence to obtain the input sequence. A2. Add position codes to the input sequence to obtain the final input sequence. A3. Process the crack target detection frame position and tower foundation crack type to obtain the target query sequence: The crack target detection box position corresponding to a crack in the tower image feature map and the type of crack in the tower foundation are combined into a target query vector. All target query vectors form a target query sequence. A4. Add a position code to the target query sequence to obtain the target sequence.

5. The method for detecting cracks in a tower foundation based on pixel-focused attention according to claim 4, wherein: The step A2 adds position codes to the input sequence to obtain the final input sequence, specifically: A2-1. Obtain the position code of each feature vector in the input sequence, specifically: The position encoding of the feature vector is composed by using the position encoding of each element in the feature vector; The position encoding of each element in the feature vector is obtained by the following formula: Among them, pos is the feature vector number in the input sequence, d is the current feature vector dimension, i' is the dimension index in the current feature vector, PE (pos,2i') is the position code of the 2i'th element in the posth eigenvector, PE (pos,2i'+1) is the position code of the 2i'+1th element in the posth eigenvector, i'∈[0,d], d is an integer; A2-2. Each eigenvector and the corresponding eigenvector position code are added together to form a sequence to obtain the final input sequence.

6. The tower foundation crack detection method based on pixel focused attention according to claim 5, characterized in that: The step A4 of adding a position code to the target query sequence to obtain the target sequence is specifically as follows: A4-1. Obtain the position encoding of each target query vector in the target query sequence: Composing a positional encoding of the target query vector using the positional encoding of each element in the target query vector; The method for obtaining the position encoding of each element in the target query vector is the same as the method for obtaining the position encoding of each element in the feature vector; A4-2. Each target query vector and the corresponding position code of the target query vector are added together to form a sequence to obtain a target sequence.

7. The method for detecting cracks in a tower foundation based on pixel-focused attention according to claim 6, wherein: The pixel attention weight is used in B3 to obtain the final input sequence attention value, specifically: PFA(X (i,j) )=A (i,j)～ρ(i,j) V ρ(i,j) +A (i,j)～σ(X) V σ(X) Among them, X (i,j) is the corresponding eigenvalue of position (i, j) in X, i∈[1,M], j∈[1,N], M is the maximum value of the pixel horizontal coordinate, N is the maximum value of the pixel vertical coordinate, PFA(X) is the final input sequence attention value, PFA(X) (i,j) ) is X (i,j) The corresponding attention value, A (i,j)～ρ(i,j) It's A (i,j) The relationship value with ρ(i,j), A (i,j)～σ(X) It's A (i,j) Relationship value with σ(X), V ρ(i,j) is the Value associated with ρ(i,j), V σ(X) A represents the Value value associated with σ(X). (m,n) is the pixel attention weight at position (m,n), d k It is the dimension of Query and key, It is V ρ(i,j) The weight matrix, It is V σ(X) The weight matrix of .

8. The tower foundation crack detection method based on pixel focused attention according to claim 7, characterized in that: The output of the multi-head attention mechanism layer in the encoder is obtained by using PFA(X) in B4, specifically: MultiHead(Q,K,V)=Concat(head1,…,head h )W O where head i* =Attention(QW i* Q ,KW i* K ,VW i* V ) Q=PFA(X)W Q K=PFA(X)W K V=PFA(X)W V Where i*∈[1,h], i* is the number of attention heads in the multi-head attention mechanism layer in the encoder, h is the total number of attention heads in the multi-head attention mechanism layer in the encoder, and head i* is the output of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the query projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Key projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Value projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, W O is the output projection matrix, Q is the Query value of the current multi-head attention mechanism layer, K is the Key value of the current multi-head attention mechanism layer, V is the Value value of the current multi-head attention mechanism layer, and W Q is the weight matrix of Q, W K is the weight matrix of K, W V is the weight matrix of V.

9. The tower foundation crack detection method based on pixel focused attention according to claim 8, characterized in that: The decoding module uses the encoding information matrix C and the target sequence to obtain the prediction score of whether the cracks in the tower image belong to a certain type, specifically: C1. Input the target sequence M into the masked multi-head attention layer to obtain the masked attention output; C2. Use the masked attention output and the target sequence M for residual connection and normalization to obtain the updated target sequence Z; C3, input the encoded information matrix C and the updated target sequence Z into the multi-head attention layer of the decoder to obtain the multi-head attention layer output matrix of the decoder; MultiHead(Q',K',V')=Concat(head1,head2,....,head h' )W' O K'=CW' K' V'=CW' V' Q'=ZW' Q' Where i'∈[1,h'], i' is the number of attention heads in the multi-head attention layer in the decoder, h' is the total number of attention heads in the multi-head attention layer in the decoder, head i” is the output of the i'th attention head, is the Query projection matrix of the i'th attention head, is the Key projection matrix of the i'th attention head, is the Value projection matrix of the i'th attention head, W' O is the output projection matrix, W' K' is the weight matrix of K', W' V' is the weight matrix of V', W' Q' is the weight matrix of Q', Q' is the Query value of the current multi-head attention layer, K' is the Key value of the current multi-head attention layer, and V' is the Value value of the current multi-head attention layer; C4, use the multi-head attention layer output matrix of the decoder obtained in C3 and the matrix Z to perform residual connection and normalization to obtain the matrix Z', then input the matrix Z' into the second feedforward control layer, input the output matrix of the second feedforward control layer and the matrix Z' into the fifth Add&Norm layer to obtain the output matrix Z* of the current decoder; C5. Use the output matrix Z* of the current decoder as the input matrix of the next decoder and repeat C1-C4 until the last decoder outputs the prediction score of whether the cracks in the tower image belong to a certain type.

Images