Iron tower foundation crack detection method based on pixel focusing attention

Abstract

The invention discloses an iron tower foundation crack detection method based on pixel focusing attention, and relates to the field of crack detection. The invention aims to solve the problem that the existing iron tower foundation crack detection accuracy is low. The method comprises the following steps: acquiring a training set by using an iron tower image data set, and training an iron tower foundation crack detection network by using the training set; inputting a to-be-detected iron tower foundation crack image into the trained ResNet network, and obtaining a to-be-detected iron tower foundation crack image feature map; and forming a to-be-detected query sequence by using the crack prediction type and the crack target prediction frame position of the iron tower image data in the training set, and inputting the to-be-detected query sequence and the to-be-detected iron tower foundation crack image feature map into the trained iron tower foundation crack detection network. And obtaining a crack prediction type and a corresponding probability in the to-be-detected iron tower foundation crack image and a prediction position of the to-be-detected iron tower foundation crack target detection frame. The method is used for detecting the iron tower foundation crack type.

Description

Technical Field

[0001] The invention relates to the field of crack detection, and in particular to a method for detecting cracks in an iron tower foundation based on pixel focused attention. Background Art

[0002] With the development of science and technology, people's demand for the Internet is no longer just satisfied with basic communication functions, and the pursuit of smooth and efficient user experience has gradually become a consensus. In order to meet people's increasing communication needs, my country is constantly expanding the number and scale of towers. As the world's largest communication tower infrastructure service provider, China Tower will operate and manage 2.046 million tower sites by the end of 2023. However, due to factors such as weather changes, material use, and construction quality, over time, the tower is prone to foundation cracking, forming various tower foundation cracks. Tower foundation cracks will affect the overall stability of the tower foundation, reduce the tower's bearing capacity for wind loads, equipment loads, etc., and even cause the tower to tilt or deform. If not handled in time, it may cause more serious safety problems, such as tower collapse or other structural accidents, posing a serious threat to the safety of surrounding personnel and equipment.

[0003] Traditional tower foundation crack detection methods use manual detection, ultrasonic detection, penetration detection, etc. These methods are prone to strong subjectivity, high cost, and complex processing. In order to solve the problems of traditional methods and efficiently detect and warn tower foundation cracks, a tower foundation crack detection method based on convolutional neural network (CNN) has been proposed in the field of artificial intelligence. Convolutional neural network (CNN) uses a fixed-size convolution kernel to extract features. However, this method has limitations when dealing with cracks with diversity and complex backgrounds. Most cracks have different shapes. CNN mainly extracts local features during training, but the location and size of cracks are mostly related to the global context. CNN has weak processing capabilities for global context information. This feature results in the existing methods not being able to recognize cracks of certain shapes or sizes, resulting in low accuracy in the existing tower foundation crack detection. Summary of the invention

[0004] The purpose of the present invention is to solve the problem of low accuracy of existing tower foundation crack detection, and propose a tower foundation crack detection method based on pixel focus attention.

[0005] A tower foundation crack detection method based on pixel focus attention, the specific process is as follows:

[0006] S1. Using the tower image data set to obtain a training set, using the training set to train the tower foundation crack detection network, obtaining the trained tower foundation crack detection network and the crack prediction type and crack target prediction frame position of the tower image data in the training set;

[0007] S2. Obtain a crack image of the iron tower foundation to be tested, input the crack image of the iron tower foundation to be tested into the trained ResNet network, and obtain a feature map of the crack image of the iron tower foundation to be tested;

[0008] S3, using the crack prediction type and crack target prediction frame position of the tower image data in the training set to form a query sequence to be tested, input the query sequence to be tested and the crack image feature map of the tower foundation to be tested into the trained tower foundation crack detection network, and obtain the crack prediction type corresponding to the maximum probability in the tower foundation crack image to be tested and the predicted position of the tower foundation crack target detection frame to be tested;

[0009] The types of cracks in the image of the iron tower to be tested include: surface cracks, deep cracks, and through cracks.

[0010] Furthermore, the step of obtaining a training set by using the tower image data set in S1, training the tower foundation crack detection network by using the training set, and obtaining a trained tower foundation crack detection network is performed by:

[0011] Step 1: Obtain a tower image dataset;

[0012] The tower image data set includes: tower image data and target detection frame position labels and crack type labels;

[0013] The target detection frame position includes: the upper left corner coordinates and the lower right corner coordinates of the crack target detection frame;

[0014] Step 2: Use the tower image data set to obtain a training set, use the training set to train the tower foundation crack detection network, and obtain a trained tower foundation crack detection network, specifically:

[0015] Step 21: Input the tower image data into the pre-trained ResNet network to obtain the tower image feature map, randomly initialize the tower foundation crack target detection frame position and the tower foundation crack type, and use the tower image feature map, the initialized tower foundation crack target detection frame position, the initialized tower foundation crack type, the tower image data target detection frame position label, and the crack type label to form a training set;

[0016] Step 22: Use the training set to train the tower foundation crack detection network to obtain a trained tower foundation crack detection network.

[0017] Furthermore, the tower foundation crack detection network includes: an input module, an encoding module, a decoding module, and an output module;

[0018] The input module uses the tower image feature map, the crack target detection frame position and the tower foundation crack type to obtain the final input sequence and the target sequence;

[0019] The input module includes: an input embedding layer, a first position encoding layer, an output embedding layer, and a second position encoding layer;

[0020] The input embedding layer is used to process the tower image feature map to obtain an input sequence;

[0021] The first position encoding layer is used to add position encoding to the input sequence to obtain a final input sequence X, and the final input sequence X is input into the encoding module;

[0022] The output embedding layer is used to process the crack target detection frame position and the corresponding crack type label to obtain a target query sequence;

[0023] The second position encoding layer is used to add position encoding to the target query sequence to obtain a target sequence, and the target sequence is input into the multi-head attention layer of each decoder in the decoding module;

[0024] The encoding module is used to encode the final input sequence X to obtain the encoding information matrix C;

[0025] The encoding module is composed of a plurality of encoders connected in series;

[0026] Each encoder includes: a multi-head attention mechanism layer, a first Add&Norm layer, a first feedforward control layer, and a second Add&Norm layer;

[0027] The first Add&Norm layer and the second Add&Norm layer both include: a residual connection layer and a layer normalization layer; the output of the residual connection layer is the input of the layer normalization layer;

[0028] The input of the first encoder is the final input sequence X, the input of the n'th encoder is the output of the n'-1th encoder, and the output of the N'th encoder is the output of the encoding module, that is, the encoding information matrix C, n'∈[2,N'], N' is the total number of encoders in the encoding module;

[0029] The decoding module uses the coding information matrix C and the target sequence to obtain a prediction score of whether the cracks in the tower image belong to a certain type;

[0030] The decoding module is composed of a plurality of decoders connected in series;

[0031] Each decoder includes: a masked multi-head attention layer, a third Add&Norm layer, a multi-head attention layer, a fourth Add&Norm layer, a second feedforward control layer, and a fifth Add&Norm layer;

[0032] The third Add&Norm layer, the fourth Add&Norm layer, and the fifth Add&Norm layer have the same structure as the first Add&Norm layer;

[0033] The input of the first decoder is the encoding information matrix C and the target sequence. The input of the m'th decoder is the output of the m'-1th decoder and the encoding information matrix C. The output of the M'th decoder is the output of the decoding module, that is, the prediction score of the crack in the tower image belonging to a certain category; m'∈[2,M'], M' is the total number of decoders;

[0034] The output module uses the prediction score of the cracks in the tower image belonging to a certain category to output the probability that the cracks in the current tower image belong to a certain category and the predicted position of the crack target detection frame;

[0035] The output module includes: a linear layer and a softmax layer in sequence.

[0036] Furthermore, the input module uses the tower image feature map, the crack target detection frame position and the tower foundation crack type label to obtain the final input sequence and target sequence, specifically:

[0037] A1. Preprocess the tower image feature map to obtain the input sequence:

[0038] A1-1, dividing the tower image feature map into blocks according to a preset size to obtain multiple feature map blocks;

[0039] A1-2. Flatten the feature block and input the flattened feature block into the fully connected layer to obtain the feature vector;

[0040] A1-3. Combine all feature vectors to obtain a feature sequence, and add the class token to the feature sequence as the starting feature vector of the feature sequence, thereby obtaining an input sequence.

[0041] A2. Add position code to the input sequence to obtain the final input sequence;

[0042] A3. Process the crack target detection frame position and the tower foundation crack type to obtain the target query sequence:

[0043] The position of the crack target detection box corresponding to a crack in the tower image feature map and the type of crack in the tower foundation are combined into a target query vector, and all target query vectors form a target query sequence;

[0044] A4. Add position code to the target query sequence to obtain the target sequence.

[0045] Furthermore, the position code is added to the input sequence in A2 to obtain the final input sequence, specifically:

[0046] A2-1. Get the position code of each feature vector in the input sequence, specifically:

[0047] The position encoding of the feature vector is composed by using the position encoding of each element in the feature vector;

[0048] The position encoding of each element in the feature vector is obtained by the following formula:

[0049]

[0050] Among them, pos is the feature vector number in the input sequence, d is the current feature vector dimension, i' is the dimension index in the current feature vector, PE (pos,2i') is the positional encoding of the 2i'th element in the posth eigenvector, PE (pos,2i'+1) is the position code of the 2i'+1th element in the posth feature vector, i'∈[0,d], d is an integer;

[0051] A2-2. Each eigenvector and the corresponding eigenvector position code are added together to form a sequence to obtain the final input sequence.

[0052] Furthermore, the step of adding a position code to the target query sequence in A4 to obtain the target sequence is specifically:

[0053] A4-1. Obtain the position encoding of each target query vector in the target query sequence:

[0054] Using the positional encoding of each element in the target query vector to form a positional encoding of the target query vector;

[0055] The method for obtaining the position encoding of each element in the target query vector is the same as the method for obtaining the position encoding of each element in the feature vector;

[0056] A4-2. Each target query vector and the position code of the corresponding target query vector are added together to form a sequence to obtain a target sequence.

[0057] Furthermore, the encoding module is used to encode the final input sequence X to obtain the encoding information matrix C, which is specifically:

[0058] B1. Use the final input sequence X to obtain the local range attention score matrix of the pixel and the global context attention score matrix of the pixel:

[0059]

[0060] ||ρ(i,j)||=k 2

[0061] ||σ(X)||=HpWp

[0062] Where (i, j) is the coordinate of any pixel position, ρ(i, j) is a set of pixels in a sliding window of size k×k centered at (i, j), k is a constant, and S (i,j)～ρ(i,j) is the local range attention score of the pixel at position (i, j), S (i,j)～σ(X) is the global context attention score of the pixel at position (i, j), Q (i,j) is the pixel Query value at position (i, j), K ρ(i,j) is the Key value of ρ(i,j), σ(X) represents the feature map obtained after X passes through a single-layer neural network, pooling, and layer normalization in sequence, K σ(X) is the Key value of σ(X), It's K ρ(i,j) The weight matrix of It's K σ(X) The weight matrix of , Hp is the height of the feature map obtained after X passes through a single-layer neural network and pooling in sequence, and Wp is the width of the feature map obtained after X passes through a single-layer neural network and pooling in sequence;

[0063] B2. Pixel attention weights using the local range attention score matrix of the pixel and the global context attention score matrix of the pixel:

[0064]

[0065] Among them, A (i,j) is the pixel attention weight at position (i, j), Concat() is the concatenation function, is the scaling factor, B(i,j) is the bias term;

[0066] B3. Use pixel attention weights to obtain the final input sequence attention value;

[0067] B4. Use PFA(X) to obtain the output of the multi-head attention mechanism layer in the encoder;

[0068] B5. Input the output of the multi-head attention mechanism layer in the current encoder and the final input sequence into the first Add&Norm layer for residual connection and normalization. Then input the output Y of the first Add&Norm layer into the first feedforward control layer. Then input the output of the first feedforward control layer and Y into the second Add&Norm layer for residual connection and layer normalization to obtain the output sequence of the current encoder.

[0069] B6. Use the output sequence of the current encoder as the input sequence of the next encoder, and repeat B1-B4 until the last encoder outputs the encoding information matrix C.

[0070] Furthermore, the pixel attention weight is used in B3 to obtain the final input sequence attention value, specifically:

[0071]

[0072] Among them, X (i,j) is the corresponding eigenvalue of position (i, j) in X, i∈[1,M], j∈[1,N], M is the maximum value of the pixel horizontal coordinate, N is the maximum value of the pixel vertical coordinate, PFA(X) is the final input sequence attention value, PFA(X) (i,j) ) is X (i,j) The corresponding attention value, A (i,j)～ρ(i,j) Yes A (i,j) The relationship value with ρ(i,j), A (i,j)～σ(X) Yes A (i,j) Relationship with σ(X), V ρ(i,j) is the Value associated with ρ(i,j), V σ(i) A represents the Value associated with σ(X). (m,n) is the pixel attention weight at position (m,n), d k It is the dimension of Query and key. Yes V ρ(i,j) The weight matrix of Yes V σ(X) The weight matrix of .

[0073] Furthermore, the output of the multi-head attention mechanism layer in the encoder is obtained by using PFA(X) in B4, specifically:

[0074] MultiHead(Q,K,V)=Concat(head1,…,head h )W O

[0075]

[0076] Q=PFA(X)W Q

[0077] K=PFA(X)W K

[0078] V=PFA(X)W V

[0079] Where i*∈[1,h], i* is the number of attention heads in the multi-head attention mechanism layer in the encoder, h is the total number of attention heads in the multi-head attention mechanism layer in the encoder, is the output of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Query projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Key projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Value projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, W O is the output projection matrix, Q is the Query value of the current multi-head attention mechanism layer, K is the Key value of the current multi-head attention mechanism layer, V is the Value value of the current multi-head attention mechanism layer, and W Q is the weight matrix of Q, W K is the weight matrix of K, W V is the weight matrix of V.

[0080] Furthermore, the decoding module uses the coding information matrix C and the target sequence to obtain a prediction score of whether the cracks in the tower image belong to a certain type, specifically:

[0081] C1. Input the target sequence M into the masked multi-head attention layer to obtain the masked attention output;

[0082] C2, use the masked attention output and the target sequence M to perform residual connection and normalization to obtain the updated target sequence Z;

[0083] C3, input the encoded information matrix C and the updated target sequence Z into the multi-head attention layer of the decoder to obtain the multi-head attention layer output matrix of the decoder;

[0084] MultiHead(Q',K',V')=Concat(head1,head2,....,head h' )W' O

[0085]

[0086] K'=CW' K'

[0087] V'=CW' V'

[0088] Q'=ZW' Q'

[0089] Where i”∈[1,h'], i” is the number of the attention head in the multi-head attention layer in the decoder, h' is the total number of attention heads in the multi-head attention layer in the decoder, and head i” is the output of the i-th attention head, is the Query projection matrix of the i-th attention head, is the Key projection matrix of the i'th attention head, is the Value projection matrix of the i'th attention head, W' O is the output projection matrix, W' K' is the weight matrix of K', W' V' is the weight matrix of V', W' Q' is the weight matrix of Q', Q' is the Query value of the current multi-head attention layer, K' is the Key value of the current multi-head attention layer, and V' is the Value value of the current multi-head attention layer;

[0090] C4, use the multi-head attention layer output matrix of the decoder obtained in C3 and the matrix Z to perform residual connection and normalization to obtain the matrix Z', then input the matrix Z' into the second feedforward control layer, input the output matrix of the second feedforward control layer and the matrix Z' into the fifth Add&Norm layer to obtain the output matrix Z* of the current decoder;

[0091] C5. Use the output matrix Z* of the current decoder as the input matrix of the next decoder, and repeat C1-C4 until the last decoder outputs the prediction score of whether the cracks in the tower image belong to a certain category.

[0092] The beneficial effects of the present invention are:

[0093] The invention improves the attention mechanism of the encoder on the basis of the Transformer network architecture and applies it to the task of detecting and classifying cracks in the iron tower foundation. The invention classifies the cracks in the iron tower foundation through a real data set and divides the cracks in the iron tower foundation into three types: surface cracks, deep cracks and through cracks, thereby reducing the labor cost of identifying cracks in the iron tower foundation and improving the speed of detecting cracks in the iron tower foundation. The encoding module of the invention adopts a pixel-focused attention mechanism to combine coarse-grained information with fine-grained information, thereby improving the overall understanding of the image of cracks in the iron tower foundation, and the fine-grained analysis improves the ability to identify cracks in the iron tower foundation, thereby improving the accuracy of detecting cracks in the iron tower foundation. The encoder of the invention adopts a Dense Query architecture and can create a comprehensive vector containing more information by splicing multiple query vectors. Therefore, the invention can still achieve accurate feature extraction in the face of images with complex backgrounds, thereby learning more complex features and greatly improving the accuracy of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 This is the Tansformer architecture diagram;

[0095] Figure 2(a) is a physical picture of the cracks on the surface of the tower foundation;

[0096] Figure 2(b) is a physical picture of the deep cracks in the tower foundation;

[0097] Figure 2(c) is a physical picture of the through cracks in the tower foundation;

[0098] Figure 3 Flowchart of the pixel-focused attention mechanism;

[0099] Figure 4 This is a diagram of a multi-head attention model;

[0100] Figure 5(a) is the Basic Query architecture diagram;

[0101] Figure 5(b) is the Dense Query architecture diagram. DETAILED DESCRIPTION

[0102] Specific implementation method 1: This implementation method is a method for detecting cracks in a tower foundation based on pixel focus attention. The specific process is as follows:

[0103] S1. Using the tower image data set to obtain a training set, using the training set to train the tower foundation crack detection network, obtaining the trained tower foundation crack detection network and the crack prediction type and crack target prediction frame position of the tower image data in the training set;

[0104] The trained tower foundation crack detection network is obtained by:

[0105] Step 1: Obtain a tower image dataset;

[0106] The tower image data set includes: tower image data and target detection frame position labels and crack type labels;

[0107] The position of the crack target detection frame of the tower foundation includes: the coordinates of the upper left corner and the lower right corner of the crack target detection frame;

[0108] The tower image dataset uses real tower data, including 1,500 towers in Heilongjiang Province, with a total of more than 30,000 images, of which about 10,000 images are of tower foundation cracks. All tower foundation cracks in the dataset are divided into three types: 1) surface cracks 2) deep cracks 3) through cracks, such as Figure 2(a)-Figure 2(c) shown.

[0109] Among them, surface cracks are shallow and mostly confined to the outer surface or surface layer, and have not penetrated into the main structure, which will not have a significant impact on the overall tower structure; deep cracks have begun to affect the main structure of the tower, and such cracks will reduce the overall stability of the structure; through cracks penetrate all layers of the tower foundation structure, seriously affecting the stability of the structure. The three types of cracks have different degrees of severity, and their warning levels and corresponding repair treatments should be effectively classified. Therefore, when the model identifies surface cracks, the impact on the tower structure is small, and the crack warning level is set as mild, which requires timely repair. When the model identifies deep cracks, there is a certain impact on the tower structure, and the crack warning level is set as moderate, and a professional evaluation is required to determine the repair plan. When the model identifies through cracks, it seriously endangers the stability of the tower, and the crack warning level is set as severe, requiring emergency treatment and reconstruction or strengthening of the structure to avoid the risk of collapse. These three types of tower foundation crack example pictures are as follows Figure 2(a)-Figure 2(c) shown.

[0110] Step 2: Use the tower image data set to obtain a training set, use the training set to train the tower foundation crack detection network, and obtain a trained tower foundation crack detection network, specifically:

[0111] Step 21: Input the tower image data into the pre-trained ResNet network to obtain the tower image feature map, randomly initialize the tower foundation crack target detection frame position and the tower foundation crack type, and use the tower image feature map, the initialized tower foundation crack target detection frame position, the initialized tower foundation crack type, the tower image data target detection frame position label, and the crack type label to form a training set;

[0112] Step 22: Use the training set to train the tower foundation crack detection network to obtain a trained tower foundation crack detection network;

[0113] like Figure 1 As shown, the tower foundation crack detection network is an improved Transformer network architecture, including: an input module, an encoding module, a decoding module, and an output module;

[0114] The input module uses the tower image feature map, the crack target detection frame position and the tower foundation crack type to obtain the final input sequence and the target sequence;

[0115] The input module includes: an input embedding layer Input Embedding, a first position encoding layer PositionalEncoding, an output embedding layer Output Embedding, and a second position encoding layer Positional Encoding;

[0116] The input embedding layer is used to process the tower image feature map to obtain an input sequence;

[0117] The first position encoding layer is used to add position encoding to the input sequence to obtain a final input sequence X, and the final input sequence is input into the encoding module;

[0118] The output embedding layer is used to process the position of the crack target detection frame and the type of cracks in the tower foundation to obtain a target query sequence;

[0119] The second position encoding layer is used to add position encoding to the target query sequence to obtain a target sequence, and the target sequence is input into the multi-head attention layer of each decoder in the decoding module;

[0120] The encoding module is used to encode the final input sequence X to obtain the encoded final input sequence, i.e., the encoding information matrix C;

[0121] The encoding module is composed of multiple encoders connected in series; each encoder includes: a multi-head attention mechanism layer Multi-Head Attention, a first Add&Norm layer, a first feedforward control layer Feed Forward, and a second Add&Norm layer;

[0122] The Add&Norm layer includes: a residual connection layer, a layer normalization layer, and a layer normalization layer. The output of the residual connection layer is the input of the layer normalization layer.

[0123] The input of the first encoder is the final input sequence X, the input of the n'th encoder is the output of the n'-1th encoder, and the output of the last encoder is the output of the encoding module, that is, the code information matrix C, n'∈[2,N'], N' is the total number of encoders in the encoding module.

[0124] The decoding module uses the coding information matrix C and the target sequence to obtain a prediction score of whether the cracks in the tower image belong to a certain type;

[0125] The decoding module is composed of a plurality of decoders connected in series; each decoder includes: a masked multi-head attention layer, a third Add&Norm layer, a multi-head attention layer, a fourth Add&Norm layer, a second feedforward control layer, and a fifth Add&Norm layer;

[0126] The input of the first decoder is the encoding information matrix C and the target sequence. The input of the m'th decoder is the output of the m'-1th decoder and the encoding information matrix C. The output of the last decoder (M'th decoder) in the decoding module is the output of the entire decoding module, that is, the prediction score of the crack in the tower image belonging to a certain category; m'∈[2,M'], M' is the total number of decoders;

[0127] The output module is used to output the probability that the cracks in the current tower image belong to a certain type and the predicted position of the crack target detection frame;

[0128] The output module includes: a linear layer and a softmax layer;

[0129] In this step, the multi-head attention mechanism layer, multi-head attention layer, and FeedForward layer in the encoding module and the decoding module are not the same layer, but different layers in their respective modules.

[0130] S2. Obtain a crack image of the iron tower foundation to be tested, input the crack image of the iron tower foundation to be tested into the trained ResNet network, and obtain a feature map of the crack image of the iron tower foundation to be tested;

[0131] S3, using the crack prediction type and crack target prediction frame position of the tower image data in the training set to form a query vector to be tested, input the query vector to be tested and the crack image feature map of the tower foundation to be tested into the trained tower foundation crack detection network, and obtain the crack prediction type corresponding to the maximum probability in the crack image of the tower foundation to be tested and the predicted position of the crack target detection frame of the tower foundation to be tested, specifically:

[0132] First, the crack types in the tower image to be tested are randomly initialized, and the position of the target detection frame of the crack to be tested is randomly initialized;

[0133] Then, the crack prediction types and crack target prediction box positions of the tower image data in the training set are used to form a query sequence to be tested;

[0134] Finally, the feature map of the tower foundation crack image to be tested and the query sequence to be tested are input into the trained tower foundation crack detection network to obtain the crack category corresponding to the maximum probability in the tower foundation crack image to be tested and the predicted position of the crack target detection frame to be tested;

[0135] Specific implementation method 2: The input module uses the tower image feature map, the crack target detection frame position and the tower foundation crack type to obtain the final input sequence and target sequence, specifically:

[0136] A1. The input embedding layer preprocesses the tower image feature map to obtain the input sequence:

[0137] A1-1, dividing the tower image feature map into blocks according to a preset size (16*16) to obtain multiple feature map blocks;

[0138] A1-2. Flatten the feature block and input the flattened feature block into the fully connected layer for linear mapping to obtain the feature vector.

[0139] A1-3. Combine all feature vectors to obtain a feature sequence, and add the class token to the feature sequence as the starting feature vector of the feature sequence, thereby obtaining an input sequence.

[0140] A2. The first position encoding layer adds position encoding to the input sequence to obtain the final input sequence:

[0141] A2-1. Get the position code of each feature vector in the input sequence:

[0142] The position encoding of the feature vector is composed by using the position encoding of each element in the feature vector;

[0143] The position encoding of each element in the feature vector is obtained by the following formula:

[0144]

[0145] Among them, pos is the feature vector number in the input sequence, d is the current feature vector dimension, i' is the dimension index in the current feature vector, PE (pos,2i') is the positional encoding of the 2i'th element in the posth eigenvector, PE (pos,2i'+1) is the position code of the 2i'+1th element in the posth feature vector, i'∈[0,d], d is an integer;

[0146] A2-2. Each eigenvector and the corresponding eigenvector position code are added together to form a sequence to obtain the final input sequence.

[0147] A3. The output embedding layer processes the crack target detection frame position and the tower foundation crack type label to obtain the target query sequence:

[0148] The crack target detection frame position corresponding to a crack in the tower image feature map and the type of crack in the tower foundation are combined into a target query vector;

[0149] A4. The second position encoding layer adds position encoding to the target query sequence to obtain the target sequence:

[0150] A4-1. Obtain the position encoding of each target query vector in the target query sequence:

[0151] Using the positional encoding of each element in the target query vector to form a positional encoding of the target query vector;

[0152] The method for obtaining the position encoding of each element in the target query vector is the same as the method for obtaining the position encoding of each element in the feature vector;

[0153] A4-2. Add each target query vector and the position code of the corresponding target query vector, and combine the summed results to obtain a target sequence.

[0154] Specific implementation method three: Figure 3 As shown, the encoding module is used to encode the final input sequence X to obtain the encoded final input sequence, that is, the encoding information matrix C, which is specifically:

[0155] B1. Use the final input sequence X to obtain the local range attention score matrix of the pixel and the global context attention score matrix of the pixel:

[0156]

[0157]

[0158] Where (i, j) is the coordinate of any pixel position, ρ(i, j) is a set of pixels in a sliding window of size k×k centered at (i, j), and ||ρ(i, j)||=k 2 , k is a constant, σ(X) represents the feature map obtained after X passes through a single-layer neural network, pooling, and layer normalization in sequence (a single-layer neural network is used for projection and activation before feature map pooling, and layer normalization is used to normalize the output after pooling), ||σ(X)||=HpWp, Hp is the height of the feature map obtained after X passes through a single-layer neural network, pooling, and layer normalization in sequence, Wp is the width of the feature map obtained after X passes through a single-layer neural network, pooling, and layer normalization in sequence, S (i,j)～ρ(i,j) is the local range attention score of the pixel at position (i, j), S (i,j)～σ(X) is the global context attention score of the pixel at position (i, j), Q (i，j) is the pixel Query value at position (i, j), K ρ(i,j) is the Key value of ρ(i,j), It's K ρ(i,j) The weight matrix, K σ(X) is the Key value of σ(X), It's K σ(X) The weight matrix of

[0159] In this step, the scaled dot product attention method is used to calculate the similarity between the query and each key. The similarity score reflects the matching degree between the query and each key. TThe similarity between the Query and each Key is calculated.

[0160] B2. Concatenate and normalize the local range attention score matrix of the pixel and the global context attention score matrix of the pixel to obtain the pixel attention weight:

[0161]

[0162] Among them, A (i,j) is the pixel attention weight at position (i, j), Concat() is the concatenation function, is a scaling factor and B(i,j) is a bias term, usually a learnable parameter, used to adjust the attention score.

[0163] In this step, after obtaining the attention score, the two attention score matrices are concatenated together through (Scaling factor) is used to normalize the concatenated scores, and the Softmax function is applied to the concatenated attention score matrix to convert the similarity into a probability distribution.

[0164] B3. Use pixel attention weights to obtain the final input sequence attention value:

[0165]

[0166] PFA(X (i,j) )=A (i,j)～ρ(i,j) V ρ(i,j) +A (i,j)～σ(X) V σ(X)

[0167]

[0168] Among them, i∈[1,M], j∈[1,N], M is the maximum value of the pixel horizontal coordinate, N is the maximum value of the pixel vertical coordinate, A (m,n) is the pixel attention weight at position (m,n), X (i,j) is the eigenvalue of position (i,j) in X, d k is the dimension of Query and key, PFA(X) is the final input sequence attention value (given by PFA(X) (i,j) ) composition), PFA(X (i,j) ) is X (i,j) The corresponding attention value, V ρ(i,j) is the Value associated with ρ(i,j), A (i,j)～ρ(i,j) Represents the attention weight A at position (i, j) (i,j) The relationship value with the pixel set ρ(i,j), A (i,j)～σ(X) Represents the attention weight A at position (i, j)(i,j) Relationship with σ(X), V σ(X) represents the Value associated with σ(X), Yes V ρ(i,j) The weight matrix of Yes V σ(X) The weight matrix of .

[0169] B4. Use PFA(X) to obtain the output of the multi-head attention mechanism layer in the encoder:

[0170] MultiHead(Q,K,V)=Concat(head1,…,head h )W O

[0171]

[0172] Q=PFA(X)W Q

[0173] K=PFA(X)W K

[0174] V=PFA(X)W V

[0175] Where i*∈[1,h], i* is the number of attention heads in the multi-head attention mechanism layer in the encoder, h is the total number of attention heads in the multi-head attention mechanism layer in the encoder, is the Query projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Key projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Value projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, W O is the output projection matrix, Q is the Query value of the current multi-head attention mechanism layer, K is the Key value of the current multi-head attention mechanism layer, V is the Value value of the current multi-head attention mechanism layer, and W Q is the weight matrix of Q, W K is the weight matrix of K, W V is the weight matrix of V;

[0176] B5. Input the output of the multi-head attention mechanism layer in the current encoder and the final input sequence into the first Add&Norm layer for residual connection and normalization. Then input the output Y of the first Add&Norm layer into the first feedforward control layer, perform effective feature transformation at each position, input the transformed Y and Y into the second Add&Norm layer for residual connection and layer normalization to obtain the output sequence of the current encoder.

[0177] B6. Use the output sequence of the current encoder as the input sequence of the next encoder, and repeat B1-B4 until the last encoder outputs the final encoded input sequence, i.e., the encoding information matrix C.

[0178] In this step, most models use a mixture of high-level information and low-level information to achieve the purpose of accurately extracting image features. However, due to the depth degradation effect, many efficient models cannot form enough information mixing through stacking. Because through deep stacking, the network will lose key information when processing data layer by layer. And because each layer of data will form different features after processing, it is easy to cause the features generated by the previous layers and the following layers to conflict, or the captured edge information is not fine enough, and the pixel-focused attention mechanism proposed in this step can effectively solve the above problems. The idea of ​​this attention mechanism has fine-grained perception near each Query, while maintaining the coarse-grained awareness of global information, which can better extract the characteristics of cracks. Therefore, the Encoder sequence of the present invention adopts a pixel-focused attention mechanism, which adopts a dual-path design, including fine-grained sliding window attention centered on Query and coarse-grained pooling attention. In addition, in order to ensure the effective mixing of fine-grained information and coarse-grained information, the mechanism uses the same Softmax to calculate the Query-Key similarity results of the two paths. This leads to competition between fine-grained and coarse-grained features, and transforms pixel-focused attention into a multi-scale attention mechanism.

[0179] Specific implementation method four: Figure 4 As shown, the decoding module uses the coding information matrix C and the target sequence M to obtain the prediction score of the cracks in the tower image belonging to a certain type, specifically:

[0180] C1. Input the target sequence M into the masked multi-head attention layer to obtain the masked attention output;

[0181] C2, use the masked attention output and the target sequence M to perform residual connection and normalization to obtain the updated target sequence Z;

[0182] C3, input the encoded information matrix C and the updated target sequence Z into the multi-head attention layer of the decoder to obtain the multi-head attention layer output matrix of the decoder;

[0183] MultiHead(Q',K',V')=Concat(head1,head2,....,head h' )W' O

[0184]

[0185] K'=CW' K'

[0186] V'=CW' V'

[0187] Q'=ZW' Q'

[0188] Where i”∈[1,h'], i” is the number of the attention head in the multi-head attention layer in the decoder, h' is the total number of attention heads in the multi-head attention layer in the decoder, and head i” is the output of the i-th attention head, is the Query projection matrix of the i-th attention head, is the Key projection matrix of the i'th attention head, is the Value projection matrix of the i'th attention head, W' O is the output projection matrix, W' K' is the weight matrix of K', W' V' is the weight matrix of V', W' Q' is the weight matrix of Q', Q' is the Query value of the current multi-head attention layer, K' is the Key value of the current multi-head attention layer, and V' is the Value value of the current multi-head attention layer;

[0189] C4, use the multi-head attention layer output matrix of the decoder obtained in C3 and the matrix Z to perform residual connection and normalization to obtain the matrix Z', then input the matrix Z' into the second feedforward control layer, input the output matrix of the second feedforward control layer and the matrix Z' into the fifth Add&Norm layer to obtain the output matrix Z* of the current decoder;

[0190] C5. Use the output matrix Z* of the current decoder as the input matrix of the next decoder, and repeat C1-C4 until the last decoder outputs the prediction score of whether the cracks in the tower image belong to a certain category.

[0191] In the present invention, the processing flow of the decoder Decoder is similar to that of the encoder Encoder. The main difference is that the encoder's multi-head attention mechanism is a pixel-focused attention mechanism, and the decoder's multi-head attention is a self-attention mechanism. In the multi-head attention mechanism of the Transformer model, the model processes all elements in the sequence at the same time, causing the model to obtain information about subsequent elements in the target sequence when processing a certain element, thereby obtaining future information. Once future information is used during model training, the performance of the evaluation model will be reduced in practical applications. Because the model allows the acquisition of future information, it will violate its due autoregressive properties, so that it can directly obtain the prediction results without going through the complete detection process, and will also produce overfitting, reduced generalization ability and other problems, and the purpose of the training model cannot be achieved. In order to solve this problem, the present invention uses two multi-head attention modules, including a masked multi-head attention module of the first layer and a multi-head attention module of the second layer. The masked multi-head attention module of the first layer uses Mask to prevent Decoder from seeing future information. That is, for a sequence, at the moment when Time_step is t, the decoded output can only depend on the output before time t, but cannot depend on the output after time t.

[0192] The image of the crack in the tower foundation needs to obtain as many image features as possible due to the complex background or blurred target, so that the query vector evolves from the initial rough representation to a fine representation containing more contextual information and target features. The query operation performed by Basic Query is simple and direct, and does not involve complex joins, aggregations or filtering operations. This operation has a weak ability to capture image details and is not suitable for image detection with more details such as cracks. However, when facing images with complex backgrounds or blurred targets, Dense Query creates a comprehensive vector containing more information by splicing multiple query vectors. It can capture the details of the image more accurately, which helps to improve the model's ability to extract features, so that it can still achieve accurate detection results even in complex backgrounds. Therefore, the Dense Query architecture is used for crack images. This architecture splices the queries generated by all the encoders that have passed through before with the query at this time in the feature dimension to generate a new query, and continues to repeat this operation until the end. The two architectures are compared. Figure 5(a)-Figure 5(b) shown.

[0193] As shown in Figure 5(a), Basic Query starts with an initial query q 0 At the beginning, the initial context information is C 0, through a series of transformations or operations, which may include feature extraction, feature mapping, or attention mechanisms, the initial query is passed to D in turn 1 ,,D 2 , D 3 , D 4 At each layer, the query is updated based on the information extracted from the data, and the context information is also updated. After being processed by all layers, the final query is contains information extracted from all levels. As shown in Figure 5(b), we also use an initial query q 0 At the beginning, the initial context information is C 0 , enter D 1 , D 2 , D 3 , D 4 , and output respectively Directly input D 2 , D 3 , D 4 layer, respectively After D 1 The query vector obtained by layer processing and the initial query q 0 As input into D 2 , while directly inputting D 3 , D 4 layer, respectively That is, the query obtained in each layer is simultaneously input into the following layers. After multi-layer query update and cross-layer fusion, the query is finally obtained. All queries are concatenated to get the final q. During this process, the context information is continuously updated.

[0194] The multi-head attention mechanism improves the expressiveness and performance of the model by executing multiple attention functions in parallel. In essence, multiple attention heads process the input in parallel and then concatenate the outputs together. The concatenated output and matrix W O Multiply them together to get the final output result. The multi-head attention mechanism process is as follows:

[0195] Initialization: First, initialize the necessary parameters, including the weights of the query, key, and value matrices, and the number of heads in the multi-head attention. These weights will be used in subsequent linear transformations.

[0196] Linear transformation: linear transformation of the input query, key and value matrices. These linear transformations are achieved by multiplying with the corresponding weight matrices. The transformed matrices will be used in subsequent multi-head attention calculations:

[0197] Q=XW Q

[0198] K=XW K

[0199] V=XW V

[0200] Split and Project: Split the linearly transformed query, key, and value matrices into multiple heads. Each head has its own query, key, and value matrices. Then, the attention scores are calculated independently in each head.

[0201] Scaling and Softmax: The attention scores of each head are scaled to avoid the problem of vanishing or exploding gradients. Then, the Softmax function is applied to normalize the attention scores so that the sum of the weights at each position is 1.

[0202] For each head, calculate the attention score:

[0203]

[0204] Apply the softmax function to obtain the normalized attention weights:

[0205] A = softmax(Attention_Scores)

[0206] The value matrix is ​​weighted summed using the normalized attention weights to obtain the output matrix of each head:

[0207] output_head i =A×V i

[0208] Concatenation and linear transformation: Concatenate the output matrices of all heads together to form a large output matrix:

[0209] concat_heads=[output_head1,output_head2,…,output_head h ]

[0210] Then, this output matrix is ​​linearly transformed to obtain the final output:

[0211] output = concat_heads_W O

[0212] Through the above process, the multi-head attention mechanism can take into account different subspaces when processing data, thereby obtaining richer information.

Claims

1. A tower foundation crack detection method based on pixel focus attention, characterized in that The specific process of the method is: S1. Using the tower image data set to obtain a training set, using the training set to train the tower foundation crack detection network, obtaining the trained tower foundation crack detection network and the crack prediction type and crack target prediction frame position of the tower image data in the training set; S2. Obtain a crack image of the iron tower foundation to be tested, input the crack image of the iron tower foundation to be tested into the trained ResNet network, and obtain a feature map of the crack image of the iron tower foundation to be tested; S3, using the crack prediction type and crack target prediction frame position of the tower image data in the training set to form a query sequence to be tested, input the query sequence to be tested and the crack image feature map of the tower foundation to be tested into the trained tower foundation crack detection network, and obtain the crack prediction type corresponding to the maximum probability in the tower foundation crack image to be tested and the predicted position of the tower foundation crack target detection frame to be tested; The types of cracks in the image of the iron tower to be tested include: surface cracks, deep cracks, and through cracks.

2. A tower foundation crack detection method based on pixel focus attention according to claim 1, characterized in that: In the step S1, a training set is obtained by using the tower image data set, and a tower foundation crack detection network is trained by using the training set to obtain a trained tower foundation crack detection network, which is obtained by the following method: Step 1: Obtain a tower image dataset; The tower image data set includes: tower image data and target detection frame position labels and crack type labels; The target detection frame position includes: the upper left corner coordinates and the lower right corner coordinates of the crack target detection frame; Step 2: Use the tower image data set to obtain a training set, use the training set to train the tower foundation crack detection network, and obtain a trained tower foundation crack detection network, specifically: Step 21: Input the tower image data into the pre-trained ResNet network to obtain the tower image feature map, randomly initialize the tower foundation crack target detection frame position and the tower foundation crack type, and use the tower image feature map, the initialized tower foundation crack target detection frame position, the initialized tower foundation crack type, the tower image data target detection frame position label, and the crack type label to form a training set; Step 22: Use the training set to train the tower foundation crack detection network to obtain a trained tower foundation crack detection network.

3. A tower foundation crack detection method based on pixel focus attention according to claim 2, characterized in that: The tower foundation crack detection network comprises: an input module, an encoding module, a decoding module and an output module; The input module uses the tower image feature map, the crack target detection frame position and the tower foundation crack type to obtain the final input sequence and the target sequence; The input module comprises: an input embedding layer, a first position encoding layer, an output embedding layer, and a second position encoding layer; The input embedding layer is used to process the tower image feature map to obtain an input sequence; The first position encoding layer is used to add position encoding to the input sequence to obtain a final input sequence X, and the final input sequence X is input into the encoding module; The output embedding layer is used to process the crack target detection frame position and the corresponding crack type label to obtain a target query sequence; The second position encoding layer is used to add position encoding to the target query sequence to obtain a target sequence, and the target sequence is input into the multi-head attention layer of each decoder in the decoding module; The encoding module is used to encode the final input sequence X to obtain the encoding information matrix C; The encoding module is composed of a plurality of encoders connected in series; Each encoder includes: a multi-head attention mechanism layer, a first Add&Norm layer, a first feedforward control layer, and a second Add&Norm layer; The first Add&Norm layer and the second Add&Norm layer both include: a residual connection layer and a layer normalization layer; the output of the residual connection layer is the input of the layer normalization layer; The input of the first encoder is the final input sequence X, the input of the n'th encoder is the output of the n'-1th encoder, and the output of the N'th encoder is the output of the encoding module, that is, the encoding information matrix C, n'∈[2,N'], N' is the total number of encoders in the encoding module; The decoding module uses the coding information matrix C and the target sequence to obtain a prediction score of whether the cracks in the tower image belong to a certain type; The decoding module is composed of a plurality of decoders connected in series; Each decoder includes: a masked multi-head attention layer, a third Add&Norm layer, a multi-head attention layer, a fourth Add&Norm layer, a second feedforward control layer, and a fifth Add&Norm layer; The third Add&Norm layer, the fourth Add&Norm layer, and the fifth Add&Norm layer have the same structure as the first Add&Norm layer; The input of the first decoder is the encoding information matrix C and the target sequence. The input of the m'th decoder is the output of the m'-1th decoder and the encoding information matrix C. The output of the M'th decoder is the output of the decoding module, that is, the prediction score of the crack in the tower image belonging to a certain category; m'∈[2,M'], M' is the total number of decoders; The output module uses the prediction score of the cracks in the tower image belonging to a certain category to output the probability that the cracks in the current tower image belong to a certain category and the predicted position of the crack target detection frame; The output module includes: a linear layer and a softmax layer in sequence.

4. A tower foundation crack detection method based on pixel focus attention according to claim 3, characterized in that: The input module uses the tower image feature map, the crack target detection frame position and the tower foundation crack type label to obtain the final input sequence and target sequence, specifically: A1. Preprocess the tower image feature map to obtain the input sequence: A1-1, dividing the tower image feature map into blocks according to a preset size to obtain multiple feature map blocks; A1-2. Flatten the feature block and input the flattened feature block into the fully connected layer to obtain the feature vector; A1-3. Combine all feature vectors to obtain a feature sequence, and add the class token to the feature sequence as the starting feature vector of the feature sequence, thereby obtaining an input sequence. A2. Add position code to the input sequence to obtain the final input sequence; A3. Process the crack target detection frame position and the tower foundation crack type to obtain the target query sequence: The position of the crack target detection box corresponding to a crack in the tower image feature map and the type of crack in the tower foundation are combined into a target query vector, and all target query vectors form a target query sequence; A4. Add position code to the target query sequence to obtain the target sequence.

5. A tower foundation crack detection method based on pixel focus attention according to claim 4, characterized in that: The step A2 adds position codes to the input sequence to obtain the final input sequence, specifically: A2-1. Get the position code of each feature vector in the input sequence, specifically: The position encoding of the feature vector is composed by using the position encoding of each element in the feature vector; The position encoding of each element in the feature vector is obtained by the following formula: Among them, pos is the feature vector number in the input sequence, d is the current feature vector dimension, i' is the dimension index in the current feature vector, PE (pos,2i') is the positional encoding of the 2i'th element in the posth eigenvector, PE (pos,2i'+1) is the position code of the 2i'+1th element in the posth feature vector, i'∈[0,d], d is an integer; A2-2. Each eigenvector and the corresponding eigenvector position code are added together to form a sequence to obtain the final input sequence.

6. A tower foundation crack detection method based on pixel focus attention according to claim 5, characterized in that: The step A4 of adding a position code to the target query sequence to obtain the target sequence is specifically as follows: A4-1. Obtain the position encoding of each target query vector in the target query sequence: Using the positional encoding of each element in the target query vector to form a positional encoding of the target query vector; The method for obtaining the position encoding of each element in the target query vector is the same as the method for obtaining the position encoding of each element in the feature vector; A4-2. Each target query vector and the position code of the corresponding target query vector are added together to form a sequence to obtain a target sequence.

7. A tower foundation crack detection method based on pixel focus attention according to claim 6, characterized in that: The encoding module is used to encode the final input sequence X to obtain the encoding information matrix C, which is specifically: B1. Use the final input sequence X to obtain the local range attention score matrix of the pixel and the global context attention score matrix of the pixel: ||ρ(i,j)||=k 2 ||σ(X)||=HpWp Where (i, j) is the coordinate of any pixel position, ρ(i, j) is a set of pixels in a sliding window of size k×k centered at (i, j), k is a constant, and S (i,j)～ρ(i,j) is the local range attention score of the pixel at position (i, j), S (i,j)～σ(X) is the global context attention score of the pixel at position (i, j), Q (i,j) is the pixel Query value at position (i, j), K ρ(i,j) is the Key value of ρ(i,j), σ(X) represents the feature map obtained after X passes through a single-layer neural network, pooling, and layer normalization in sequence, K σ(X) is the Key value of σ(X), It's K ρ(i,j) The weight matrix of It's K σ(X) The weight matrix of , Hp is the height of the feature map obtained after X passes through a single-layer neural network and pooling in sequence, and Wp is the width of the feature map obtained after X passes through a single-layer neural network and pooling in sequence; B2. Pixel attention weights using the local range attention score matrix of the pixel and the global context attention score matrix of the pixel: Among them, A (i,j) is the pixel attention weight at position (i, j), Concat() is the concatenation function, is the scaling factor, B(i,j) is the bias term; B3. Use pixel attention weights to obtain the final input sequence attention value; B4. Use PFA(X) to obtain the output of the multi-head attention mechanism layer in the encoder; B5. Input the output of the multi-head attention mechanism layer in the current encoder and the final input sequence into the first Add&Norm layer for residual connection and normalization. Then input the output Y of the first Add&Norm layer into the first feedforward control layer. Then input the output of the first feedforward control layer and Y into the second Add&Norm layer for residual connection and layer normalization to obtain the output sequence of the current encoder. B6. Use the output sequence of the current encoder as the input sequence of the next encoder, and repeat B1-B4 until the last encoder outputs the encoding information matrix C.

8. The method for detecting cracks in a tower foundation based on pixel focus according to claim 7, characterized in that: The pixel attention weight is used in B3 to obtain the final input sequence attention value, specifically: PFA(X (i,j) )=A (i,j)～ρ(i,j) V ρ(i,j) +A (i,j)～σ(X) V σ(X) Among them, X (i,j) is the corresponding eigenvalue of position (i, j) in X, i∈[1,M], j∈[1,N], M is the maximum value of the pixel horizontal coordinate, N is the maximum value of the pixel vertical coordinate, PFA(X) is the final input sequence attention value, PFA(X) (i,j) ) is X (i,j) The corresponding attention value, A (i,j)～ρ(i,j) Yes A (i,j) The relationship value with ρ(i,j), A (i,j)～σ(X) Yes A (i,j) Relationship with σ(X), V ρ(i,j) is the Value associated with ρ(i,j), V σ(X) A represents the Value associated with σ(X). (m,n) is the pixel attention weight at position (m,n), d k It is the dimension of Query and key. Yes V ρ(i,j) The weight matrix of Yes V σ(X) The weight matrix of .

9. A tower foundation crack detection method based on pixel focus attention according to claim 8, characterized in that: The output of the multi-head attention mechanism layer in the encoder is obtained by using PFA(X) in B4, specifically: MultiHead(Q,K,V)=Concat(head1,…,head h )W O Q=PFA(X)W Q K=PFA(X)W K V=PFA(X)W V Where i*∈[1,h], i* is the number of attention heads in the multi-head attention mechanism layer in the encoder, h is the total number of attention heads in the multi-head attention mechanism layer in the encoder, and head i* is the output of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Query projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Key projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, is the Value projection matrix of the i*th attention head in the multi-head attention mechanism layer in the encoder, W O is the output projection matrix, Q is the Query value of the current multi-head attention mechanism layer, K is the Key value of the current multi-head attention mechanism layer, V is the Value value of the current multi-head attention mechanism layer, and W Q is the weight matrix of Q, W K is the weight matrix of K, W V is the weight matrix of V.

10. The method for detecting cracks in a tower foundation based on pixel focus according to claim 9, characterized in that: The decoding module uses the coding information matrix C and the target sequence to obtain the prediction score of the cracks in the tower image belonging to a certain type, specifically: C1. Input the target sequence M into the masked multi-head attention layer to obtain the masked attention output; C2, use the masked attention output and the target sequence M to perform residual connection and normalization to obtain the updated target sequence Z; C3, input the encoded information matrix C and the updated target sequence Z into the multi-head attention layer of the decoder to obtain the multi-head attention layer output matrix of the decoder; MultiHead(Q',K',V')=Concat(head1,head2,....,head h' )W' O K'=CW' K' V'=CW' V' Q'=ZW' Q' Where i”∈[1,h'], i” is the number of the attention head in the multi-head attention layer in the decoder, h' is the total number of attention heads in the multi-head attention layer in the decoder, and head i” is the output of the i-th attention head, is the Query projection matrix of the i-th attention head, is the Key projection matrix of the i'th attention head, is the Value projection matrix of the i'th attention head, W' O is the output projection matrix, W' K' is the weight matrix of K', W' V' is the weight matrix of V', W' Q' is the weight matrix of Q', Q' is the Query value of the current multi-head attention layer, K' is the Key value of the current multi-head attention layer, and V' is the Value value of the current multi-head attention layer; C4, use the multi-head attention layer output matrix of the decoder obtained in C3 and the matrix Z to perform residual connection and normalization to obtain the matrix Z', then input the matrix Z' into the second feedforward control layer, input the output matrix of the second feedforward control layer and the matrix Z' into the fifth Add&Norm layer to obtain the output matrix Z* of the current decoder; C5. Use the output matrix Z* of the current decoder as the input matrix of the next decoder, and repeat C1-C4 until the last decoder outputs the prediction score of whether the cracks in the tower image belong to a certain category.

Images