Engineering structure crack identification method based on transfer learning
By adopting a transfer learning-based method in engineering structural crack detection, combining U-Net and Transformer technology, a full-scene crack detection network is built and transfer learning is performed in specific scenarios, the problem of failure to fully utilize the similarity of crack data between different structures in the existing technology is solved, and higher detection performance and robustness are achieved.
Patent Information
- Application Number
- CN202510315135.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-18
- Publication Date
- 2025-04-18
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art fails to fully utilize the potential similarity of crack data between different structures in the detection of engineering structural cracks, resulting in limited detection performance.
Using a transfer learning-based method, a full-scene crack detection network is constructed by pre-training the data set containing cracks of multiple engineering structures, combining U-Net and Transformer technologies to learn common features, and transfer learning is performed in specific scenarios to optimize crack detection performance.
The generalization ability of the model and the sensitivity and accuracy of crack detection are improved, making the model more robust to crack identification of different materials and structures, and reducing misidentification.
Smart Images
Figure CN119850598A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of model application, and in particular to a method for identifying cracks in an engineering structure based on transfer learning. Background Art
[0002] Due to long-term fatigue, engineering structures may develop cracks of various degrees, which will seriously affect the safety performance of the structure. These cracks can be classified into building structure cracks, bridge structure cracks and tunnel structure cracks according to the type of structure they appear in; they can be divided into concrete cracks and steel cracks according to the type of material.
[0003] Currently, most crack detection methods are designed for specific engineering structures and materials by using specific datasets (such as steel bridge crack datasets) to train the detection network. However, this method does not fully utilize the potential similarities of crack data between different engineering structures.
[0004] Therefore, a new solution to the above problems needs to be proposed. Summary of the invention
[0005] The purpose of the present invention is to provide a method for identifying cracks in engineering structures based on transfer learning to solve the technical deficiencies mentioned in the background technology.
[0006] To achieve the above object, the present invention provides the following technical solution: a method for identifying cracks in an engineering structure based on transfer learning, comprising at least the following steps: S1: Pre-train all data sets containing cracks of various engineering structures to build a full-scenario crack detection network, that is, build a full-scenario crack detection network model based on pre-training. The full-scenario crack detection network model can learn and identify the common features of different cracks, thereby improving the generalization ability of the model; S2: Using the full-scenario crack detection network model pre-trained in S1, transfer learning is performed on a single data set for specific application scenarios to optimize and enhance the crack detection performance of specific scenarios, and obtain a specific scenario crack segmentation network based on transfer learning; S3: Use the trained and optimized network model to identify cracks in engineering structures.
[0007] Furthermore, the full-scenario crack detection network model combines U-Net and Transformer technologies, utilizing the powerful spatial feature extraction capability of U-Net and the long-distance dependency capture capability of Transformer; The full scene crack detection network model receives two inputs, one of which is the original image. , and the other input is the crack boundary obtained by fast contour recognition of the original image ; The process of fast contour recognition first performs threshold processing, then performs Canny edge detection to extract edges, and finally uses connected domain analysis to obtain contour boundaries.
[0008] Furthermore, the U-Net network in the full-scenario crack detection network model adopts a convolution operation in the downsampling module, as shown in formula (1):
[0009] in, X is the input feature map, W is the convolution kernel weight, b is the bias term, * represents the convolution operation, is the ReLU activation function; The U-Net network in the full-scenario crack detection network model adopts a transposed convolution operation in the upsampling module, as shown in formula (2):
[0010] in, W T is the convolution kernel weight; Through its symmetrical structure and skip connections, the U-Net network can effectively aggregate and utilize information from up- and down-sampling stages of different resolutions, thereby making more accurate segmentation at the local and global levels of the image.
[0011] Furthermore, the features of downsampling and upsampling are fused through a jump connection, as shown in formula (3):
[0012] in, is the feature map sampled from the next layer, It is the feature map passed from the corresponding downsampling layer through the skip connection. Represents the concatenation of feature maps in the channel dimension.
[0013] Furthermore, a Transformer module is inserted between the encoder and decoder of the U-Net network after being processed by the embedding module, and a self-attention mechanism is used; This enables the network to process and understand the complex relationships and dependencies between the high-dimensional features of the original image and the various parts of the crack boundary; The embedding module uses global maximum pooling to reduce the spatial dimension of the input boundary image, as shown in formula (4):
[0014] in, is the input feature map, is the output feature map after pooling, are the coordinates in the output feature map, k is the step size of the pooling operation, which indicates the compensation for the window moving on the input feature map. is the index traversed in the pooling window, and the size of the sliding window is , Indicates taking the maximum value within a given window; The calculation formula of the self-attention mechanism in the Transformer module is shown in equations (5)-(9):
[0015] in, They are Q(Qurey, Query ) , K(Key, key ) , V(Vlaue, value ) The weight matrix obtained by neural network learning, X is the input feature map; Formula (8) represents the calculation of attention weight, yes Key The dimension of the vector used for normalization of the scaling factor; T Represents the transpose of a matrix; Formula (9) represents the final output weighted value.
[0016] Furthermore, the data set used in the pre-training in S1 is all data sets related to cracks, and the pre-training is to train a neural network; The training of the neural network adopts cross entropy loss, as shown in formula (10):
[0017] in, is the true label, is the predicted probability, N is the total number of pixels; The initial learning rate is , and uses the Adam optimizer.
[0018] Furthermore, the architecture of the specific scene crack segmentation network based on transfer learning is the same as the architecture of the full scene crack detection network model. The U-Net network obtained in the pre-training stage is trained to fine-tune the network parameters of the input layer, output layer and neck, and the network parameters of other positions are frozen; The transfer learning process is shown in formula (11), that is, the gradient descent method is used to fine-tune the parameters that are not frozen;
[0019] in, L represents the loss function, represents the model parameters, represents the learning rate of the parameter, It means to find partial derivative; The input layer and output layer are fine-tuned because these two layers need to directly process task-specific data, so they need to be fine-tuned according to the specific data of the task; The neck network is fine-tuned because it is a high-dimensional feature processing part, and adaptive adjustment to specific tasks is particularly critical; At the same time, in order to achieve the purpose of fine-tuning the neural network, a lower initial learning rate is used at this stage, that is, the initial learning rate is , also using the Adam optimizer.
[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. The full-scenario crack detection network basic model proposed in this invention combines U-Net and Transformer technologies, and uses the powerful spatial feature extraction capability of U-Net and the long-distance dependency capture capability of Transformer. By using various crack-related data sets, including crack data of different engineering structure types (such as buildings, bridges, roads, etc.) and different materials (such as concrete, steel, etc.), the pre-trained model can learn a wider range of features. Such feature coverage makes the model more adaptable to various possible practical application scenarios and improves the generalization ability of the model; Diversified training data helps improve the model's sensitivity and accuracy in crack detection. The shape, size, and pattern of cracks vary in different materials and structures. Learning from these changes can make the model more robust in crack recognition and reduce misidentification caused by environmental changes or differences in crack characteristics. 2. Based on the pre-trained model, the present invention can further enhance the crack detection performance of the model in the scenario by performing transfer learning on the data set for the specific scenario. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0022] Figure 1 It is a schematic diagram of the main steps of the present invention; Figure 2 This is a full-scenario crack detection network diagram based on pre-training of the present invention; Figure 3 This is a network diagram of crack detection in a specific scenario based on transfer learning in the present invention. DETAILED DESCRIPTION
[0023] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0024] See also Figure 1 , a method for identifying cracks in engineering structures based on transfer learning, comprising at least the following steps: S1: Pre-train all data sets containing cracks of various engineering structures to build a full-scenario crack detection network, that is, build a full-scenario crack detection network model based on pre-training. The full-scenario crack detection network model can learn and identify the common features of different cracks, thereby improving the generalization ability of the model; S2: Using the full-scenario crack detection network model pre-trained in S1, transfer learning is performed on a single data set for specific application scenarios to optimize and enhance the crack detection performance of specific scenarios, and obtain a specific scenario crack segmentation network based on transfer learning; S3: Use the trained and optimized network model to identify cracks in engineering structures.
[0025] See also Figure 2 ,The full-scenario crack detection network model combines U-Net and Transformer technologies, utilizing the powerful spatial feature extraction capability of U-Net and the long-distance dependency capture capability of Transformer; The full scene crack detection network model receives two inputs, one of which is the original image. , and the other input is the crack boundary obtained by fast contour recognition of the original image ; The process of fast contour recognition first performs threshold processing, then performs Canny edge detection to extract edges, and finally uses connected domain analysis to obtain contour boundaries.
[0026] The U-Net network in the full-scene crack detection network model uses convolution operation in the downsampling module, as shown in formula (1):
[0027] in, X is the input feature map, W is the convolution kernel weight, b is the bias term, * represents the convolution operation, is the ReLU activation function; The U-Net network in the full-scene crack detection network model uses a transposed convolution operation in the upsampling module, as shown in formula (2):
[0028] in, W T is the convolution kernel weight; Through its symmetrical structure and skip connections, the U-Net network can effectively aggregate and utilize information from up- and down-sampling stages of different resolutions, thereby making more accurate segmentation at the local and global levels of the image.
[0029] The features of downsampling and upsampling are fused through skip connections, as shown in formula (3):
[0030] in, is the feature map sampled from the next layer, It is the feature map passed from the corresponding downsampling layer through the skip connection. Represents the concatenation of feature maps in the channel dimension.
[0031] The Transformer module is inserted between the encoder and decoder of the U-Net network after being processed by the embedding module, and the self-attention mechanism is used ( Figure 2 Labeled as Q, K, V, namely Query, Key, Value); This enables the network to process and understand the complex relationships and dependencies between the high-dimensional features of the original image and the various parts of the crack boundary; The embedding module uses global maximum pooling to reduce the spatial dimension of the input boundary image, as shown in formula (4):
[0032] in, is the input feature map, is the output feature map after pooling, are the coordinates in the output feature map, k is the step size of the pooling operation, which indicates the compensation for the window moving on the input feature map. is the index traversed in the pooling window, and the size of the sliding window is , Indicates taking the maximum value within a given window; The calculation formula of the self-attention mechanism in the Transformer module is shown in equations (5)-(9):
[0033] in, They are Q(Qurey, Query ) , K(Key, key ) , V(Vlaue, value ) The weight matrix obtained by neural network learning, X is the input feature map; Formula (8) represents the calculation of attention weight, yes Key The dimension of the vector used for normalization of the scaling factor; T Represents the transpose of a matrix; Formula (9) represents the final output weighted value.
[0034] The data set used for pre-training in S1 is all data sets related to cracks, and the pre-training is to train the neural network; The training of the neural network adopts the cross entropy loss, as shown in formula (10):
[0035] in, is the true label, is the predicted probability, N is the total number of pixels; The initial learning rate is , and uses the Adam optimizer.
[0036] See also Figure 3 ,Based on the pre-trained model, by performing transfer learning on the dataset of a specific scenario, the crack detection performance of the model in this scenario can be further enhanced, e.g. Figure 3 For example, for a specific type of bridge crack dataset k Transfer learning can enable the model to more accurately identify and handle crack problems on this type of bridge, thereby achieving more accurate crack detection and classification.
[0037] The architecture of the specific scene crack segmentation network based on transfer learning is the same as that of the full scene crack detection network model. The U-Net network obtained in the pre-training stage is trained to fine-tune the network parameters of the input layer, output layer, and neck, and the network parameters of other positions are frozen; The transfer learning process is shown in formula (11), that is, the gradient descent method is used to fine-tune the parameters that are not frozen;
[0038] in, L represents the loss function, represents the model parameters, represents the learning rate of the parameter, It means to find partial derivative; The input layer and output layer are fine-tuned because these two layers need to directly process task-specific data, so they need to be fine-tuned according to the specific data of the task; The neck network is fine-tuned because it is a high-dimensional feature processing part, and adaptive adjustment to specific tasks is particularly critical; At the same time, in order to achieve the purpose of fine-tuning the neural network, a lower initial learning rate is used at this stage, that is, the initial learning rate is , also using the Adam optimizer.
[0039] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.
Claims
1. A method for identifying cracks in engineering structures based on transfer learning, characterized in that: At least the following steps are included: S1: Pre-train all data sets containing cracks of various engineering structures to build a full-scenario crack detection network, that is, build a full-scenario crack detection network model based on pre-training. The full-scenario crack detection network model can learn and identify the common features of different cracks, thereby improving the generalization ability of the model; The full-scenario crack detection network model combines U-Net and Transformer technologies, using the powerful spatial feature extraction capability of U-Net and the long-distance dependency capture capability of Transformer; The full scene crack detection network model receives two inputs, one of which is the original image. , and the other input is the crack boundary obtained by fast contour recognition of the original image ; The process of fast contour recognition first performs threshold processing, then performs Canny edge detection to extract edges, and finally uses connected domain analysis to obtain contour boundaries; The U-Net network in the full-scenario crack detection network model adopts convolution operation in the downsampling module, as shown in formula (1): ， in, X is the input feature map, W is the convolution kernel weight, b is the bias term, * represents the convolution operation, is the ReLU activation function; The U-Net network in the full-scenario crack detection network model adopts a transposed convolution operation in the upsampling module, as shown in formula (2): ， in, W T is the convolution kernel weight; The U-Net network, through its symmetrical structure and skip connections, can effectively aggregate and utilize information from up- and down-sampling stages of different resolutions, thereby making more accurate segmentation at the local and global levels of the image. S2: Using the full-scenario crack detection network model pre-trained in S1, transfer learning is performed on a single data set for specific application scenarios to optimize and enhance the crack detection performance of specific scenarios, and obtain a specific scenario crack segmentation network based on transfer learning; S3: Use the trained and optimized network model to identify cracks in engineering structures.
2. The method for identifying cracks in engineering structures based on transfer learning according to claim 1, characterized in that: The downsampling and upsampling are fused by skip connection, as shown in formula (3): ， in, is the feature map after upsampling from the next layer, It is the feature map passed from the corresponding downsampling layer through the skip connection. Represents the concatenation of feature maps in the channel dimension.
3. The method for identifying cracks in engineering structures based on transfer learning according to claim 2, characterized in that: A Transformer module is inserted between the encoder and decoder of the U-Net network after being processed by the embedding module, and a self-attention mechanism is used; This enables the network to process and understand the complex relationships and dependencies between the high-dimensional features of the original image and the various parts of the crack boundary; The embedding module uses global maximum pooling to reduce the spatial dimension of the input boundary image, as shown in formula (4): ， in, is the input feature map, is the output feature map after pooling, are the coordinates in the output feature map, k is the step size of the pooling operation, which indicates the compensation for the window moving on the input feature map. is the index traversed in the pooling window, and the size of the sliding window is , Indicates taking the maximum value within a given window; The calculation formula of the self-attention mechanism in the Transformer module is shown in equations (5)-(9): ， ， in, They are Q(Qurey, Query ) , K(Key, key ) , V(Vlaue, value ) The weight matrix obtained by neural network learning, X is the input feature map; Formula (8) represents the calculation of attention weight, yes Key The dimension of the vector used for normalization of the scaling factor; T Represents the transpose of a matrix; Formula (9) represents the final output weighted value.
4. The method for identifying cracks in engineering structures based on transfer learning according to claim 3, characterized in that: The data set used for the pre-training in S1 is all data sets related to cracks, and the pre-training is to train the neural network; The training of the neural network adopts cross entropy loss, as shown in formula (10): ， in, is the true label, is the predicted probability, N is the total number of pixels; The initial learning rate is , and uses the Adam optimizer.
5. The method for identifying cracks in engineering structures based on transfer learning according to claim 4, characterized in that: The architecture of the specific scene crack segmentation network based on transfer learning is the same as the architecture of the full scene crack detection network model. The U-Net network obtained in the pre-training stage is trained to fine-tune the network parameters of the input layer, output layer and neck, and the network parameters of other positions are frozen; The transfer learning process is shown in formula (11), that is, the gradient descent method is used to fine-tune the parameters that are not frozen; ， in, L represents the loss function, represents the model parameters, represents the learning rate of the parameter, It means to find partial derivative; The input layer and output layer are fine-tuned because these two layers need to directly process task-specific data, so they need to be fine-tuned according to the specific data of the task; The neck network is fine-tuned because it is a high-dimensional feature processing part, and adaptive adjustment to specific tasks is particularly critical; At the same time, in order to achieve the purpose of fine-tuning the neural network, a lower initial learning rate is used at this stage, that is, the initial learning rate is , also using the Adam optimizer.