Multi-level composite structure damage identification method and system based on deep learning

Abstract

The invention discloses a multi-level composite structure damage identification method and system based on deep learning, and relates to the field of structure health monitoring, and the method comprises the steps: sequentially constructing a first-level damage identification model and a second-level damage identification model based on a structure damage data set; performing primary damage identification on the image data in the structural damage data set based on the trained primary damage identification model, and dividing the image data in the structural damage data set into a non-damage image data set and a damaged image data set based on a primary damage identification result; and performing secondary damage identification on the image data in the damaged image data set based on the trained secondary damage identification model to obtain a secondary damage identification result. According to the method, the multi-level composite deep learning model is adopted, detection tasks with different complexities can be considered, the requirements of structural damage detection are comprehensively covered, and the recognition precision and comprehensiveness are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of structural health monitoring, and in particular to a multi-level composite structure damage identification method and system based on deep learning. Background Art

[0002] Structural health monitoring is an important research direction in the engineering field. In particular, timely detection and prevention of structural damage is of vital importance in bridges, tunnels, buildings and large infrastructure. With the aging of infrastructure and the increasing impact of the environment, such as earthquakes, wind loads, corrosion and other factors, the risk of structural damage and degradation is gradually increasing. If these damages are not discovered and repaired in time, they may lead to serious safety hazards and even catastrophic consequences. Therefore, how to efficiently and accurately detect structural damage has become an urgent problem to be solved in the engineering community.

[0003] Traditional structural damage detection methods mainly rely on manual detection and monitoring equipment, such as ultrasonic detection, infrared detection, laser scanning, etc. Although these methods can detect structural damage to a certain extent, there are several major problems: First, manual detection often relies on the experience of the inspectors and is easily affected by subjective factors, resulting in misjudgment or missed judgment; second, traditional detection methods are inefficient, especially when facing large structures or complex environments, it is difficult to conduct comprehensive and rapid detection; third, conventional methods are difficult to automate, consume a lot of manpower and material resources, and due to equipment limitations, it is difficult to monitor the health of the structure in real time.

[0004] In recent years, with the rapid development of computer vision and deep learning technologies, automated structural damage detection technology based on image processing has gradually attracted attention. Through the training of a large amount of data, deep learning can extract high-level features in images and automatically identify them, showing potential far beyond traditional methods. Compared with traditional methods, structural damage detection based on deep learning has the following significant advantages:

[0005] 1. High degree of automation: Deep learning algorithms can automatically handle the entire process from image acquisition to feature extraction, greatly reducing reliance on manual experience. Through training with a large amount of image data, deep learning models can more accurately identify damage features and significantly reduce the misjudgment rate and missed judgment rate.

[0006] 2. Efficient and accurate: After sufficient training, the deep learning model can process a large amount of image data in a short time and complete damage analysis of complex structures. In particular, deep learning performs far better than traditional methods when identifying micro-damage features.

[0007] 3. Multi-task processing: Deep learning can not only be used for image classification, but also can be combined with target detection and semantic segmentation technology to achieve multi-dimensional recognition of damage. Through these technologies, deep learning can not only determine the type of damage, but also accurately locate the specific location and shape of the damage, obtain the boundary information of the damage, and provide more accurate data support for subsequent repair decisions.

[0008] However, existing deep learning-based structural damage detection methods still face some challenges. First, there are many types of structural damage, including cracks, spalling, corrosion, deformation, etc., and the morphology and scale of different types of damage vary greatly. A single deep learning model is often difficult to deal with all types of damage at the same time, and existing deep learning methods have certain limitations when detecting complex damage. Secondly, deep learning models usually require a lot of computing resources, especially when processing high-definition image data, the computing cost of training and inference is high. In addition, in practical applications, especially in scenarios where real-time monitoring is required, how to improve the computing speed while ensuring detection accuracy is still an urgent problem to be solved. In addition, the performance of deep learning models depends on the diversity and quality of data sets, and there may be a certain gap between training data and actual scenarios. Under different environmental conditions (such as changes in lighting, different angles, etc.), the performance of the model may be unstable, and the generalization ability needs to be improved.

[0009] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0010] In response to the problems in the related art, the present invention proposes a multi-level composite structure damage identification method and system based on deep learning to overcome the above-mentioned technical problems existing in the existing related technology.

[0011] To this end, the specific technical solution adopted by the present invention is as follows:

[0012] According to one aspect of the present invention, a multi-level composite structure damage identification method based on deep learning is provided, the method comprising the following steps:

[0013] S1. Obtain a structural damage data set of a multi-level composite structure in the engineering field, construct a primary damage identification model and a secondary damage identification model based on the structural damage data set, and train the primary damage identification model and the secondary damage identification model in turn;

[0014] S2. Based on the trained primary damage recognition model, perform primary damage recognition on the image data in the structural damage data set, and divide the image data in the structural damage data set into a non-damage image data set and a damaged image data set based on the primary damage recognition result;

[0015] S3. Based on the trained secondary damage recognition model, secondary damage recognition is performed on the image data in the damaged image data set to obtain secondary damage recognition results.

[0016] Furthermore, a structural damage data set of a multi-level composite structure in the engineering field is obtained, a primary damage identification model and a secondary damage identification model are sequentially constructed based on the structural damage data set, and the primary damage identification model and the secondary damage identification model are sequentially trained, including the following steps:

[0017] S11. Obtain image data of multi-level composite structures in the engineering field from the data set and establish a structural damage data set;

[0018] S12, preprocessing the image data in the structural damage data set;

[0019] S13, annotating all image data in the preprocessed structural damage dataset, obtaining primary and secondary structural damage datasets, and dividing the primary and secondary structural damage datasets into corresponding training sets and validation sets according to preset standards, wherein the secondary structural damage dataset includes target and segmented structural damage datasets;

[0020] S14. Based on the deep learning algorithm, a primary damage recognition model and a secondary damage recognition model are constructed. The secondary damage recognition model includes a target detection model and a semantic segmentation model.

[0021] S15, training the primary damage identification model using the training set of the primary structural damage data set;

[0022] S16. Use the training set in the secondary structure damage data set to train the secondary damage recognition model.

[0023] Furthermore, preprocessing the image data in the structural damage dataset includes the following steps:

[0024] S121, performing an image scaling operation on sample data in the structural damage data set;

[0025] S122, performing color adjustment on the sample data after the image scaling operation to modify the color attribute of the sample data;

[0026] S123, performing rotation correction on the sample data after color modification, and performing noise reduction processing on the sample data after rotation correction.

[0027] Furthermore, based on the deep learning algorithm, a primary damage recognition model and a secondary damage recognition model are constructed. The secondary damage recognition model includes a target detection model and a semantic segmentation model, and includes the following steps:

[0028] S141. According to the deep learning algorithm, a ResNet-50 architecture is designed, and an image classification model is constructed based on the ResNet-50 architecture, where the image classification model is a primary damage recognition model;

[0029] S142. Design the YoLo-v8 architecture based on the deep learning algorithm, and build a target detection model based on the YoLo-v8 architecture;

[0030] S143. According to the deep learning algorithm, design the Deeplab semantic segmentation network architecture, and build a semantic segmentation model based on the Deeplab semantic segmentation network architecture.

[0031] Furthermore, training the primary damage identification model using the training set of the primary structural damage data set includes the following steps:

[0032] S151, inputting the training set of the primary structural damage data set into the image classification model, and extracting image features layer by layer through the deep network structure of the image classification model;

[0033] S152. Based on the extracted image features, classification prediction is performed at the end of the deep network structure through a fully connected layer and an activation function, and the convergence of the image classification model is evaluated using a binary cross entropy loss function.

[0034] S153, using the back propagation algorithm and the gradient descent algorithm to update the parameters of the image classification model;

[0035] S154. Repeat steps S151 to S153 until a preset stop condition is reached to obtain a damage classification result.

[0036] Furthermore, the formula for the binary cross entropy loss function is:

[0037]

[0038] In the formula, Loss(1) represents the total loss value of the image classification model;

[0039] N represents the total number of sample data;

[0040] i represents the index;

[0041] y i Represents the actual category distribution probability of the i-th category;

[0042] y i ′ represents the predicted category distribution probability of the image classification model of the i-th category.

[0043] Furthermore, training the secondary damage recognition model using the training set in the secondary structure damage data set includes the following steps:

[0044] S161, inputting the training set of the target structure damage data set into the target detection model, extracting the position features through the convolution layer and predicting the damage position at multiple scales, optimizing the target detection model using the target detection loss function and the gradient descent algorithm, so that the output after the input image processing includes the damage position and the category prediction result;

[0045] S162. Input the training set of the segmented structural damage data set into the semantic segmentation model, classify each pixel, calculate the pixel-level cross entropy loss function and use the gradient descent algorithm to optimize the semantic segmentation model, so that each pixel after the input image is processed is classified as damaged or non-damaged, and output the damaged area image data.

[0046] Furthermore, the formula of the target detection loss function is:

[0047]

[0048] In the formula, Loss(2) represents the total loss value of the target detection model;

[0049] λ CIoU represents the weight of the bounding box regression loss;

[0050] L CIoU represents the bounding box regression loss;

[0051] λ Classification Represents the weight of classification loss;

[0052] L Classification represents the classification loss;

[0053] λ Confidence Represents the weight of confidence loss;

[0054] L Confidence represents the confidence loss;

[0055] IoU represents the intersection over union of the predicted bounding box and the true bounding box;

[0056] ρ 2 (b,b g ) represents the square of the Euclidean distance between the center point of the predicted box and the center point of the real box; c represents the diagonal length of the minimum bounding box of the predicted box and the real box;

[0057] α represents the consistency correction term of aspect ratio;

[0058] v represents the consistency correction term of aspect ratio;

[0059] C represents the number of categories;

[0060] i represents the index;

[0061] yi Represents the actual category distribution probability of the i-th category;

[0062] y i ′ represents the predicted category distribution probability of the target detection model of the i-th category;

[0063] y obj The true label indicating the existence of the target;

[0064] y obj ′ represents the confidence of the object existence predicted by the target detection model.

[0065] Furthermore, the formula of the pixel-level cross entropy loss function is:

[0066]

[0067] In the formula, Loss(3) represents the total loss value of the semantic segmentation model;

[0068] N represents the total number of sample data;

[0069] i represents the index;

[0070] y i represents the true label of the i-th pixel;

[0071] y i ′ represents the probability that the semantic segmentation model predicts that the i-th pixel belongs to the target area.

[0072] According to another aspect of the present invention, there is also provided a multi-level composite structure damage identification system based on deep learning, the system comprising a model building and training module, a primary damage identification module and a secondary damage identification module;

[0073] The model building and training module is used to obtain the structural damage data set of multi-level composite structures in the engineering field, build the first-level damage identification model and the second-level damage identification model in turn based on the structural damage data set, and train the first-level damage identification model and the second-level damage identification model in turn;

[0074] A primary damage recognition module, used to perform primary damage recognition on the image data in the structural damage data set based on the trained primary damage recognition model, and divide the image data in the structural damage data set into a non-damage image data set and a damaged image data set based on the primary damage recognition result;

[0075] The secondary damage recognition module is used to perform secondary damage recognition on the image data in the damaged image data set based on the trained secondary damage recognition model to obtain the secondary damage recognition result.

[0076] The beneficial effects of the present invention are:

[0077] 1. The present invention adopts a multi-level composite deep learning model, including three layers: image classification, target detection and semantic segmentation, to analyze structural damage from different angles; the image classification layer quickly screens and determines whether there is damage, and filters a large number of normal images to reduce the subsequent calculation burden; the target detection layer locates the damaged area and determines the approximate position through the bounding box; the semantic segmentation layer accurately segments the damaged area, obtains the specific shape and boundary of the damage, and realizes pixel-by-pixel recognition; this multi-level architecture covers detection tasks of different complexities, from rough damage detection to precise segmentation, comprehensively covers the needs of structural damage detection, and greatly improves the accuracy and comprehensiveness of recognition.

[0078] 2. The present invention realizes the automation of the entire process from image acquisition to damage identification through a deep learning model, reduces the misjudgment rate and missed detection rate, and uses a large-scale structural damage data set for training. The model can autonomously learn and extract complex damage features, realize automated structural health monitoring, and improve detection efficiency and accuracy. In addition, the increase in the degree of automation reduces the labor intensity of manual inspection, and is particularly suitable for large infrastructure such as bridges and tunnels and high-risk areas that are difficult to reach manually.

[0079] 3. The present invention has flexibility and adaptability through multi-level model design. Different levels of models can be selected to work jointly or independently according to task requirements, damage complexity and hardware resource conditions. In scenarios where resources are limited or rapid detection is required, the system can only use the image classification layer or the target detection layer. In high-precision detection scenarios, the target detection and semantic segmentation layers can be enabled to achieve precise positioning and segmentation. This flexible architecture enables the system to adapt to various applications from low-resource environments to high-precision requirements, thereby providing targeted solutions.

[0080] 4. The present invention is optimized through a multi-level deep learning model and can efficiently handle structural damage detection tasks, especially in the detection of large-scale structures (such as bridges and buildings), showing remarkable real-time and high efficiency; the image classification layer quickly screens normal structural images, reduces the calculation burden of subsequent models, and improves detection efficiency; the target detection layer and the semantic segmentation layer achieve efficient positioning and damage segmentation through optimized loss functions and calculation methods, reduce detection time, and meet real-time monitoring needs; the system's high efficiency supports rapid response in scenarios with high detection frequency and large data volumes, and is suitable for regular monitoring of large infrastructure and real-time detection of emergencies. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0082] Figure 1 is a flow chart of a multi-level composite structure damage identification method based on deep learning according to an embodiment of the present invention;

[0083] Figure 2 is a working principle diagram of a multi-level composite structure damage identification method based on deep learning according to an embodiment of the present invention;

[0084] Figure 3 It is the overall architecture of a multi-level composite structure damage identification method based on deep learning according to an embodiment of the present invention;

[0085] Figure 4 is a workflow diagram of an image classification layer in a multi-level composite structure damage identification method based on deep learning according to an embodiment of the present invention;

[0086] Figure 5 is a workflow diagram of a target detection layer in a multi-level composite structure damage identification method based on deep learning according to an embodiment of the present invention;

[0087] Figure 6 is a workflow diagram of a semantic segmentation layer in a multi-level composite structure damage identification method based on deep learning according to an embodiment of the present invention;

[0088] Figure 7 It is a principle block diagram of a multi-level composite structure damage identification system based on deep learning according to an embodiment of the present invention.

[0089] In the figure:

[0090] 1. Model building and training module; 2. Level 1 damage identification module; 3. Level 2 damage identification module. DETAILED DESCRIPTION

[0091] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention and are mainly used to illustrate the embodiments. They can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementation methods and advantages of the present invention.

[0092] According to an embodiment of the present invention, a multi-level composite structure damage identification method and system based on deep learning are provided.

[0093] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1-Figure 3 As shown, according to the multi-level composite structure damage identification method based on deep learning according to an embodiment of the present invention, the method comprises the following steps:

[0094] S1. Obtain a structural damage dataset of a multi-level composite structure in the engineering field, construct a primary damage identification model and a secondary damage identification model based on the structural damage dataset, and train the primary damage identification model and the secondary damage identification model in turn.

[0095] It should be noted that the first-level damage identification model and the second-level damage identification model are constructed in a cascade manner; the first-level damage identification model is an image classification model, which is used for preliminary screening; the second-level damage identification model includes a target detection model and a semantic segmentation model. The target detection model is used to locate the fuzzy damage area, and the semantic segmentation model is used to accurately identify the damage boundary.

[0096] In addition, considering the characteristics of different damage identification tasks as well as computing resources and time requirements, a three-level series-parallel structure damage identification model is created, which includes an image classification model, a target detection model and a semantic segmentation model. Each layer of the model meets different needs and environments and can be activated independently or complete the identification task in stages and levels.

[0097] Specifically, a structural damage data set of a multi-level composite structure in the engineering field is obtained, a primary damage identification model and a secondary damage identification model are sequentially constructed based on the structural damage data set, and the primary damage identification model and the secondary damage identification model are sequentially trained, including the following steps:

[0098] S11. Obtain image data of multi-level composite structures in the engineering field from the data set and establish a structural damage data set.

[0099] It should be noted that a large number of structural damage images are collected from public data sets or by using drones, fixed cameras, handheld devices, etc., covering different types of damage (such as cracks, corrosion, deformation, etc.) and normal structural images to ensure that the data set is diverse and representative. At the same time, it is necessary to combine the actual application scenarios to collect structural images from multiple angles and under different lighting and weather conditions to enhance the model's adaptability to complex environments and obtain a structural damage data set.

[0100] S12. Preprocess the image data in the structural damage dataset.

[0101] Specifically, preprocessing the image data in the structural damage dataset includes the following steps:

[0102] S121, performing an image scaling operation on sample data in the structural damage data set;

[0103] S122, performing color adjustment on the sample data after the image scaling operation to modify the color attribute of the sample data;

[0104] S123, performing rotation correction on the sample data after color modification, and performing noise reduction processing on the sample data after rotation correction.

[0105] It should be noted that image scaling is to scale the image to a size suitable for deep learning model input (such as 224*224, 512*512, etc.) to ensure consistent image size and facilitate subsequent model training and reasoning; color adjustment is to modify the color properties of the image, such as brightness, contrast, saturation, and hue, to enhance image features or adapt to specific analysis needs; rotation and correction is to adjust the direction of the image as needed, including rotation and flipping; denoising is to use filtering or denoising technology to reduce the noise in the image, such as uneven lighting, blur, etc., to improve the image quality.

[0106] S13. Annotate all image data in the preprocessed structural damage dataset to obtain primary and secondary structural damage datasets, and divide the primary and secondary structural damage datasets into corresponding training sets and validation sets according to preset standards. The secondary structural damage dataset includes target and segmented structural damage datasets.

[0107] It should be noted that all images in the structural damage dataset are annotated, and the annotation information includes the damage type, spatial location and boundary of the damage, which is used for subsequent target detection and semantic segmentation model training. Among them, the image classification task only needs to mark the image as "no damage" or "damaged" to obtain the image structural damage dataset, which is the first-level structural damage dataset; the target detection task needs to draw a bounding box for the damaged area in the image to obtain the target structural damage dataset; the semantic segmentation task requires manual selection of the damaged area on the image and labeling of the category to obtain the segmented structural damage dataset, among which the target and segmented structural damage datasets both belong to the second-level structural damage dataset.

[0108] The primary structure damage dataset, target structure damage dataset, and segmentation structure damage dataset are divided into training set and validation set according to the ratio of 80% for training and 20% for validation, and stored according to the requirements of image classification, target detection, and semantic segmentation tasks. The specific division steps are as follows:

[0109] The primary structural damage dataset is used for image classification layer model training: the training set is used to train the image classification model in order to learn how to classify images into "non-damaged" or "damaged" categories. Each image only requires one classification label ("normal" or "damaged"). The target structure damage dataset is used for target detection layer training: the specific location of the damage is selected so that each image not only has a classification label but also contains the bounding box information of the damage. The segmentation structure damage dataset is used for semantic segmentation layer training: the damaged area is manually selected and outlined. Each selected area on the image corresponds to a pixel-level annotation label, which is used to accurately describe the boundary and shape of the damaged area.

[0110] S14. Based on the deep learning algorithm, a primary damage recognition model and a secondary damage recognition model are constructed. The secondary damage recognition model includes a target detection model and a semantic segmentation model.

[0111] Specifically, based on the deep learning algorithm, a primary damage recognition model and a secondary damage recognition model are constructed. The secondary damage recognition model includes a target detection model and a semantic segmentation model, and includes the following steps:

[0112] S141. According to the deep learning algorithm, a ResNet-50 architecture is designed, and an image classification model is constructed based on the ResNet-50 architecture. The image classification model is a first-level damage recognition model.

[0113] S142. Design the YoLo-v8 architecture based on the deep learning algorithm, and build a target detection model based on the YoLo-v8 architecture.

[0114] S143. According to the deep learning algorithm, design the Deeplab semantic segmentation network architecture, and build a semantic segmentation model based on the Deeplab semantic segmentation network architecture.

[0115] It should be noted that the image classification model only classifies the entire image and determines whether the image contains damage, but lacks detailed information on the location or features of the damage; the object detection model can not only identify the damage, but also locate it using a bounding box, which is very suitable for identifying multiple objects, but still cannot capture the exact shape of the damage; the semantic segmentation model adopts an encoder-decoder structure, which gradually reduces the spatial dimension of the image while increasing the feature dimension in the encoder stage; in the decoder stage, the spatial resolution of the image is gradually restored, the feature dimension is reduced, and the perception is increased by using void convolution without increasing the number of network parameters or the amount of computation, and multi-scale information is captured through void spatial pyramid pooling, which usually combines image features of different scales to improve segmentation results, etc. The image is analyzed at the pixel level to classify each pixel, providing precise information about the location, shape, size and boundary of the damage.

[0116] S15. Use the training set of the primary structural damage data set to train the primary damage identification model.

[0117] Specifically, training the primary damage identification model using the training set of the primary structural damage data set includes the following steps:

[0118] S151. Input the training set of the primary structural damage data set into the image classification model, and extract image features layer by layer through the deep network structure of the image classification model.

[0119] S152. Based on the extracted image features, classification prediction is performed through a fully connected layer and an activation function at the end of the deep network structure, and the convergence of the image classification model is evaluated using a binary cross entropy loss function.

[0120] Specifically, the formula for the binary cross entropy loss function is:

[0121]

[0122] In the formula, Loss(1) represents the total loss value of the image classification model;

[0123] N represents the total number of sample data;

[0124] i represents the index;

[0125] y i represents the actual category distribution probability of the i-th category, which takes the value of 0 or 1 (0 represents "no damage" and 1 represents "damage");

[0126] y i ′ represents the predicted category distribution probability of the image classification model of the i-th category, that is, the probability that the image belongs to “damaged”, and its value range is [0, 1].

[0127] S153. Update the parameters of the image classification model using the back propagation algorithm and the gradient descent algorithm.

[0128] S154. Repeat steps S151 to S153 until a preset stop condition is reached to obtain a damage classification result.

[0129] S16. Use the training set in the secondary structure damage data set to train the secondary damage recognition model.

[0130] Specifically, training the secondary damage recognition model using the training set in the secondary structure damage data set includes the following steps:

[0131] S161. Input the training set of the target structure damage data set into the target detection model, extract the position features through the convolution layer and predict the damage location at multiple scales, use the target detection loss function and gradient descent algorithm to optimize the target detection model, so that after the input image is processed, the output includes the damage location and category prediction results.

[0132] Specifically, the formula of the target detection loss function is:

[0133]

[0134] In the formula, Loss(2) represents the total loss value of the target detection model;

[0135] λ CIoU represents the weight of the bounding box regression loss;

[0136] L CIoU represents the bounding box regression loss;

[0137] λ Classification Represents the weight of classification loss;

[0138] L Classification represents the classification loss;

[0139] λ Confidence Represents the weight of confidence loss;

[0140] L Confidence represents the confidence loss;

[0141] IoU represents the intersection over union of the predicted bounding box and the true bounding box;

[0142] ρ 2 (b,b g ) represents the square of the Euclidean distance between the center point of the predicted box and the center point of the real box;

[0143] c represents the diagonal length of the minimum bounding box of the predicted box and the real box;

[0144] α represents the consistency correction term of aspect ratio;

[0145] v represents the consistency correction term of aspect ratio;

[0146] C represents the number of categories;

[0147] i represents the index;

[0148] y i Represents the actual category distribution probability of the i-th category, which takes a value of 0 or 1;

[0149] y i ′ represents the predicted category distribution probability of the target detection model of the i-th category;

[0150] y obj The true label indicating the existence of the target, with a value of 0 or 1;

[0151] y obj ′ represents the confidence of the object existence predicted by the target detection model.

[0152] It should be noted that the gradient descent method is used to optimize the target detection loss function, which usually includes the loss of bounding box coordinates, category loss and confidence loss. Through back propagation, the weights of the model are iteratively updated multiple times until the performance of the target detection model reaches the optimal level or the loss function converges, and the optimal target detection layer model is obtained and saved.

[0153] S162. Input the training set of the segmented structural damage data set into the semantic segmentation model, classify each pixel, calculate the pixel-level cross entropy loss function and use the gradient descent algorithm to optimize the semantic segmentation model, so that each pixel after the input image is processed is classified as damaged or non-damaged, and output the damaged area image data.

[0154] Specifically, the formula for the pixel-level cross entropy loss function is:

[0155]

[0156] In the formula, Loss(3) represents the total loss value of the semantic segmentation model;

[0157] N represents the total number of sample data;

[0158] i represents the index;

[0159] y i Represents the true label of the i-th pixel, which takes a value of 0 or 1 (0 represents the background, 1 represents the target area);

[0160] y i ′ represents the probability that the semantic segmentation model predicts that the i-th pixel belongs to the target area.

[0161] It should be noted that the pixel-level cross entropy loss function is used, and the network is trained through the gradient descent algorithm. The weights of the semantic segmentation model are updated repeatedly until the loss function converges, and the optimal semantic segmentation layer model is obtained and saved.

[0162] In addition, during the training process, each layer of the model is evaluated and optimized on its corresponding structural damage dataset. According to the task characteristics of the image classification layer, object detection layer and semantic segmentation layer, different performance evaluation indicators such as accuracy, recall rate, precision and F1 score are used to comprehensively measure the detection and recognition effects of the model.

[0163] Based on the deep learning architecture, a three-layer structural damage identification model is built to meet the needs of different detection tasks. The model is trained on the corresponding training data set, and the preprocessed data is used to train the model on the deep learning processing unit. The network parameters are adjusted to minimize the loss function, and the model performance is monitored through the validation set to obtain a high-precision structural damage identification model. Through the self-learning and optimization of the deep learning model, the system has good adaptability and scalability, and can cope with the detection needs in various complex environments.

[0164] By combining the above three-level recognition modules to meet different needs and environments, a multi-model fusion strategy is adopted in the deep learning processing and analysis steps. Each layer can be activated independently or complete the recognition task in stages, allowing large-scale screening of potential problem areas from the first-level recognition (image classification layer) to the second-level recognition of these problems (target detection layer), or the second-level recognition of the scope and shape of the problem (semantic segmentation layer). A multi-model fusion strategy is adopted. Each layer can be activated independently or complete the recognition task in stages, or the analysis results of multiple deep neural network models are integrated to improve the accuracy and reliability of damage identification.

[0165] S2. Based on the trained primary damage recognition model, perform primary damage recognition on the image data in the structural damage data set, and divide the image data in the structural damage data set into a non-damage image data set and a damaged image data set based on the primary damage recognition result.

[0166] like Figure 4 As shown in the figure, the workflow diagram of the image classification layer is displayed, which includes the image input to the first-level recognition model to determine whether it contains damage. For the image blocks judged as undamaged by the first-level recognition model, no processing is performed, and the original image is saved until the final result is output. For the image blocks judged as damaged by the first-level recognition model, if the task not only needs to determine whether there is damage, but also does not require precise positioning of the damage, but only needs to determine whether there is damage, the target detection layer and semantic segmentation layer of the second-level recognition can be skipped to directly output the recognition result of the image classification layer. If it is also necessary to locate the damaged area, the image part identified as damaged in the image classification layer can be input into the target detection layer model or the semantic segmentation layer model of the second-level recognition for more accurate recognition.

[0167] S3. Based on the trained secondary damage recognition model, secondary damage recognition is performed on the image data in the damaged image data set to obtain secondary damage recognition results.

[0168] It is necessary to add that, if Figure 5As shown in the figure, the workflow diagram of the target detection layer is displayed. For the image block judged as damaged by the first-level recognition model, if the task requirement does not require precise positioning of the damage, or if the task requirement requires positioning the approximate area of ​​the damage but does not require precise range and boundaries, the image portion identified as damaged in the image classification layer can be input into the target detection layer model of the second-level recognition, and the damaged area location with a bounding box mark is output. If the task requirement requires identifying the precise range and boundaries of the damage, the image portion identified as damaged in the image classification layer can be input into the semantic segmentation layer model of the second-level recognition, and the recognition result of accurately positioning the boundary of the damaged area is output.

[0169] like Figure 6 As shown in the figure, the workflow diagram of the semantic segmentation layer is displayed. For an image block that is judged as damaged by the first-level recognition model, if the task requirement does not require precise positioning of the damage, but if the task requirement requires identification of the precise range and boundary of the damage, the image portion identified as damaged in the image classification layer can be input into the semantic segmentation layer model of the second-level recognition, and the recognition result of accurately locating the boundary of the damaged area can be output.

[0170] In addition, the first-level damage recognition model (image classification model) is used to perform preliminary screening of the input image to determine whether there is structural damage; the second-level damage recognition model (target detection model and semantic segmentation model), the target detection model is used to locate the damaged area and determine the specific location of the damage through the bounding box; the semantic segmentation model is used to perform pixel-by-pixel segmentation of the damaged area to obtain the shape, boundary and specific range of the damage.

[0171] In a specific embodiment, the workflow is:

[0172] (1) Pre-observation of the test object: Analyze the structural characteristics of the test object and formulate a test and data collection plan;

[0173] (2) Damage data collection: Use cameras, drones, fixed cameras, handheld devices, etc. to collect image data of the inspection object according to the data collection plan to obtain the original image data to be inspected;

[0174] (3) Image preprocessing: Screen the original image data to be detected, remove duplicate images, retain high-quality and clear images, and obtain the image data set to be detected;

[0175] (4) Multi-level damage identification: According to different task requirements and scenarios, the first-level damage identification model and the second-level damage identification model are used to automatically detect and identify the image data set to be inspected, and identify detailed information such as the type or location of the damage.

[0176] Among them, the multi-level composite deep learning model has broad application prospects, especially in the following fields:

[0177] Building and infrastructure health monitoring: regular health checks and damage detection of structures such as bridges, tunnels, and large buildings, providing efficient and accurate damage detection reports; Energy and industrial facilities: damage detection and maintenance of key infrastructure such as power towers, oil and gas pipelines, and offshore platforms.

[0178] Through the construction and optimization of multi-level deep learning models, the automation, efficiency and precision of structural damage detection are achieved. Its flexibility, real-time and robustness significantly improve the shortcomings of traditional methods. It can provide a full range of solutions in the field of complex structural health monitoring and effectively respond to the growing needs of infrastructure safety management.

[0179] like Figure 7 As shown, according to another embodiment of the present invention, a multi-level composite structure damage identification system based on deep learning is also provided, the system includes a model building and training module 1, a primary damage identification module 2 and a secondary damage identification module 3;

[0180] Model building and training module 1 is used to obtain a structural damage data set of a multi-level composite structure in the engineering field, build a primary damage identification model and a secondary damage identification model based on the structural damage data set, and train the primary damage identification model and the secondary damage identification model in turn;

[0181] The primary damage recognition module 2 is used to perform primary damage recognition on the image data in the structural damage data set based on the trained primary damage recognition model, and divide the image data in the structural damage data set into a non-damage image data set and a damaged image data set based on the primary damage recognition result;

[0182] The secondary damage recognition module 3 is used to perform secondary damage recognition on the image data in the damaged image data set based on the trained secondary damage recognition model to obtain the secondary damage recognition result.

[0183] In summary, with the help of the above technical solution of the present invention, by adopting a multi-level composite deep learning model, including three layers of image classification, target detection and semantic segmentation, structural damage is analyzed from different angles; the image classification layer quickly screens and determines whether there is damage, and filters a large number of normal images to reduce the subsequent calculation burden; the target detection layer locates the damaged area and determines the approximate position through the bounding box; the semantic segmentation layer accurately segments the damaged area, obtains the specific shape and boundary of the damage, and realizes pixel-by-pixel recognition; the multi-level architecture covers detection tasks of different complexities, from rough damage detection to precise segmentation, and comprehensively covers the needs of structural damage detection, greatly improving the accuracy and comprehensiveness of recognition. Through the deep learning model, the whole process from image acquisition to damage recognition is automated, the false positive rate and missed detection rate are reduced, and the model can autonomously learn and extract complex damage features by training with a large-scale structural damage data set, realize automated structural health monitoring, and improve detection efficiency and accuracy; in addition, the improvement of the degree of automation reduces the labor intensity of manual detection, which is especially suitable for large infrastructure such as bridges and tunnels and high-risk areas that are difficult for humans to reach. The multi-level model design is flexible and adaptable. Different levels of models can be selected to work together or independently according to task requirements, damage complexity and hardware resource conditions. In scenarios where resources are limited or rapid detection is required, the system can only use the image classification layer or the target detection layer. In high-precision detection scenarios, the target detection and semantic segmentation layers can be enabled to achieve precise positioning and segmentation. This flexible architecture enables the system to adapt to various applications from low-resource environments to high-precision requirements, thereby providing targeted solutions. Through the optimization of multi-level deep learning models, it can efficiently handle structural damage detection tasks, especially in the detection of large-scale structures (such as bridges and buildings), showing significant real-time and high efficiency. The image classification layer quickly screens normal structural images, reduces the calculation burden of subsequent models, and improves detection efficiency. The target detection layer and the semantic segmentation layer achieve efficient positioning and damage segmentation through optimized loss functions and calculation methods, reduce detection time, and meet real-time monitoring needs. The system's high efficiency supports rapid response in scenarios with high detection frequency and large data volume, and is suitable for regular monitoring of large infrastructure and real-time detection of emergencies.

[0184] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-level composite structure damage identification method based on deep learning, characterized in that: The method comprises the following steps: S1. Obtain a structural damage data set of a multi-level composite structure in the engineering field, construct a primary damage identification model and a secondary damage identification model based on the structural damage data set, and train the primary damage identification model and the secondary damage identification model in turn; S2. Based on the trained primary damage recognition model, perform primary damage recognition on the image data in the structural damage data set, and divide the image data in the structural damage data set into a non-damage image data set and a damaged image data set based on the primary damage recognition result; S3. Based on the trained secondary damage recognition model, secondary damage recognition is performed on the image data in the damaged image data set to obtain secondary damage recognition results.

2. According to the multi-level composite structure damage identification method based on deep learning in claim 1, it is characterized in that: The method of obtaining a structural damage data set of a multi-level composite structure in the engineering field, sequentially constructing a primary damage identification model and a secondary damage identification model based on the structural damage data set, and sequentially training the primary damage identification model and the secondary damage identification model comprises the following steps: S11, obtaining image data of multi-level composite structures in the engineering field from the data set, and establishing a structural damage data set; S12, preprocessing the image data in the structural damage data set; S13, annotating all image data in the preprocessed structural damage dataset, obtaining primary and secondary structural damage datasets, and dividing the primary and secondary structural damage datasets into corresponding training sets and validation sets according to preset standards, wherein the secondary structural damage dataset includes target and segmented structural damage datasets; S14. Based on the deep learning algorithm, construct a primary damage recognition model and a secondary damage recognition model, wherein the secondary damage recognition model includes a target detection model and a semantic segmentation model; S15, training the primary damage identification model using the training set of the primary structural damage data set; S16. Use the training set in the secondary structure damage data set to train the secondary damage recognition model.

3. The multi-level composite structure damage identification method based on deep learning according to claim 2 is characterized in that: The preprocessing of the image data in the structural damage data set comprises the following steps: S121, performing an image scaling operation on sample data in the structural damage data set; S122, performing color adjustment on the sample data after the image scaling operation to modify the color attribute of the sample data; S123, performing rotation correction on the sample data after color modification, and performing noise reduction processing on the sample data after rotation correction.

4. The multi-level composite structure damage identification method based on deep learning according to claim 3 is characterized in that: The method of constructing a primary damage recognition model and a secondary damage recognition model based on a deep learning algorithm, wherein the secondary damage recognition model includes a target detection model and a semantic segmentation model, comprises the following steps: S141. According to the deep learning algorithm, a ResNet-50 architecture is designed, and an image classification model is constructed based on the ResNet-50 architecture, where the image classification model is a primary damage recognition model; S142. Design the YoLo-v8 architecture based on the deep learning algorithm, and build a target detection model based on the YoLo-v8 architecture; S143. According to the deep learning algorithm, design the Deeplab semantic segmentation network architecture, and build a semantic segmentation model based on the Deeplab semantic segmentation network architecture.

5. The multi-level composite structure damage identification method based on deep learning according to claim 4 is characterized in that: The training of the primary damage identification model using the training set of the primary structural damage data set comprises the following steps: S151, inputting the training set of the primary structural damage data set into the image classification model, and extracting image features layer by layer through the deep network structure of the image classification model; S152. Based on the extracted image features, classification prediction is performed at the end of the deep network structure through a fully connected layer and an activation function, and the convergence of the image classification model is evaluated using a binary cross entropy loss function. S153, using the back propagation algorithm and the gradient descent algorithm to update the parameters of the image classification model; S154. Repeat steps S151 to S153 until a preset stop condition is reached to obtain a damage classification result.

6. The multi-level composite structure damage identification method based on deep learning according to claim 5 is characterized in that: The formula of the binary cross entropy loss function is: In the formula, Loss(1) represents the total loss value of the image classification model; N represents the total number of sample data; i represents the index; y i Represents the actual category distribution probability of the i-th category; y i ′ represents the predicted category distribution probability of the image classification model of the i-th category.

7. The multi-level composite structure damage identification method based on deep learning according to claim 6 is characterized in that: The training of the secondary damage recognition model using the training set in the secondary structure damage data set comprises the following steps: S161, inputting the training set of the target structure damage data set into the target detection model, extracting the position features through the convolution layer and predicting the damage position at multiple scales, optimizing the target detection model using the target detection loss function and the gradient descent algorithm, so that the output after the input image processing includes the damage position and the category prediction result; S162. Input the training set of the segmented structural damage data set into the semantic segmentation model, classify each pixel, calculate the pixel-level cross entropy loss function and use the gradient descent algorithm to optimize the semantic segmentation model, so that each pixel after the input image is processed is classified as damaged or non-damaged, and output the damaged area image data.

8. The multi-level composite structure damage identification method based on deep learning according to claim 7 is characterized in that: The formula of the target detection loss function is: In the formula, Loss(2) represents the total loss value of the target detection model; λ CIoU represents the weight of the bounding box regression loss; L CIoU represents the bounding box regression loss; λ Classification Represents the weight of classification loss; L Classification represents the classification loss; λ Confidence Represents the weight of confidence loss; L Confidence represents the confidence loss; IoU represents the intersection over union of the predicted bounding box and the true bounding box; ρ 2 (b,b g ) represents the square of the Euclidean distance between the center point of the predicted box and the center point of the real box; c represents the diagonal length of the minimum bounding box of the predicted box and the real box; α represents the consistency correction term of aspect ratio; v represents the consistency correction term of aspect ratio; C represents the number of categories; i represents the index; y i Represents the actual category distribution probability of the i-th category; y i ′ represents the predicted category distribution probability of the target detection model of the i-th category; y obj The true label indicating the existence of the target; y obj ′ represents the confidence of the object existence predicted by the target detection model.

9. The multi-level composite structure damage identification method based on deep learning according to claim 8, characterized in that: The formula of the pixel-level cross entropy loss function is: In the formula, Loss(3) represents the total loss value of the semantic segmentation model; N represents the total number of sample data; i represents the index; y i represents the true label of the i-th pixel; y i ′ represents the probability that the semantic segmentation model predicts that the i-th pixel belongs to the target area.

10. A multi-level composite structure damage identification system based on deep learning, used to implement the multi-level composite structure damage identification method based on deep learning according to any one of claims 1 to 9, characterized in that: The system includes a model building and training module, a primary damage identification module, and a secondary damage identification module; The model building and training module is used to obtain a structural damage data set of a multi-level composite structure in the engineering field, sequentially build a primary damage identification model and a secondary damage identification model based on the structural damage data set, and sequentially train the primary damage identification model and the secondary damage identification model; The primary damage recognition module is used to perform primary damage recognition on the image data in the structural damage data set based on the trained primary damage recognition model, and divide the image data in the structural damage data set into a non-damage image data set and a damaged image data set based on the primary damage recognition result; The secondary damage recognition module is used to perform secondary damage recognition on the image data in the damaged image data set based on the trained secondary damage recognition model to obtain the secondary damage recognition result.

Images