Civil engineering structure apparent damage small sample intelligent diagnosis method and system

Abstract

The invention provides a civil engineering structure apparent damage small sample intelligent diagnosis method and system, and relates to the field of artificial intelligence. Comprising the following steps: S1, inputting a labeled source domain data set and a labeled small sample target domain data set, and carrying out data preprocessing; s2, training an encoder through an adversarial training mode containing domain adversarial loss, and extracting domain invariant features; s3, extracting domain invariant features from the target domain data set by using the trained encoder to obtain a target domain invariant feature set; using the target domain invariant feature set training condition to generate an adversarial network, and obtaining a target domain sample set; combining the target domain data set and the target domain sample set to form a mixed training set; s4, using the mixed training set to finely adjust a diagnosis network; and S5, extracting features of a target domain image to be diagnosed by the trained encoder, and inputting the features into the trained diagnosis network to obtain a damage identification result. According to the invention, high-precision, high-robustness and high-adaptability intelligent structural damage diagnosis is realized.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to an intelligent diagnostic method and system for small samples of apparent damage to civil engineering structures. Background Technology

[0002] During long-term service, civil engineering structures (such as bridges, buildings, and tunnels) are subject to environmental erosion, material aging, and load effects, resulting in various types of damage on their surfaces, including cracks, spalling, rust swelling, and leakage. Timely and accurate identification and classification of this damage are fundamental to assessing structural health and developing scientific maintenance strategies. Traditional damage detection methods mainly rely on manual inspections, which suffer from low efficiency, high subjectivity, high risk, and difficulty in quantification.

[0003] In recent years, with the advancement of computer vision and deep learning technologies, automated damage recognition methods based on convolutional neural networks (CNNs) have become a research hotspot. Existing technical solutions typically follow this process: 1. Use high-definition cameras or drones to collect a large number of structural appearance images; 2. Domain experts perform fine-grained pixel-level or region-level annotations on the damaged areas in the images to form a large-scale, high-quality training dataset; 3. Use this dataset (e.g., tens of thousands of labeled images) to train a deep CNN model (such as Faster R-CNN, Mask R-CNN, DeepLab, etc.). 4. Deploy the trained model on-site to automatically identify and classify damage in newly acquired images.

[0004] However, the aforementioned existing technologies have the following significant drawbacks and limitations: (1) The contradiction between data demand and quality is prominent: To achieve high accuracy and strong generalization ability, deep CNN models usually require a massive amount (usually tens of thousands or even hundreds of thousands of images) of training samples with extremely high annotation quality. However, in actual civil engineering applications, it is extremely difficult and costly to obtain such a large amount of labeled data covering various types of damage, different environments, and multiple structural backgrounds. On the one hand, damage samples of specific structures are scarce; on the other hand, data annotation work is highly dependent on experts with profound civil engineering knowledge, which is time-consuming and labor-intensive. This has hindered the application of existing deep learning-based methods in real-world scenarios where data is scarce, forming a fundamental contradiction between "strong model capabilities" and "limited data supply".

[0005] (2) Poor domain adaptability: Most studies train and test models on publicly available, limited benchmark datasets, where the acquisition environment, imaging equipment, lighting conditions, and structural backgrounds are relatively uniform. When the trained model is directly applied to new engineering sites with different characteristics (e.g., a model trained on a concrete bridge dataset is applied to brick and stone ancient buildings, or a model trained on sunny day data is applied to images acquired on rainy days), the model performance drops sharply due to the difference in domain distribution. Existing technologies lack effective mechanisms to bridge the feature distribution differences between the source domain (training data) and the target domain (actual application data).

[0006] (3) Insensitive to small sample sizes and novel damage types: When the number of samples of a certain type of damage is extremely small (for example, there are only a few images of a rare erosion pattern) or when a novel type of damage that has not appeared in the training set appears, traditional supervised learning-based CNN models cannot effectively identify and generalize. The model is essentially fitting the existing data distribution, which makes it difficult to deal with the "long-tailed distribution" problem and damage category identification in the open world.

[0007] (4) Insufficient interpretability of the model: Existing methods are mostly regarded as "black boxes". The model only gives the location and category of damage, but cannot explain the visual basis for its judgment (such as whether it depends on the texture of the crack or the edge features). This reduces the civil engineers' trust in the automated diagnosis results and makes it difficult to associate the diagnosis results with the underlying mechanical mechanism and degradation cause.

[0008] While existing patented technologies (such as CN113822880A "A Crack Recognition Method Based on Deep Learning") and academic research have continuously improved model accuracy, they have not systematically and fundamentally solved the three core challenges of "small sample size", "cross-domain", and "weak annotation". Summary of the Invention

[0009] Therefore, the technical problem to be solved by the present invention is to provide a small sample intelligent diagnosis method and system for apparent damage to civil engineering structures, which can still achieve high-precision, robust and highly adaptable intelligent diagnosis of structural damage in real civil engineering environments where training samples are scarce, annotation costs are high and application scenarios are varied.

[0010] In a first aspect, the present invention provides an intelligent diagnostic method for small samples of apparent damage to civil engineering structures, comprising the following steps: S1: Input the labeled source domain dataset and the labeled small sample target domain dataset, and perform data preprocessing; S2, using the preprocessed source domain dataset and target domain dataset, train the encoder through adversarial training with domain adversarial loss to extract domain-invariant features that are insensitive to the source domain or target domain; S3, using the trained encoder to extract domain-invariant features from the target domain dataset to obtain the target domain-invariant feature set; A generative adversarial network is trained using the target domain invariant feature set to obtain the target domain sample set; The target domain dataset and the target domain sample set are merged to form a hybrid training set; S4, Use the hybrid training set to train a diagnostic network for damage classification; S5, the target domain image to be diagnosed is first processed by the trained encoder to extract features, and then input into the trained diagnostic network to obtain the damage recognition result.

[0011] Furthermore, S2 specifically includes: S21, using the pre-trained convolutional neural network's front convolutional layer as the encoder. E The initial weights; Building a classifier C Domain discriminant And randomly initialize the classifier. C Domain discriminant The weights; the domain discriminator It includes a binary classifier composed of fully connected layers to determine whether a feature comes from the source domain or the target domain; S22, Perform the adversarial training according to the following optimization objectives;

[0012] in, To balance the hyperparameters of the two losses; To combat losses in the domain, To monitor and categorize losses; These represent the batch sample size for the source domain and the target domain, respectively. For the first One source domain sample image; For the first One target domain sample image; The probability value output by the domain discriminator; The feature vector extracted by the encoder; K Total number of damage categories; For source domain samples The true category label; For the classifier connected after the encoder C For source domain samples Category prediction probability; S23, repeat S22 until the model's classification accuracy on the validation set stabilizes, and the domain discriminator... It cannot effectively distinguish features from the source domain and the target domain.

[0013] Furthermore, the forward propagation of the adversarial training includes: Will and Input the encoder respectively , to obtain features and ; Will Input the classifier Source domain prediction ; Will and Input the domain discriminator To obtain the domain prediction probability and ; in, For a batch of source domain image data; This refers to a batch of target domain image data.

[0014] Furthermore, the backpropagation and parameter update of the adversarial training include: maximize Updated via backpropagation parameters The updated formula is:

[0015] minimize ,fixed Update the encoder via backpropagation and classifier parameters and The updated formula is:

[0016] in, The learning rate; This is the gradient operator.

[0017] Furthermore, S3 specifically includes: S31, the preprocessed target domain dataset All images are input into the trained encoder Extracting the set of domain-invariant feature vectors and obtain its corresponding tag. ; S32, Construct a conditional generative adversarial network and train the discriminator. Training generator ; S33, Once the generated image quality meets the preset criteria, freeze the trained generator. ; For each type of damage in the target domain, random sampling noise is used. Combined with the conditional vector of this type of damage, using Batch generation of synthetic images constitutes the generated dataset. ; S34, and The samples are merged to form the final hybrid training set. ; The labels in the sample inheritance generation conditions.

[0018] Furthermore, in S32, The conditional generative adversarial network includes: a build generator. and discriminator ; The input is noise and condition vector The output is a composite image. ; The input is a real image. or composite image and its corresponding condition vector Embedding ) represents the label embedding vector; The training discriminator Includes: sampling a batch of real target domain images and its conditions ; and a batch of generators Synthetic image ; Calculate the loss function of the discriminator Dg and update The parameters make maximize; The training generator Includes: fixed Generate a batch of images ; Calculate loss ;renew The parameters make minimize.

[0019] Furthermore, in S3, the training objective of the conditional generative adversarial network is:

[0020] Where G is the generator; For target domain image samples; For the generated target domain image; The distribution of the generated image; The distribution of the real target domain image; It is a conditional vector; Points are randomly sampled along the line connecting real data points and generated data points; represents the weight coefficient of the gradient penalty term.

[0021] Further, S includes: The input to the diagnostic network is the images in the mixed training set; The diagnostic network outputs the following: for classification tasks, it outputs the probability distribution of damage categories; for detection or segmentation tasks, it outputs bounding boxes or pixel-level segmentation masks. The training loss uses cross-entropy loss. :

[0022] in This is the batch size of the mixed training set.

[0023] Furthermore, following step S5, the following steps are also included: S6, for the predicted category The Grad-CAM algorithm is used to calculate the feature map of the last convolutional layer of the network. The gradient is used to generate a class activation heatmap as follows. ;

[0024] in, Z represents the activation value at spatial location (i,j) in the k-th feature map of the last convolutional layer of the diagnostic network; Z is the k-th feature map. Total number of pixels; To diagnose the last convolutional layer of the network Each feature map; Heatmap Upsample to the original image size, overlay it with the original image, and then visualize the output.

[0025] Secondly, the present invention provides a small-sample intelligent diagnostic system for apparent damage to civil engineering structures, the system comprising: The data preprocessing module is used to take the labeled source domain dataset and the labeled small sample target domain dataset as input and perform data preprocessing. The domain-adaptive feature extraction module uses the preprocessed source domain dataset and the target domain dataset to train the encoder through an adversarial training method that includes domain adversarial loss, and extracts domain-invariant features that are insensitive to the source domain or the target domain. The conditional sample generation module extracts domain-invariant features from the target domain dataset using the trained encoder to obtain a target domain-invariant feature set, trains a conditional generative adversarial network using the target domain-invariant feature set to obtain a target domain sample set, and merges the target domain dataset and the target domain sample set to form a hybrid training set. The diagnostic model training module uses the hybrid training set to train a diagnostic network for damage classification. The damage intelligent diagnosis module is used to extract features from the target domain image to be diagnosed by the trained encoder, and then input the features into the trained diagnosis network to obtain the damage recognition result.

[0026] Beneficial effects: 1. This invention solves the problem of model training and generalization under small sample conditions: using a very small number of target domain damage images (e.g., less than 50 samples per type of damage) or even only image-level weak annotations, a stable and reliable damage recognition model can be trained, breaking the dependence of deep learning on massive amounts of labeled data.

[0027] 2. Solved the performance degradation problem caused by cross-domain distribution differences: An adaptive mechanism was designed to enable knowledge acquired from one data source (such as public datasets or simulation data) to be effectively transferred and adapted to another real engineering target scenario with different data distribution characteristics (such as different materials, lighting, and seasons), thereby improving the domain robustness of the model.

[0028] 3. Achieved data diversity under limited samples: Based on the existing small number of real samples, a large number of new samples with reasonable semantics, diverse features and conforming to the visual characteristics of the target domain are generated, thereby expanding the training set at the data level and enriching the damage morphology and background changes seen by the model.

[0029] 4. Improved interpretability and credibility of the model decision-making process: During the diagnosis process, not only is the type and location of the damage output, but also key visual evidence (such as salient areas) is provided for the model to make this judgment, making the diagnosis results more convincing and verifiable for civil engineers.

[0030] 5. An end-to-end, lightweight complete system was built: the above technologies (transfer, generation, diagnosis, interpretation) were organically integrated to form an automated system from input of a small amount of target data to output of the final damage diagnosis report, while taking into account the convenience of actual deployment and computational efficiency. Attached Figure Description

[0031] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0032] Figure 1 This is an architecture diagram of the intelligent diagnostic system for small samples of apparent damage to civil engineering structures, as described in Embodiment 1 of the present invention.

[0033] Figure 2 This is a flowchart illustrating the workflow of the intelligent diagnostic method for small samples of apparent damage to civil engineering structures in Embodiment 2 of the present invention. Detailed Implementation

[0034] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. The principles and features of the present invention are described below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other. The embodiments given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0035] Example 1 This embodiment proposes a two-stage training framework that integrates domain-adaptive transfer learning and conditional generative adversarial networks (GANs) to construct a corresponding intelligent diagnostic system. The core idea is as follows: First, using large-scale source domain data (such as public datasets) and a small amount of target domain data, a domain-adaptive module learns domain-invariant feature representations. Then, based on these features and a small number of real samples, a conditional GAN ​​is used to collaboratively generate high-quality, diverse target domain damage samples in both the feature space and image space. Finally, a lightweight diagnostic network is fine-tuned using the augmented hybrid dataset, and an interpretability module is integrated.

[0036] The system architecture of this embodiment is as follows: Figure 1 As shown, it mainly includes five core modules: data preprocessing module, domain-adaptive feature extraction module, conditional sample generation module, damage intelligent diagnosis module, and interpretable visualization module.

[0037] 1.1 Domain Adaptive Feature Extraction Module The goal of this module is to learn a shared feature encoder. This makes the source domain and target domain The image after After encoding, its high-level feature distribution is aligned as much as possible, thereby extracting domain-invariant features that are insensitive to changes in the domain.

[0038] Network structure: Encoder Typically, the first few layers of a ResNet or VGG pre-trained on ImageNet are used as the backbone. This is then followed by a domain discriminator. It is a binary classifier consisting of fully connected layers, used to determine whether a feature comes from the source domain or the target domain.

[0039] Adversarial training objective: The training process follows the principles of generative adversarial networks. Encoder Attempting to generate a domain discriminator Indistinguishable features, and Then we strive to distinguish them. This is achieved by minimizing a domain against loss. To achieve this.

[0040] in: :No. One source domain sample image.

[0041] :No. A sample image of the target domain.

[0042] : Batch sample size for the source and target domains.

[0043] : The probability value output by the domain discriminator (the probability of belonging to the source domain).

[0044] : Feature vector extracted by the encoder.

[0045] At the same time, encoder On the source domain data, a supervised classification loss also needs to be minimized. (For example, cross-entropy loss) to ensure that the extracted features are semantically discriminative.

[0046] in: K Total number of damage categories.

[0047] Source domain samples The true category label (one-hot vector).

[0048] Diagnostic Network (connected to) (Then) for source domain samples Category The predicted probability.

[0049] Therefore, the overall optimization objective of this module is: ; in It is a hyperparameter that balances the two losses. Through this adversarial training, the encoder... Learn to extract domain-invariant features that can accurately classify but are difficult to distinguish in terms of origin. .

[0050] 1.2 Conditional Sample Generation Module This module aims to utilize the acquired domain-invariant characteristics. and corresponding damage category labels To generate new damage samples that conform to the style of the target domain, a conditional generative adversarial network was designed.

[0051] generator The input is a noise vector sampled from a prior distribution (such as a Gaussian distribution). and conditional vectors (by field-invariant features) and / or tags Encoded), the output is the generated image. .

[0052] Discriminator The input is a real target domain image. Or generate an image The output, along with its corresponding conditional information, is the probability of determining the authenticity of the image.

[0053] Conditional Generative Adversarial Loss: Wasserstein distance with gradient penalty (WGAN-GP) is used to improve training stability.

[0054] in: Distribution of generated images.

[0055] Distribution of the real target domain image.

[0056] :Conditional vector, for example .

[0057] Points randomly sampled on the line connecting real data points and generated data points.

[0058] : The weight coefficient of the gradient penalty term.

[0059] In addition, to ensure that the content (damage morphology) of the generated image is consistent with the conditions, a feature matching loss is introduced. The requirement is to generate an image. via encoder Extracted features Conditional features of the input Get as close as possible.

[0060] Therefore, generator The overall training objective is: ; Through training, the generator It can learn the joint distribution of target domain image style and damage semantics, thereby generating a large number of realistic and diverse target domain style damage samples. ,in .

[0061] 1.3 Intelligent Damage Diagnosis Module Original small sample target domain data With generated samples The datasets are merged to form a rich, mixed training set. A lightweight diagnostic network is then trained on this dataset. Fine-tune or train from scratch using (e.g., MobileNetV3, EfficientNet-Lite).

[0062] Diagnostic network input: Images from the mixed training set.

[0063] Diagnostic network output: For classification tasks, outputs the probability distribution of damage categories; for detection / segmentation tasks, outputs bounding boxes or pixel-level segmentation masks.

[0064] Training loss: Depends on the task type. For example, for multi-class classification tasks, the standard cross-entropy loss is used. .

[0065] in This is the batch size of the mixed training set.

[0066] Since the generated samples also have accurate labels (from conditional information), they can effectively participate in supervised training, significantly improving the diagnostic network's generalization ability in the target domain.

[0067] 1.4 Explainability Visualization Module To enhance the reliability of diagnostic results, this system integrates the gradient-based activation graph method. For diagnostic networks... Given the prediction results, calculate their relevance to the final predicted category. The gradient is calculated and backpropagated to the feature map of the last convolutional layer to generate a heatmap. .

[0068] in: Diagnostic network's last convolutional layer Each feature map.

[0069] :category For feature maps The weights are obtained by performing global average pooling on the gradient: .

[0070] Feature map The total number of pixels.

[0071] Used to highlight areas that make a positive contribution to the prediction.

[0072] This heat map When overlaid with the original image, it can intuitively display the key areas of the image on which the model makes decisions (i.e., the location where the damage occurs), providing engineers with visual verification evidence.

[0073] This invention systematically solves the problem of small-sample, cross-domain civil damage diagnosis through collaborative innovation of domain-adaptive alignment features, conditional generation of expanded data, lightweight network for accurate diagnosis, and heatmap for interpretation. It has good practicality, robustness and interpretability.

[0074] Example 2 This embodiment provides an intelligent diagnostic method for small samples of apparent damage to civil engineering structures. Figure 2 The diagram shown is a flowchart of the workflow of this method. The workflow of this method is logically rigorous and progressively advanced.

[0075] The method flow of this invention includes 1) Collect large-scale source domain labeled datasets and small-scale target domain datasets (This may be a weak annotation only).

[0076] 2) Use and Training encoder Domain discriminant Obtain a domain-invariant feature extractor .

[0077] 3) Fix ,use and its characteristics Training Conditional Generative Adversarial Networks (GANs) and Subsequently, using Generate a large number of target domain style samples .

[0078] 4) Merging and Training or fine-tuning lightweight diagnostic networks .

[0079] 5) Input the target domain image to be diagnosed into the trained system. The damage identification results are obtained, and Grad-CAM is used to generate an interpretable heat map, forming the final diagnostic report.

[0080] Specifically, it includes: 2.1 First Stage: Data Preparation and Preprocessing The goal of this stage is to standardize the input data, providing a standardized input for subsequent deep learning models.

[0081] 2.1.1. Data Input: Source domain dataset : Contains a large number of (e.g.) Annotated images of civil structure damage. Annotation information should include at least image-level damage category labels. For better results, bounding boxes or pixel-level segmentation masks can be included. This data can come from public datasets (such as SDNET2018, CrackTree200) or accumulated from historical projects.

[0082] Target domain dataset : Contains a small amount (e.g. Damage images from specific target engineering scenarios (such as "XX Bridge in a certain city"). Labeling requirements are lenient; images can be labeled at the image level (weak supervision), or some images can even be left unlabeled and used solely for feature distribution learning.

[0083] 2.1.2. Data Preprocessing: Size normalization: scaling all images to a uniform size (e.g., ...) or (pixels) to meet the input requirements of the diagnostic network.

[0084] Data augmentation (only for) and some Applications: A series of random transformations are applied to the source domain data to increase diversity, including random horizontal / vertical flipping, small-angle rotation (±10°), and fine-tuning of brightness and contrast. For the target domain data, to avoid destroying its unique domain characteristics (such as specific lighting and texture), only slight standardization processing (such as mean subtraction and variance normalization) is applied that does not affect style characteristics.

[0085] Target domain dataset It can include both labeled and unlabeled images. During the domain-adaptive training phase, all target domain images (regardless of whether they are labeled or not) are used for domain distribution alignment; during the conditional sample generation phase, labeled target domain images and their labels are mainly utilized.

[0086] Dataset partitioning: dividing the source domain data Divide into training sets according to a ratio (e.g., 7:2:1). Validation set and test set Target domain data All of it was used for training.

[0087] 2.1.3. Output: Preprocessed, normalized source domain dataset and target domain dataset .

[0088] 2.2 Second Stage: Domain-Adaptive Feature Extraction The core objective of this stage is to train a shared encoder capable of extracting "domain-invariant features". .

[0089] 2.2.1. Network Initialization: The pre-convolutional layers of a convolutional neural network (e.g., ResNet50) pre-trained on a large natural image dataset (such as ImageNet) serve as the encoder. The initial weights.

[0090] Build a classifier (Typically composed of a global average pooling layer and a fully connected layer) and a domain discriminator. (Constitutes several fully connected layers), and its weights are randomly initialized.

[0091] 2.2.2. Adversarial training iteration: In each training batch, a batch of source domain data is randomly sampled. and a batch of target domain data .

[0092] Forward propagation: Will and Input encoders respectively , to obtain features and .

[0093] Will Input classifier Source domain prediction .

[0094] Will and Input domain discriminator To obtain the domain prediction probability and .

[0095] Loss Calculation: Calculate the classification loss: .

[0096] Computational domain adversarial loss: .

[0097] Calculate the total loss: ,in It is a hyperparameter used to balance the two terms.

[0098] Backpropagation and parameter update: renew :maximize That is, only updates are made through backpropagation. parameters This allows for better differentiation between source and target domain features. The gradient update formula is: .

[0099] renew and : minimize .fixed Update the encoder via backpropagation and classifier parameters and The key point is that, for The gradient of the term is propagated back It needs to be multiplied by (Gradient Reversal Layer, GRL), thereby driving Generate obfuscation The characteristics of . The updated formula is: .

[0100] 2.2.3. Convergence Judgment and Output: Repeat step 2 until the model is on the validation set. The classification accuracy on the domain discriminator tends to stabilize, and the domain discriminator The accuracy rate is close to 50% (indicating that the features are no longer sufficient to distinguish the source).

[0101] Save the trained domain-invariant feature extractor Discard Domain Discriminator Classifier It can serve as an initial reference for subsequent diagnostic networks.

[0102] 2.3 Third Stage: Conditional Generative Adversarial Example Augmentation This stage utilizes the trained encoder. With a small amount of target domain data, a large number of realistic new samples are generated.

[0103] 2.3.1. Feature Extraction: Extracting features from the target domain dataset. All image inputs Extract its domain-invariant eigenvector set and obtain its corresponding tag. .

[0104] 2.3.2. Conditional GAN ​​Training: Network Build: Build Generator and discriminator . The input is noise and condition vector The output is a composite image. . The input is a real image. or composite image and its corresponding condition vector .

[0105] Iterative training: Training the discriminator : Sample a batch of real target domain images and its conditions and a batch of generators Synthetic image Calculate the WGAN-GP loss. and update The parameters are set to maximize its ability to distinguish between true and false.

[0106] Training Generator :fixed Generate a batch of images Calculate the loss. .renew The parameters are minimized This makes the generated image both realistic (and fooling) It is also consistent with the real sample in the feature space.

[0107] 2.3.3. Sample Generation: Once the generated image quality meets the preset criteria (such as an FID score below a threshold), the generator is frozen. .

[0108] For each type of damage in the target domain, noise is randomly sampled. And combine the conditional vectors of this type of damage, using Generate a large number of synthetic images (e.g., 1000 images per class) in batches to form the generated dataset. .

[0109] 2.3.4. Construction of Hybrid Datasets: Constructing a Hybrid Dataset from the Original Small Sample Target Domain Dataset With the generated sample set The samples are merged to form the final hybrid training set. . The samples in the dataset inherit the labels from the generation conditions.

[0110] 2.4 Fourth Stage: Diagnostic Model Training, Interpretation, and Deployment This stage uses the augmented data to train the final diagnostic model and achieve interpretable predictions.

[0111] 2.4.1. Diagnostic Model Training: Network selection: Choose a computationally efficient, lightweight network as the diagnostic network. (e.g., MobileNetV3-Small).

[0112] Supervised training: in a mixed training set Top Perform end-to-end supervised training. The loss function is set according to the task (cross-entropy for classification, FocalLoss for detection, etc.).

[0113] Output: The optimized diagnostic model. .

[0114] 2.4.2. Online Diagnosis and Interpretation: Deployment: The trained... and They are concatenated and encapsulated into a deployable inference model.

[0115] Inference: For a new image of a target scene to be detected The forward propagation process is as follows: 1. Feature extraction: .

[0116] 2. Damage Diagnosis: Output (The specific output depends on the task).

[0117] Interpretability analysis: During the reasoning process, for the predicted category The Grad-CAM algorithm is used to calculate the feature map of the last convolutional layer of the network. The gradient is used to generate a class activation heatmap. .

[0118] Heatmap Upsampled to the original image size and compared with the original image. By overlaying, the key areas of focus of the model can be visualized.

[0119] 2.4.3. Report Generation: The system automatically generates a structured diagnostic report, which includes: The types of damage detected and their confidence levels.

[0120] Damage location information (bounding box coordinates or pixel region).

[0121] Interpretable heatmaps highlight the visual basis for the model's judgments.

[0122] Optionally, simple trend analysis or risk level alerts can be based on historical data.

[0123] The workflow of this invention, through a progressive design of "preprocessing alignment → adversarial learning alignment feature distribution → conditional generation alignment data distribution → efficient training and reliable interpretation", constructs a complete closed-loop system that starts from limited data and ultimately achieves powerful, reliable, and transparent intelligent diagnosis, effectively solving the core problem of small sample damage identification in civil engineering.

[0124] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for intelligent diagnosis of apparent damage in civil engineering structures using small samples, characterized in that, Includes the following steps: S1: Input the labeled source domain dataset and the labeled small sample target domain dataset, and perform data preprocessing; S2, using the preprocessed source domain dataset and the target domain dataset, train the encoder through adversarial training with domain adversarial loss to extract domain-invariant features that are insensitive to the source domain or the target domain; S3, using the trained encoder to extract domain-invariant features from the target domain dataset to obtain the target domain-invariant feature set; A generative adversarial network is trained using the target domain invariant feature set to obtain the target domain sample set; The target domain dataset and the target domain sample set are merged to form a hybrid training set; S4, Use the hybrid training set to train a diagnostic network for damage classification; S5, the target domain image to be diagnosed is first processed by the trained encoder to extract features, and then input into the trained diagnostic network to obtain the damage recognition result.

2. The method according to claim 1, characterized in that, S2 specifically includes: S21, using the pre-trained convolutional neural network's front convolutional layer as the encoder. E The initial weights; Building a classifier C Domain discriminant And randomly initialize the classifier. C Domain discriminant The weights; the domain discriminator It includes a binary classifier composed of fully connected layers to determine whether a feature comes from the source domain or the target domain; S22, Perform the adversarial training according to the following optimization objectives; in, To balance the hyperparameters of the two losses; To combat losses in the domain, To monitor and categorize losses; These represent the batch sample size for the source domain and the target domain, respectively. For the first One source domain sample image; For the first One target domain sample image; The probability value output by the domain discriminator; The feature vector extracted by the encoder; K Total number of damage categories; For source domain samples The true category label; For the classifier connected after the encoder C For source domain samples Category prediction probability; S23, repeat S22 until the model's classification accuracy on the validation set stabilizes, and the domain discriminator... It cannot effectively distinguish features from the source domain and the target domain.

3. The method according to claim 2, characterized in that, The forward propagation of the adversarial training includes: Will and Input the encoder respectively , to obtain features and ; Will Input the classifier Source domain prediction ; Will and Input the domain discriminator To obtain the domain prediction probability and ; in, For a batch of source domain image data; This refers to a batch of target domain image data.

4. The method according to claim 2, characterized in that, The backpropagation and parameter update of the adversarial training include: maximize Updated via backpropagation parameters The updated formula is: minimize ,fixed Update the encoder via backpropagation and classifier parameters and The updated formula is: in, The learning rate; This is the gradient operator.

5. The method according to claim 1, characterized in that, S3 specifically includes: S31, the preprocessed target domain dataset All images are input into the trained encoder Extracting the set of domain-invariant feature vectors and obtain its corresponding tag. ; S32, Construct a conditional generative adversarial network and train the discriminator. Training generator ; S33, Once the generated image quality meets the preset criteria, freeze the trained generator. ; For each type of damage in the target domain, random sampling noise is used. Combined with the conditional vector of this type of damage, using Batch generation of synthetic images constitutes the generated dataset. ; S34, and The samples are merged to form the final hybrid training set. ; The labels in the sample inheritance generation conditions.

6. The method according to claim 5, characterized in that, In S32 The conditional generative adversarial network includes: a build generator. and discriminator ; The input is noise and condition vector The output is a composite image. ; The input is a real image. or composite image and its corresponding condition vector Embedding ) represents the label embedding vector; The training discriminator Includes: sampling a batch of real target domain images and its conditions ; and a batch of generators Synthetic image ; Calculate the loss function of the discriminator Dg and update The parameters make maximize; The training generator Includes: fixed Generate a batch of images ; Calculate loss ;renew The parameters make minimize.

7. The method according to claim 1, characterized in that, In step S3, the training objective of the conditional generative adversarial network is: Where G is the generator; For target domain image samples; For the generated target domain image; The distribution of the generated image; The distribution of the real target domain image; It is a conditional vector; Points are randomly sampled along the line connecting real data points and generated data points; represents the weight coefficient of the gradient penalty term.

8. The method according to claim 1, characterized in that, S4 specifically includes: The input to the diagnostic network is the images in the mixed training set; The diagnostic network outputs the following: for classification tasks, it outputs the probability distribution of damage categories; for detection or segmentation tasks, it outputs bounding boxes or pixel-level segmentation masks. The training loss uses cross-entropy loss. : in This is the batch size of the mixed training set.

9. The method according to claim 1, characterized in that, Following step S5, the following is also included: S6, for the predicted category The Grad-CAM algorithm is used to calculate the feature map of the last convolutional layer of the network. The gradient is used to generate a class activation heatmap as follows. ; in, Z represents the activation value at spatial location (i,j) in the k-th feature map of the last convolutional layer of the diagnostic network; Z is the k-th feature map. Total number of pixels; To diagnose the last convolutional layer of the network Each feature map; The heat map Upsample to the original image size, overlay it with the original image, and then visualize the output.

10. A small-sample intelligent diagnostic system for apparent damage to civil engineering structures, characterized in that, The system includes: The data preprocessing module is used to take the labeled source domain dataset and the labeled small sample target domain dataset as input and perform data preprocessing. The domain-adaptive feature extraction module uses the preprocessed source domain dataset and the target domain dataset to train the encoder through an adversarial training method that includes domain adversarial loss, and extracts domain-invariant features that are insensitive to the source domain or the target domain. The conditional sample generation module extracts domain-invariant features from the target domain dataset using the trained encoder to obtain a target domain-invariant feature set, trains a conditional generative adversarial network using the target domain-invariant feature set to obtain a target domain sample set, and merges the target domain dataset and the target domain sample set to form a hybrid training set. The diagnostic model training module uses the hybrid training set to train a diagnostic network for damage classification. The damage intelligent diagnosis module is used to extract features from the target domain image to be diagnosed by the trained encoder, and then input the features into the trained diagnosis network to obtain the damage recognition result.

Images