Fire-damaged concrete structure damage image generation method based on generative adversarial network

Abstract

The application discloses a fire-damaged concrete structure damage image generation method based on a generative adversarial network, relates to the technical field of computer vision and deep learning, and comprises the following steps: step 1, data set making; step 2, generating a fire-damaged concrete structure damage image by using an improved cyclic generative adversarial network architecture; the improved cyclic generative adversarial network architecture is improved for a generator and a discriminator; and step 3, network training and performance evaluation.The application can generate damage images of multiple types, which are realistic and have controllable attributes, under the condition of limited real samples, so as to expand a damage image data set, solve the problem of insufficient data, and provide reliable data support for subsequent damage detection and quantitative evaluation.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, specifically to a method for generating images of post-fire concrete structure damage based on generative adversarial networks. Background Technology

[0002] After a fire, concrete structures typically exhibit damage such as cracks, spalling, exposed reinforcement, and blackening. These damages not only affect the structure's appearance but also directly impact its mechanical properties and safety. Current research largely relies on manual inspections or on-site image acquisition to build datasets, which are then combined with deep learning methods for damage identification and assessment. While image-based deep learning methods have made some progress in detecting common structural damage, in fire scenarios, the difficulty in obtaining damage images, the limited number of samples, and the complexity of damage manifestations lead to insufficient training data, poor model generalization ability, and unstable assessment results. Summary of the Invention

[0003] This invention provides a method for generating images of concrete structure damage after a fire based on generative adversarial networks. This invention can generate multiple types of realistic damage images with controllable properties under the condition of limited real samples, thereby expanding the damage image dataset, solving the problem of insufficient data, and providing reliable data support for subsequent damage detection and quantitative assessment.

[0004] To achieve the above objectives, the technical solution of the present invention is as follows:

[0005] A method for generating images of post-fire concrete structure damage based on generative adversarial networks includes the following steps:

[0006] Step 1: Dataset creation;

[0007] Step 2: Use an improved recurrent generative adversarial network architecture to generate damage images of the concrete structure after the fire; the improved recurrent generative adversarial network architecture is improved for both the generator and the discriminator.

[0008] Step 3: Network training and performance evaluation.

[0009] Preferably, step 1 includes: the dataset includes real images of concrete structure damage after a fire and images of concrete damage generated under simulated fire conditions in laboratory fire test specimens; collecting photos of concrete structure damage and undamaged concrete surfaces after a fire, all images being square-shaped, and using image augmentation technology to expand the dataset, dividing the expanded dataset into a training set and a validation set in a 7:3 ratio.

[0010] Preferably, in step 2, the generator improvement includes:

[0011] 1) Conditional Encoding and Injection Module: A conditional encoding mechanism is introduced into the input layer of the generator; it receives a one-hot encoded conditional vector specifying the target damage category and maps it to a high-dimensional feature space through a fully connected network; the conditional encoder further extracts features from the input image through convolutional layers, using reflection padding to maintain the feature map size; subsequently, through spatial copying and channel concatenation operations, the generated conditional feature map is fused with the visual feature map extracted by the image encoder.

[0012] 2) Feature enhancement module with integrated attention mechanism: It integrates dual attention mechanisms. The self-attention mechanism calculates the relationship between all pixels in the feature map, enabling the generator to model long-distance dependencies in the image. The damage-aware attention mechanism generates a spatial attention map guided by the aforementioned conditional vector, which dynamically highlights key areas in the image that are prone to damage and suppresses irrelevant background. This module is embedded in multiple residual blocks, each of which uses 3×3 convolution and maintains 256 channels.

[0013] 3) Adaptive normalization module: It adopts dynamic adaptive normalization technology and no longer uses fixed affine transformation parameters. The scaling and offset parameters used for feature normalization are dynamically generated by the output of the conditional encoder. This process runs through 6 consecutive residual blocks, each of which uses 3×3 convolution and 256 channels.

[0014] 4) Multi-scale output and feature pyramid structure: The generator is designed to have multi-scale output capability. At the end of the decoder, multiple parallel branches are responsible for upsampling and outputting images at different resolutions: low-resolution output is used to constrain the overall structure, and high-resolution output is used to enrich the fine textures. The upsampling process is achieved through transposed convolution, using convolutional layers with kernel size of 3×3 and output channels of 128 and 64 respectively, with a stride of 2, to gradually restore the spatial resolution. At the same time, a feature pyramid network is constructed between the encoder and decoder through skip connections to fuse the high-resolution detailed features at the bottom layer with the high-level semantic features. Finally, the network integrates the results at different scales through a multi-scale output head and further optimizes the output quality by combining a damage-level attention mechanism.

[0015] Preferably, in step (2), the target damage category includes bursting, exposed tendons, cracks, and blackening.

[0016] Preferably, in step (2), the discriminator improvement includes:

[0017] 1) Multi-scale input pyramid structure: The original image and images of various resolutions obtained through downsampling are input to the discriminator in parallel;

[0018] 2) Shared Backbone and Domain-Specific Classification Head Architecture: All input scales share a common feature extraction backbone network, which extracts features through multiple convolutional layers: the first layer uses a 4×4 convolutional kernel, 64 output channels, and a stride of 2; the second layer uses a 4×4 convolutional kernel, 128 output channels, and a stride of 2; the third layer uses a 4×4 convolutional kernel, 256 output channels, and a stride of 2; and the fourth layer uses a 4×4 convolutional kernel, 512 output channels, and a stride of 1. The final extracted common features are represented as a 30×30×512 feature tensor. Based on this, multiple parallel, lightweight domain-specific classification heads are connected. "Lightweight" means that the classification heads consist of a small number of convolutional or fully connected layers, with fewer parameters and computational cost, avoiding an overly large model and thus reducing computational overhead and improving training and inference efficiency; "domain-specific" means that each classification head is specifically designed for a particular damage category (such as bursting, cracking, exposed tendons, blackening) or normal images, equivalent to a small "expert module". This "shared backbone + domain-specific classification head" design can not only ensure efficient discrimination of different damage features, but also reduce redundant parameters and improve the overall accuracy and efficiency of the model.

[0019] 3) Conditional projection discrimination mechanism: Introducing conditional projection operation, the conditional vector of the damage category is mapped to a feature vector through an embedding layer, and the inner product operation is performed with the image features extracted from the shared backbone in the feature space, and the result is used as the discrimination criterion;

[0020] 4) Enhanced stability: Multiple regularization methods are integrated. First, spectral normalization is applied in almost all convolutional layers of the discriminator. Second, mini-batch discrimination is introduced. Finally, a gradient penalty term is added.

[0021] Preferably, step 3 includes: evaluating the model performance of the improved recurrent generative adversarial network architecture using the following two evaluation metrics: Fréchet Inception Distance (FID) and Inception Score (IS). The former measures the quality and realism of the generated image by calculating the distribution difference between the generated image and the real image in the feature space, as shown in Equation (1). The lower the FID value, the closer the generated image is to the real image. The latter measures the diversity and sharpness of the generated image based on the classification probability distribution of the image, as shown in Equation (2). The higher the IS value, the better the quality and diversity of the generated image. In addition, structural similarity is introduced to measure the generation effect of a single image, considering the similarity in three aspects: brightness, contrast, and structure, as shown in Equation (3). The closer the SSIM value is to 1, the more structurally similar the generated image is to the real image.

[0022] ;

[0023] ;

[0024] ;

[0025] In formulas (1)-(3), x and y are the real image and the generated image, respectively, and μ x and μ y C is the mean of the image. x and C y Let represent the covariance matrices of the real image and the generated image, respectively; Represents the trace of a matrix; Indicates from the generating distribution Medium-sampled images; p represents the Kullback-Leibler divergence between distributions p and q; p(y|x) is the conditional class distribution; p(y) = ∫p(y|x)p g ; dx(x) is the marginal class distribution; σ x and σ y It is the standard deviation of the image, σ xy C1 and C2 are small constants used to avoid zero denominators, where C is the covariance of the image.

[0026] Preferably, step 3 further includes:

[0027] A. An alternating update strategy is adopted, and each training iteration consists of two core steps: the first step is to fix the generator and update the discriminator parameters; the second step is to fix the discriminator and update the generator parameters.

[0028] B. Training relies on a composite loss function, which is a weighted sum of multiple losses, including:

[0029] (1) Adversarial loss: The least squares loss form is used to provide the basic objective function for the adversarial training of the generator and the discriminator. The calculation method is as shown in equation (4):

[0030] ;

[0031] In equation (4), D(x) is the output of the discriminator to the real image x, and G(z) is the output of the generator to the noise z;

[0032] (2) Cyclic consistency loss: Calculate the L1 distance between the reconstructed image and the original image, as shown in equation (5):

[0033] ;

[0034] In equation (5), F is another generator used to transform the image from the target domain back to the original domain;

[0035] (3) Identity loss: Calculate the L1 distance between the generator's output and input of the target domain image, as shown in equation (6):

[0036] ;

[0037] In equation (6), G(x) is the generator output of the target domain image x;

[0038] (4) Feature matching loss: Calculate the L1 distance between the feature representations of the real image and the generated image in the intermediate layer of the discriminator network, as shown in equation (7):

[0039] ;

[0040] In equation (7), Let N represent the i-th layer feature of the discriminator, and N be the number of feature matching layers.

[0041] (5) Gradient penalty: Based on the WGAN-GP framework, the gradient norm of the discriminator relative to the input image is constrained, forcing it to satisfy the Lipschitz continuity condition. The calculation method is as shown in equation (8):

[0042] ;

[0043] In equation (8), It is the interpolation between the real image and the generated image;

[0044] The weighted sum of the above losses yields the total loss function. The importance of each loss is balanced by hyperparameters, and the calculation method is as shown in equation (9):

[0045] ;

[0046] In equation (9), λ adv , λ cyc , λ id , λ fm and λ gp These are the weighting hyperparameters for each loss;

[0047] C. Use the Adam optimizer and adopt a strategy of dynamically adjusting the learning rate;

[0048] D. Apply multi-scale training, that is, start training from low-resolution images and gradually transition to high-resolution images.

[0049] The beneficial effects of the present invention on the method for generating images of post-fire concrete structure damage based on generative adversarial networks are as follows:

[0050] (1) Compared with existing traditional Generative Adversarial Networks (GANs), this invention significantly improves the feature extraction capability and image generation quality of the generative model during training by introducing a multi-scale feature extraction module, residual blocks, and Conditional Adaptive Normalization (AdaIN) technology into the generator. The multi-scale feature extraction module enables the network to capture local details and global structural features simultaneously, resulting in visually more realistic and detailed damage images. The residual block structure effectively alleviates the gradient vanishing problem in deep networks, enhances the transmission capability of deep features, and further improves the quality of generated images. The Conditional Adaptive Normalization (AdaIN) technology makes the generated images more consistent with the damage characteristics of concrete structures after a fire in terms of style and content. In terms of the discriminator, this invention adopts a multi-scale discriminator module and a Feature Pyramid Network (FPN) to enhance the model's ability to identify and distinguish damage features at different scales, thereby improving the accuracy and robustness of the discriminator. The multi-scale discriminant module enables the discriminator to evaluate the authenticity of generated images at multiple scales, while the Feature Pyramid Network (FPN) helps to fuse feature information at different levels and improve the ability to identify subtle damage features.

[0051] (2) By defining adversarial loss and cycle consistency loss, and by adopting multi-scale training and adaptive learning rate strategies, this invention further optimizes the training process of the generator and discriminator, thereby improving the convergence speed of the model and the stability of the generated images. The combination of these technical features enables this invention to maintain good generalization ability even under small sample conditions, and the generated images are not only highly realistic, but also accurately reflect the damage characteristics of concrete structures after a fire.

[0052] (3) The improved GAN network proposed in this invention not only enhances the realism and robustness of generated images of concrete structure damage after a fire, but also efficiently generates large-scale, high-quality damage image datasets even when data is scarce, providing reliable data support for subsequent structural damage identification, quantitative assessment, and structural safety research. This invention has significant engineering application value and promising prospects for improving the efficiency and accuracy of post-fire concrete structure damage assessment. Attached Figure Description

[0053] Figure 1 Examples of multi-class datasets;

[0054] Figure 2 CycleGAN network structure;

[0055] Figure 3 CycleGAN-fire structure diagram;

[0056] Figure 4 The structure of self-attention mechanisms;

[0057] Figure 5 The structure of spatial attention mechanisms;

[0058] Figure 6 Multi-scale output and feature pyramid structure;

[0059] Figure 7 CycleGAN generator structure diagram;

[0060] Figure 8 ,CycleGAN-fire generator structure diagram;

[0061] Figure 9 ,CycleGAN discriminator structure diagram;

[0062] Figure 10 ,CycleGAN-fire discriminator structure diagram;

[0063] Figure 11 Partial post-fire damage images generated by CycleGAN-fire;

[0064] Figure 12 Images generated by different generative models;

[0065] Figure 13 Segmentation results samples of images generated by different models. Detailed Implementation

[0066] The following description provides a detailed explanation of the embodiments of the present invention in a step-by-step manner. This description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0067] In the description of this invention, it should be noted that the terms "upper," "lower," "left," "right," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or a specific orientational structure and operation. Therefore, they should not be construed as limiting this invention.

[0068] Example 1: A method for generating images of post-fire concrete structure damage based on generative adversarial networks, comprising the following steps:

[0069] Step 1: Dataset creation;

[0070] Step 2: Use an improved recurrent generative adversarial network architecture to generate damage images of the concrete structure after the fire; the improved recurrent generative adversarial network architecture is improved for both the generator and the discriminator.

[0071] Step 3: Network training and performance evaluation.

[0072] Example 2: Based on Example 1, this example discloses:

[0073] Step 1 includes: the dataset includes real-world images of concrete structure damage after a fire and images of concrete damage simulated under fire conditions in laboratory fire test specimens; such as... Figure 1 As shown, a total of 2000 photos of concrete structure damage after a fire and 500 photos of undamaged concrete surfaces were collected. All images were square-sized. To avoid overfitting during model training, image augmentation techniques were used to expand the dataset, increasing the number of images to eight times that of the original dataset. The expanded dataset was then divided into training and validation sets in a 7:3 ratio. During training, to enhance data diversity and improve model generalization ability, the input images were resized to generate image samples with different aspect ratios: 0.5, 0.75, 1.0, 1.25, 1.5, and 1.75.

[0074] Example 3: Based on Examples 1 and 2, this example discloses:

[0075] Step 2 includes: using an improved Cycle Generative Adversarial Network (CycleGAN) architecture to generate damage images of the concrete structure after a fire, the original CycleGAN network structure as follows: Figure 2 The improved CycleGAN network structure is as follows: Figure 3 Compared to the traditional CycleGAN, the improved CycleGAN-fire significantly improves the network structure and training strategy of the generator and discriminator, aiming to enhance the realism, diversity, and detail fidelity of the generated images.

[0076] The following is a detailed introduction to the improvements based on the CycleGAN network:

[0077] (1) Generator improvement

[0078] This invention fundamentally innovates the generator in generative adversarial networks, aiming to solve the technical problem that existing technologies cannot generate high-fidelity, structurally sound damaged images according to specified categories. The original generator structure is as follows: Figure 7 As shown, the improved generator structure is as follows: Figure 8 As shown, its core innovations are reflected in the following four aspects:

[0079] 1) Conditional Encoding and Injection Module: To enable the generator to generate specific damage types on demand, this invention introduces a conditional encoding mechanism at the generator's input layer. This module receives a one-hot encoded conditional vector specifying the target damage category (such as bursting, exposed tendons, cracks, blackening), and maps it to a high-dimensional feature space through a fully connected network. The conditional encoder further extracts features from the input image through convolutional layers (including a convolutional layer with a kernel size of 7×7 and 64 output channels, and a downsampling convolution with a kernel size of 3×3, 128 output channels, and a stride of 2), during which reflection padding is used to maintain the feature map size. Subsequently, through spatial copying and channel concatenation operations, the generated conditional feature map is fused with the visual feature map extracted by the image encoder, thereby accurately injecting abstract category semantic information into every spatial location of the image generation process, realizing the controllability of the generation process.

[0080] 2) Feature enhancement module integrating attention mechanism: To improve the global structural rationality and local detail realism of the generated image, this invention integrates a dual attention mechanism, such as... Figure 4 The self-attention mechanism calculates the relationships between all pixels in the feature map, enabling the generator to model long-range dependencies in the image and ensuring that generated extensible damage (such as long cracks) remains coherent and reasonable globally. Figure 5 The damage-aware attention mechanism, guided by the aforementioned conditional vector, generates a spatial attention map that dynamically highlights key areas in the image prone to damage while suppressing irrelevant background. This guides the model to concentrate computational resources on these areas, significantly enhancing the texture details and realism of the damaged regions. This module is embedded in multiple residual blocks, each using a 3×3 convolution and maintaining 256 channels to ensure rich feature representation.

[0081] 3) Adaptive Normalization Module: To achieve fine-grained modulation of the generated style by conditional information, this invention employs a dynamic adaptive normalization technique (AdaIN). This module no longer uses fixed affine transformation parameters; instead, the scaling and offset parameters used for feature normalization are dynamically generated from the output of the conditional encoder. This allows different damage types to dynamically modulate the mean and variance of the generator's intermediate layer features, thereby achieving precise control over the generated image style (i.e., specific damage morphology, texture, and color), ensuring consistent style for similar damage types and distinct features for dissimilar damage types. This process is carried out across six consecutive residual blocks, each employing a 3×3 convolution and a 256-channel configuration to ensure that features are fully modulated across multiple levels.

[0082] 4) Multi-scale Output and Feature Pyramid Structure: To balance the macroscopic structural correctness and microscopic detail clarity of the generated images, and to improve training stability, the generator is designed with multi-scale output capability. At the decoder end, multiple parallel branches are responsible for upsampling and outputting images at different resolutions: low-resolution output (e.g., 64×64×3) is used to constrain the overall structure, while high-resolution output (e.g., 256×256×3) is used to enrich fine textures. The upsampling process is implemented through transposed convolution, using convolutional layers with a kernel size of 3×3 and output channels of 128 and 64 respectively, with a stride of 2, to gradually restore spatial resolution. Simultaneously, a Feature Pyramid Network is constructed between the encoder and decoder through skip connections, fusing the high-resolution detail features at the bottom layer with the high-level semantic features, effectively mitigating the detail loss problem during upsampling and ensuring that the generated damaged texture edges are clear and the features are well-defined. Finally, the network integrates the results at different scales through a multi-scale output head and further optimizes the output quality by combining a damage-level attention mechanism.

[0083] (2) Discriminator improvement

[0084] This invention introduces a collaborative innovation in the discriminator network structure, aiming to address key issues in multi-class image generation, such as the discriminator's limited capabilities, unstable training, and inability to provide fine-grained gradient feedback. The improved discriminator not only effectively distinguishes between real and fake images but also accurately determines the damage category of an image, thus providing more instructive training signals for the generator. The original discriminator structure is as follows: Figure 6 As shown, the improved discriminator structure is as follows: Figure 7 As shown, its core innovations are reflected in the following four aspects:

[0085] 1) Multi-scale input pyramid structure. To enhance the discriminator's ability to perceive damage features of different sizes, this invention designs a multi-scale input processing flow. The original image (256×256×3) and images of various resolutions obtained through downsampling (such as 128×128×3 at half scale) are input to the discriminator in parallel. This structure ensures that the discriminator can capture subtle texture features (such as fine cracks and surface blackening) from high-resolution images, and also perceive macroscopic structural patterns (such as large-area bursts and rebar distribution) from low-resolution images, thereby achieving cross-scale, all-round discrimination of damage features and greatly enhancing the accuracy of quality assessment of the generated images.

[0086] 2) Shared Backbone and Domain-Specific Classification Head Architecture: To efficiently achieve multi-domain (multi-damage category) discrimination and avoid parameter redundancy, this invention adopts a parameter-sharing design concept. All input scales share a common feature extraction backbone network, which extracts features through multiple convolutional layers: the first layer uses a 4×4 convolutional kernel, 64 output channels, and a stride of 2. The second layer uses a 4×4 convolutional kernel, 128 output channels, and a stride of 2. The third layer uses a 4×4 convolutional kernel, 256 output channels, and a stride of 2. The fourth layer uses a 4×4 convolutional kernel, 512 output channels, and a stride of 1. The final extracted common features are represented as a 30×30×512 feature tensor. This is followed by multiple parallel, lightweight domain-specific classification heads, each responsible for discriminating a specific damage category (bursting, cracking, exposed tendons, blackening) or normal images. This design ensures efficient discrimination of different damage features while significantly reducing the number of model parameters and computational complexity, thus improving training efficiency.

[0087] 3) Conditional Projection Discrimination Mechanism. To achieve true conditional discrimination and guide the generator to produce images conforming to a specified category, this invention introduces a conditional projection operation. This mechanism maps the conditional vector of the damage category (such as one-hot encoding) to a feature vector through an embedding layer, and performs an inner product operation with the image features extracted from the shared backbone in the feature space, using the result as the discrimination criterion. This process forces the discriminator not only to judge the authenticity of the image but also to evaluate the consistency between the image content and the given conditions, thereby providing the generator with an accurate gradient signal that "the generated image should simultaneously satisfy both authenticity and condition," fundamentally solving the problem of mismatch between the generated image and the target category.

[0088] 4) Stability Enhancement Techniques. To ensure the stability of the multi-domain adversarial training process, this invention integrates several advanced regularization techniques. First, spectral normalization is applied to almost all convolutional layers of the discriminator. By constraining the spectral norm of the weight matrix of each layer, the Lipschitz constant of the discriminator is effectively controlled, preventing gradient explosion and thus stabilizing the training dynamics. Second, a minibatch discrimination technique is introduced, enabling the discriminator to perceive the diversity among generated samples within a batch and feed this back to the generator through adversarial loss, effectively alleviating the problems of monotonous generation patterns and mode collapse. Finally, a gradient penalty term is added to force the discriminator to satisfy the Lipschitz constraint on local regions between real and generated data, further enhancing training stability and convergence.

[0089] Example 4: Based on Examples 1-3, this example discloses:

[0090] Step 3 includes:

[0091] 3.1 Experimental Environment: The experimental environment for damage segmentation based on CycleGAN-Fire is shown in Table 1.

[0092] Table 1 Experimental Environment Configuration Table

[0093] Name Information Graphics card NVIDIA GeForce RTX 3090 Video memory 24GB Processor 2.3GHz, 36-core Intel-E5 CPU Hard disk Solid state disk Operating system Windows 10 Deep learning framework Pytorch 1.12.1 CUDA CUDA 12.2 Programming language Python 3.8.10

[0094] 3.2 Evaluation Metrics and Training Strategies: To quantify the model's performance, this invention employs two evaluation metrics: Fréchet Inception Distance (FID) and Inception Score (IS). The former measures the quality and realism of the generated image, evaluated by calculating the distribution difference between the generated image and the real image in the feature space, as shown in Equation (1). The lower the FID value, the closer the generated image is to the real image. The latter assesses the diversity and sharpness of the generated image, based on the image's classification probability distribution, as shown in Equation (2). The higher the IS value, the better the quality and diversity of the generated image. In addition, Structural Similarity (SSIM) is introduced to measure the generation effect of a single image. It considers the similarity in brightness, contrast, and structure, as shown in Equation (3). The closer the SSIM value is to 1, the more structurally similar the generated image is to the real image. SSIM is more sensitive to the visual effect of the image and can better reflect the visual quality of the image.

[0095] ;

[0096] ;

[0097] ;

[0098] In equations (1)-(3), x and y are the real image and the generated image, respectively, and μ x and μ y It is the mean of the image. C x and C y Let represent the covariance matrices of the real image and the generated image, respectively. Represents the trace of a matrix. Indicates from the generating distribution Images sampled from the middle. This represents the Kullback-Leibler (KL) divergence between distributions p and q. p(y|x) is a conditional class distribution. p(y) = ∫p(y|x)p g; dx(x) is the marginal class distribution. σ x and σ y It is the standard deviation of the image, σ xy C1 and C2 are small constants used to avoid zero denominators, where C is the covariance of the image.

[0099] The training process of this invention employs a multi-objective, phased optimization strategy with strong regularization. This strategy aims to collaboratively train the improved generator and discriminator, ensuring the stability and convergence of the training process, ultimately achieving high-quality, multi-class damage image generation. The core of the entire training strategy lies in constructing a composite loss function and solving it through alternating optimization. This invention uses an alternating update strategy. Each training iteration consists of two core steps: the first step fixes the generator and updates the discriminator parameters, aiming to accurately distinguish between real and generated images and provide accurate domain classifications for different damage types; the second step fixes the discriminator and updates the generator parameters, aiming to generate conditional images that can both deceive the discriminator and satisfy cycle consistency constraints.

[0100] Secondly, training relies on a carefully designed composite loss function. This function is a weighted sum of multiple losses, mainly including:

[0101] Adversarial loss: The least squares loss (LSGAN) is adopted to provide a basic objective function for the adversarial training of the generator and the discriminator. Its goal is to make the generated image output in the corresponding domain of the discriminator as close to the real image as possible. The calculation method is as shown in Equation (4):

[0102] ;

[0103] In equation (4), D(x) is the output of the discriminator to the real image x, and G(z) is the output of the generator to the noise z.

[0104] Cycle consistency loss: Calculate the L1 distance between the reconstructed image and the original image. This is the core of the CycleGAN architecture, which ensures the high preservation of image content information during domain transformation and prevents mode collapse. The calculation method is as shown in equation (5):

[0105] ;

[0106] In equation (5), F is another generator used to transform the image from the target domain back to the original domain.

[0107] Identity loss: Calculate the L1 distance between the generator's output and input of the target domain image. This loss, as a regularization term, helps the generator learn to maintain the color and texture distribution of the input image, making the generated result more natural. The calculation method is as shown in equation (6):

[0108] ;

[0109] In equation (6), G(x) is the generator output of the target domain image x.

[0110] Feature matching loss: Calculates the L1 distance between the feature representations of the real image and the generated image in the intermediate layer of the discriminator network. This loss does not directly participate in adversarial processing, but rather acts as a soft constraint to stabilize the generator's training process and improve the perceptual quality of the generated image. The calculation method is as shown in equation (7):

[0111] ;

[0112] In equation (7), Let N represent the i-th layer feature of the discriminator, and N be the number of feature matching layers.

[0113] Gradient penalty: Based on the WGAN-GP framework, the gradient norm of the discriminator relative to the input image is constrained, forcing it to satisfy the Lipschitz continuity condition. This is a key technology to ensure efficient calculation of Wasserstein distance and stable training. The calculation method is as shown in Equation (8).

[0114] ;

[0115] In equation (8), It is the interpolation between the real image and the generated image.

[0116] The final total loss function is a weighted sum of the losses mentioned above, with the importance of each loss balanced by hyperparameters, calculated as shown in Equation 9. In the optimization implementation, the Adam optimizer is used, employing a strategy of dynamically adjusting the learning rate. Furthermore, this invention utilizes multi-scale training techniques, starting training from low-resolution images and gradually transitioning to high-resolution images. This effectively accelerates the model convergence process and further improves the quality of the final generated images. Through the above systematic training strategies, this invention successfully solves common problems in multi-class image generation, such as mode collapse and training instability.

[0117] ;

[0118] In equation (9), λ adv , λ cyc , λ id , λ fm and λ gp It is the weight hyperparameter for each loss.

[0119] 3.3 Training Results

[0120] Figure 8This study showcases some post-fire damage images generated by the proposed model. The model not only generates high-quality and diverse damage but also accurately recovers detailed image information, such as concrete surface texture and defects, ensuring close alignment between the background areas of the generated images and their corresponding locations in the original images. This finding demonstrates the excellent detail capture and reconstruction capabilities of the developed model, enabling it to effectively generate damage while preserving the structural and textural information of the original images. This consistency allows the model to be directly applied to damage detection and segmentation tasks, simplifying the workflow and significantly improving the efficiency of data preparation.

[0121] 3.4 Comparative Experiment

[0122] To evaluate the performance of the proposed algorithm in generating images of concrete damage after a fire, it was comprehensively compared and analyzed with two classic image generation models (CycleGAN and DCGAN). Table 2 shows the objective evaluation metrics for each model after generating 500 images randomly. The results show that the proposed algorithm performs best in the FID (Firmware ID) metric, with a value significantly lower than the comparison models, indicating that the generated images are closer to the real images in terms of feature distribution and have higher visual realism. In terms of diversity, the proposed algorithm also achieves a significantly higher IS (Integrity Score) than DCGAN and CycleGAN, demonstrating its significant advantage in generating diverse damage structures. Comprehensive analysis shows that the proposed model can not only synthesize realistic damage details with precision but also ensure the richness of the generated samples in terms of structure and morphology. The objective evaluation results verify that the proposed algorithm outperforms existing models in both the realism and diversity of the generated images, especially in maintaining background consistency and generating structural diversity. This conclusion is consistent with... Figure 9 The high degree of consistency in the subjective visual evaluations presented further confirms the effectiveness and advancement of this method.

[0123] 3.5 Comparison of Segmentation Performance

[0124] To validate the quality of the damaged images generated by the proposed model, the YOLOv10-seg model was trained and evaluated on four different datasets: the real dataset, the CycleGAN dataset, the DCGAN dataset, and the CycleGAN-fire dataset. Each dataset contains 4000 images, which are divided into training and test sets in a 3:1 ratio. The real dataset was expanded to 4000 images by scaling the original damaged images to 256×256 and then applying conventional augmentation methods (such as random rotation, brightness transformation, and noise addition). The CycleGAN, CycleGAN-fire, and DCGAN datasets consist of 4000 damaged images (256×256) generated by their respective models. As shown in Table 3, the proposed algorithm outperforms the real dataset and the CycleGAN and DCGAN-augmented datasets on all metrics, indicating that the high-quality damaged images generated by the algorithm can be used for sample augmentation.

[0125] Table 3 Performance evaluation results of different generative models in the segmentation task

[0126] Data source Precision Recall mAP(M)@0.5 IoU Real data 0.891 0.882 0.884 0.831 CycleGAN 0.837 0.632 0.776 0.723 DCGAN 0.846 0.631 0.763 0.733 CycleGAN-fire 0.920 0.902 0.912 0.853

[0127] Note: Precision: Precision rate, indicating how many samples predicted as positive by the model are actually positive.

[0128] Recall (M): Recall rate, representing the percentage of all true positive samples that are correctly predicted as positive by the model.

[0129] IoU (Intersection over Union) is used to measure the degree of overlap between two bounding boxes.

[0130] mAP(M)@0.5: Mean Average Precision when the IoU (Intersection over Union) threshold is 0.5.

[0131] Figure 13 The images showcase the segmentation and detection results of the YOLOv10-seg model trained on four damage datasets. These images demonstrate that the damage image dataset generated by the proposed model can be effectively segmented by the model. This finding further confirms that the generated damage images can be directly used for sample augmentation and model training.

[0132] A significant advantage of this invention is its ability to generate corresponding concrete damage images based on conditional labels for different damage types without requiring pixel-level alignment or annotation. This greatly reduces the cost and complexity of data preparation, making the invention more feasible and flexible in practical applications. Through these improvements, this invention can efficiently generate large-scale, high-quality datasets of post-fire concrete structure damage images, providing strong technical support for subsequent damage identification, quantitative assessment, and structural safety research.

[0133] In summary, this invention proposes a method for generating post-fire concrete structure damage images based on an improved generative adversarial network (GAN). This method can automatically generate multiple types of damage images, such as concrete cracks, bursts, exposed rebar, and blackening, that are highly consistent with the actual post-fire damage morphology. By introducing a multi-scale feature extraction module and residual blocks into the generator, combined with conditional adaptive normalization (AdaIN) technology, and employing a multi-scale discrimination module and feature pyramid network (FPN) in the discriminator, significant improvements in the realism and detail accuracy of the damage images are achieved.

[0134] Furthermore, this invention optimizes the training process of the generator and discriminator by defining adversarial loss and cycle consistency loss, and employing multi-scale training and adaptive learning rate strategies, ensuring high quality and diversity of generated images. The combination of these key technologies overcomes the limitations of traditional methods relying on experiments or limited real-world images, which suffer from high data acquisition costs and insufficient diversity in damage morphology. This enables the efficient generation of large-scale, controllable datasets of post-fire concrete damage images.

Claims

1. A method for generating images of post-fire concrete structure damage based on generative adversarial networks, characterized by: Includes the following steps: Step 1: Dataset creation; Step 2: Use an improved recurrent generative adversarial network architecture to generate damage images of the concrete structure after the fire; the improved recurrent generative adversarial network architecture is improved for both the generator and the discriminator. Step 3: Network training and performance evaluation; In step 2, the generator improvement includes: 1) Conditional Encoding and Injection Module: A conditional encoding mechanism is introduced into the input layer of the generator; it receives a one-hot encoded conditional vector specifying the target damage category and maps it to a high-dimensional feature space through a fully connected network; the conditional encoder further extracts features from the input image through convolutional layers, using reflection padding to maintain the feature map size; subsequently, through spatial copying and channel concatenation operations, the generated conditional feature map is fused with the visual feature map extracted by the image encoder. 2) Feature enhancement module with integrated attention mechanism: It integrates dual attention mechanisms. The self-attention mechanism calculates the relationship between all pixels in the feature map, enabling the generator to model long-distance dependencies in the image. The damage-aware attention mechanism generates a spatial attention map guided by the aforementioned conditional vector, which dynamically highlights key areas in the image that are prone to damage and suppresses irrelevant background. This module is embedded in multiple residual blocks, each of which uses 3×3 convolution and maintains 256 channels. 3) Adaptive normalization module: It adopts dynamic adaptive normalization technology and no longer uses fixed affine transformation parameters. The scaling and offset parameters used for feature normalization are dynamically generated by the output of the conditional encoder. This process runs through 6 consecutive residual blocks, each of which uses 3×3 convolution and 256 channels. 4) Multi-scale output and feature pyramid structure: The generator is designed to have multi-scale output capability. At the end of the decoder, multiple parallel branches are responsible for upsampling and outputting images at different resolutions: low-resolution output is used to constrain the overall structure, and high-resolution output is used to enrich the fine textures. The upsampling process is achieved through transposed convolution, using convolutional layers with kernel size of 3×3 and output channels of 128 and 64 respectively, with a stride of 2, to gradually restore the spatial resolution. At the same time, a feature pyramid network is constructed between the encoder and decoder through skip connections to fuse the high-resolution detail features at the bottom layer with the high-level semantic features. Finally, the network integrates the results at different scales through a multi-scale output head and further optimizes the output quality by combining a damage-level attention mechanism. In step 2, the discriminator improvement includes: 1) Multi-scale input pyramid structure: The original image and images of various resolutions obtained through downsampling are input to the discriminator in parallel; 2) Shared backbone and domain-specific classification head architecture: All input scales share a common feature extraction backbone network, which extracts features through multiple convolutional layers: the first layer uses a 4×4 convolutional kernel, 64 output channels, and a stride of 2; the second layer uses a 4×4 convolutional kernel, 128 output channels, and a stride of 2; the third layer uses a 4×4 convolutional kernel, 256 output channels, and a stride of 2; the fourth layer uses a 4×4 convolutional kernel, 512 output channels, and a stride of 1; the final extracted common features are represented as a 30×30×512 feature tensor; based on this, multiple parallel, lightweight domain-specific classification heads are connected; 3) Conditional projection discrimination mechanism: Introducing conditional projection operation, the conditional vector of the damage category is mapped to a feature vector through an embedding layer, and the inner product operation is performed with the image features extracted from the shared backbone in the feature space, and the result is used as the discrimination criterion; 4) Enhanced stability: Multiple regularization methods are integrated. First, spectral normalization is applied in almost all convolutional layers of the discriminator. Second, mini-batch discrimination is introduced. Finally, a gradient penalty term is added.

2. The method for generating images of post-fire concrete structure damage based on generative adversarial networks as described in claim 1, characterized in that, Step 1 includes: the dataset includes real images of concrete structure damage after a fire and images of concrete damage generated under simulated fire conditions in laboratory fire test specimens; photos of concrete structure damage and undamaged concrete surfaces after a fire are collected, all images are taken in a square format, and the dataset is expanded using image augmentation technology, and the expanded dataset is divided into training set and validation set in a 7:3 ratio.

3. The method for generating post-fire concrete structure damage images based on generative adversarial networks as described in claim 2, characterized in that, In step 2, the target damage categories include bursting, exposed reinforcement, cracks, and blackening.

4. The method for generating post-fire concrete structure damage images based on generative adversarial networks as described in claim 3, characterized in that, Step 3 includes: evaluating the model performance of the improved recurrent generative adversarial network architecture using the following two evaluation metrics: Fréchet Inception Distance (FID) and Inception Score (IS). The former measures the quality and realism of the generated image by calculating the distribution difference between the generated image and the real image in the feature space, as shown in Equation (1). The lower the FID value, the closer the generated image is to the real image. The latter measures the diversity and sharpness of the generated image based on the classification probability distribution of the image, as shown in Equation (2). The higher the IS value, the better the quality and diversity of the generated image. In addition, structural similarity is introduced to measure the generation effect of a single image, considering the similarity in three aspects: brightness, contrast, and structure, as shown in Equation (3). The closer the SSIM value is to 1, the more structurally similar the generated image is to the real image. （1）； （2）； （3）； In formulas (1)-(3), x and y are the real image and the generated image, respectively, and μ x and μ y C is the mean of the image. x and C y Let represent the covariance matrices of the real image and the generated image, respectively; The trace of a matrix; x ~ p g Indicates that from the generating distribution p g Medium-sampled images; p represents the Kullback-Leibler divergence between distributions p and q; p(y|x) is the conditional class distribution; p(y) = ∫p(y|x)p g (x)dx is the marginal class distribution; σ x and σ y It is the standard deviation of the image, σ xy C1 and C2 are small constants used to avoid zero denominators, where C is the covariance of the image.

5. The method for generating post-fire concrete structure damage images based on generative adversarial networks as described in claim 4, characterized in that, Step 3 further includes: A. An alternating update strategy is adopted, and each training iteration consists of two core steps: the first step is to fix the generator and update the discriminator parameters; the second step is to fix the discriminator and update the generator parameters. B. Training relies on a composite loss function, which is a weighted sum of multiple losses, including: (1) Adversarial loss: The least squares loss form is used to provide the basic objective function for the adversarial training of the generator and the discriminator. The calculation method is as shown in equation (4): （4）； In equation (4), D(x) is the output of the discriminator to the real image x, and G(z) is the output of the generator to the noise z; (2) Cyclic consistency loss: Calculate the L1 distance between the reconstructed image and the original image, as shown in equation (5): （5）； In equation (5), F is another generator used to transform the image from the target domain back to the original domain; (3) Identity loss: Calculate the L1 distance between the generator's output and input of the target domain image, as shown in equation (6): （6）； In equation (6), G(x) is the generator output of the target domain image x; (4) Feature matching loss: Calculate the L1 distance between the feature representations of the real image and the generated image in the intermediate layer of the discriminator network, as shown in equation (7): （7）； In equation (7), Let N represent the i-th layer feature of the discriminator, and N be the number of feature matching layers. (5) Gradient penalty: Based on the WGAN-GP framework, the gradient norm of the discriminator relative to the input image is constrained, forcing it to satisfy the Lipschitz continuity condition. The calculation method is as shown in equation (8): （8）； In equation (8), It is the interpolation between the real image and the generated image; The weighted sum of the above losses yields the total loss function. The importance of each loss is balanced by hyperparameters, and the calculation method is as shown in equation (9): （9）； Where, λ adv , λ cyc , λ id , λ fm and λ gp These are the weighting hyperparameters for each loss; C. Use the Adam optimizer and adopt a strategy of dynamically adjusting the learning rate; D. Apply multi-scale training, that is, start training from low-resolution images and gradually transition to high-resolution images.

Images