RC structure earthquake damage assessment method based on damage segmentation map
The seismic damage assessment method for RC structures driven by damage segmentation maps, combined with lightweight SAM and LLaVA modules, solves the problem of the disconnect between damage identification and assessment, and achieves efficient and interpretable damage assessment, which is suitable for mobile terminals and edge computing devices.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-22
- Publication Date
- 2026-03-10
AI Technical Summary
In the post-earthquake damage assessment of concrete structures, existing technologies suffer from a disconnect between damage identification and assessment methods, and the model output lacks interpretability, making it difficult to achieve efficient and transparent damage assessment.
A damage segmentation map-driven seismic damage assessment method for RC structures is adopted. The method uses a lightweight SAM damage segmentation module for semantic segmentation, and combines multi-scale spatial attention weight coefficients and a lightweight LLaVA damage understanding module to achieve a nonlinear mapping between damage type and attention intensity. A two-stage optimization mechanism is used to reduce the number of model parameters and improve assessment accuracy.
It realizes the whole process evaluation of seismic damage in RC structures, including perception, identification, segmentation, assessment and description, which improves the efficiency of post-earthquake assessment and the interpretability of results. It is suitable for real-time deployment on mobile terminals and edge computing devices.
Smart Images

Figure CN121640465A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of seismic damage identification technology for concrete structures, and in particular to a method for seismic damage assessment of RC structures driven by damage segmentation maps. Background Technology
[0002] Post-earthquake damage assessment of concrete structures is a crucial step in determining subsequent reinforcement strategies. A well-reasoned assessment can not only prevent structural collapse due to insufficient reinforcement but also reduce the waste that may result from over-reinforcement. In recent years, computer vision and deep learning technologies have been widely applied in post-earthquake assessment. However, existing technologies typically focus on identifying and locating localized structural damage, while research using visual damage identification results to guide the assessment of structural damage severity is relatively limited. This restricts the application of research findings in structural damage identification to practical post-earthquake assessment work.
[0003] Currently, deep learning-based structural damage identification and segmentation methods have made significant progress, effectively locating and segmenting damaged areas, and replacing the human eye in preliminary visual detection of damage. However, the key evaluation indicator of the degree of structural damage still relies on human experience combined with damage images for judgment. Therefore, its contribution to improving the efficiency of post-earthquake emergency assessment is relatively limited. On the other hand, although existing research has developed end-to-end models aimed at directly predicting damage indices based on damage images, these models generally have two major limitations: first, they fail to fully explore and utilize the rich visual damage information contained in the images; second, their prediction results lack transparency and interpretability, making it difficult to explain the basis of the assessment conclusions. Summary of the Invention
[0004] The present invention aims to at least partially solve one of the technical problems in the related art.
[0005] Therefore, the first objective of this invention is to propose a damage segmentation map-driven method for seismic damage assessment of RC structures.
[0006] The second objective of this invention is to provide a damage segmentation map-driven seismic damage assessment device for RC structures.
[0007] To achieve the above objectives, a first aspect of the present invention proposes a method for seismic damage assessment of RC structures driven by damage segmentation maps, comprising: S1, the original image of the damaged component is semantically segmented by the lightweight SAM damage segmentation module, and the pixel-level segmentation map containing different types of structural damage is output. S2, construct multi-scale spatial attention weight coefficients based on the category statistical features of the segmentation map. The category statistical features include the pixel proportion and spatial distribution entropy value of various types of damage. Establish a nonlinear mapping relationship between damage type and attention intensity through a lightweight weight generation sub-network. S3, input the original image of the damaged component and the segmentation map into the lightweight LLaVA damage understanding module, use the spatial attention weight coefficient to guide the visual encoder to enhance the features of the key damage area, and simultaneously output the damage index and the natural language description corresponding to the damage features. S4. The SAM module and LLaVA module are jointly trained through a two-stage optimization mechanism, which includes a short-term freeze-and-fine-tuning stage and a long-term joint optimization stage, in order to achieve a balance between reducing the number of model parameters and improving evaluation accuracy.
[0008] In one embodiment of the present invention, S2 includes: S21 removes redundant multi-head attention layers in the lightweight SAM module through a structured pruning strategy, while retaining key feature extraction paths to reduce computational complexity. S22 introduces a dynamic parameter sharing mechanism in the encoder, enabling feature maps of different scales to share some learnable parameters to improve the model's generalization ability.
[0009] In one embodiment of the present invention, S3 includes: S31, through hierarchical knowledge distillation, transfers the complex visual-language alignment capability of the fine-tuned model to a simplified architecture, while retaining key semantic association features; S32 uses low-rank tensor decomposition technology to compress the high-dimensional weight tensor of the decoder, decomposing the weight matrix into a low-rank core tensor and a factor matrix to reduce storage requirements.
[0010] In one embodiment of the present invention, S4 includes: S41, fix the backbone network parameters during the short-term freeze-fine-tuning phase, and train only the lightweight weight generation subnetwork to achieve fast convergence; S42, during the long-term joint optimization phase, unfreezes the backbone network parameters and simultaneously trains the sub-networks and backbone network through an alternating optimization strategy to improve overall performance.
[0011] In one embodiment of the present invention, it further includes: S5, perform multimodal training and annotation on the original damaged component image, including: S51, pixel-level annotation of different damage categories on damaged components in the image, covering typical damage types such as cracking, spalling, and exposed reinforcement; S52, synchronously generate a text description containing the component damage index and typical damage characteristics, the text description specifically explaining key parameters such as the concrete spalling range, longitudinal reinforcement buckling angle and distribution density.
[0012] To achieve the above objectives, a second aspect of the present invention provides a seismic damage assessment device for RC structures based on damage segmentation maps, comprising: The lightweight SAM damage segmentation module is used to perform semantic segmentation on the original images of damaged components and output pixel-level segmentation maps containing different types of structural damage. A multi-scale spatial attention weight construction module is used to construct multi-scale spatial attention weight coefficients based on the category statistical features of the segmentation map. The category statistical features include the pixel proportion and spatial distribution entropy value of various types of damage. A non-linear mapping relationship between damage type and attention intensity is established through a lightweight weight generation sub-network. The lightweight LLaVA damage understanding module is used to input the original image of the damaged component and the segmentation map into the lightweight LLaVA damage understanding module, and use the spatial attention weight coefficient to guide the visual encoder to perform feature enhancement on the key damage area, and simultaneously output the damage index and the natural language description corresponding to the damage features. A two-stage optimization joint training module is used to jointly train the SAM module and the LLaVA module through a two-stage optimization mechanism. The two-stage optimization mechanism includes a short-term freeze-fine-tuning stage and a long-term joint optimization stage to achieve a balance between reducing the number of model parameters and improving evaluation accuracy.
[0013] The beneficial effects of this invention are: This invention overcomes the following two existing technical challenges: (1) damage identification and assessment methods are fragmented, and existing damage identification and assessment methods lack synergy; (2) the output results of existing seismic damage assessment models for RC structures lack interpretability. This invention organically integrates semantic segmentation technology and multimodal large model technology. In addition, it also optimizes the model architecture in lightweight form based on actual application scenarios, significantly reducing the number of model parameters and facilitating the deployment and application of the model on local devices. In summary, this invention can realize the entire process evaluation of seismic damage in RC structures, including perception, identification, segmentation, assessment, and description, thereby improving the efficiency of post-earthquake assessment work.
[0014] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0015] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a damage segmentation map-driven seismic damage assessment method for RC structures according to an embodiment of the present invention; Figure 2 This is an architecture diagram of a damage segmentation map-driven seismic damage assessment method for RC structures according to an embodiment of the present invention; Figure 3 This is a structural diagram of a damage segmentation map-driven seismic damage assessment device for RC structures according to an embodiment of the present invention. Detailed Implementation
[0016] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0018] The following describes, with reference to the accompanying drawings, a method for seismic damage assessment of RC structures driven by damage segmentation maps, according to an embodiment of the present invention.
[0019] Example 1 Figure 1 This is a flowchart of a damage segmentation map-driven seismic damage assessment method for RC structures according to an embodiment of the present invention, such as... Figure 1 As shown, it includes: S1 performs semantic segmentation on the original seismically damaged component image through the lightweight SAM damage segmentation module, and outputs a pixel-level segmentation map containing different types of structural damage.
[0020] Specifically, a lightweight SAM damage segmentation module is constructed. Its core objective is to achieve efficient and accurate semantic segmentation of original seismically damaged component images, outputting pixel-level segmentation maps containing different types of structural damage (such as cracking, spalling, exposed reinforcement, etc.). Based on the original SAM model, this module integrates lightweight techniques such as knowledge distillation-driven structured pruning, dynamic parameter sharing, and low-rank tensor decomposition to significantly reduce the model's computational complexity and storage requirements while maintaining high segmentation accuracy.
[0021] In some implementations, a hierarchical knowledge distillation strategy is first employed to transfer the semantic segmentation capability of the backbone network in the original SAM model to a lightweight convolutional-attention hybrid architecture. This architecture retains key feature extraction capabilities while removing redundant multi-head attention layers through structured pruning techniques, retaining only the attention heads that contribute significantly to damage feature perception, thereby reducing redundant computation. Secondly, a dynamic parameter sharing mechanism is introduced in the encoder, sharing some learnable parameters among feature maps at different scales to improve model efficiency and enhance the expressive power of cross-scale damage features. Finally, a block-based low-rank tensor decomposition method is used in the decoder to decompose the high-dimensional weight tensor into a low-rank core tensor and a factor matrix, thereby significantly compressing the number of model parameters without significantly sacrificing performance.
[0022] Specifically, the number of backbone network channels in the lightweight SAM module is reduced from the original 768 to 256, the number of attention heads is reduced from 8 to 4, the rank of tensor decomposition in the decoder is set to 16, the overall model parameter size is controlled within 10MB, and the inference speed reaches more than 30FPS, meeting the requirements for real-time performance and deployment in actual engineering.
[0023] In practical applications, this module can be deployed on mobile terminals or edge computing devices to quickly acquire pixel-level damage maps of earthquake-damaged components on-site, providing structured input for subsequent damage assessment and description. Its output segmentation map not only includes the precise location of damaged areas but also distinguishes damage types through category labels, providing crucial visual-semantic alignment for the LLaVA module.
[0024] This lightweight SAM module maintains a high mIoU (mean crossover ratio) index (≥0.85) while significantly improving the deployment flexibility and operational efficiency of the model, laying a solid foundation for realizing multimodal assessment of seismic damage to RC structures.
[0025] S2, construct multi-scale spatial attention weight coefficients based on the category statistical features of the segmentation map. The category statistical features include the pixel proportion and spatial distribution entropy value of various types of damage. Establish a nonlinear mapping relationship between damage type and attention intensity through a lightweight weight generation sub-network.
[0026] Specifically, the constructed lightweight weight generation subnetwork aims to generate multi-scale spatial attention weight coefficients for the LLaVA damage understanding module based on the categorical statistical features of the damage segmentation map. This subnetwork consists of 1×1 convolutional layers and fully connected layers, exhibiting extremely low computational overhead and parameter count, making it suitable for deployment on edge devices.
[0027] In this embodiment of the invention, the sub-network first receives statistical features of the damage segmentation map as input, including the pixel ratio and spatial distribution entropy of various types of damage (such as minor cracks, severe spalling, and rebar buckling). The pixel ratio reflects the significance of each type of damage in the overall damage by statistically analyzing the proportion of pixels occupied by each type of damage in the image; the spatial distribution entropy is calculated based on the Shannon entropy formula and is used to quantify the spatial dispersion of various types of damage, thereby reflecting their potential impact on the overall structural performance. These statistical features are input into a 1×1 convolutional layer for cross-channel feature fusion and preliminary nonlinear mapping, and then output a multi-scale spatial attention weight coefficient matrix through a fully connected layer.
[0028] Specifically, in this embodiment of the invention, the number of channels in a 1×1 convolutional layer is typically set to match the number of damage categories (e.g., 3 channels for 3 damage categories), and the activation function can be either ReLU or Swish to enhance nonlinear expressive power. The output dimension of the fully connected layer matches the scale number of the attention mechanism in the LLaVA module; for example, it outputs weight coefficients at 3 scales, corresponding to local, mesoscopic, and global damage features, respectively. A two-stage optimization mechanism is adopted during training: in the short-term freeze-fine-tuning stage, only the weight generation subnetwork is trained, and the LLaVA backbone parameters are frozen; in the long-term joint optimization stage, the backbone is unfrozen, and end-to-end collaborative training is performed to enhance the overall consistency of the model.
[0029] This sub-network can be embedded in edge computing devices or mobile terminals to generate attention weights in real time, assisting the LLaVA module in more accurately understanding damage semantics and thus outputting textual descriptions and damage indices of structural damage. Its technical value lies in establishing a nonlinear mapping relationship between damage type and attention intensity by quantifying the statistical characteristics of the damage map, enabling key damage regions to receive higher attention weights in model inference, thereby improving the accuracy and interpretability of the evaluation results.
[0030] Furthermore, S2 includes: S21 removes redundant multi-head attention layers in the lightweight SAM module through a structured pruning strategy, while retaining key feature extraction paths to reduce computational complexity.
[0031] Specifically, this step removes redundant multi-head attention layers from the lightweight SAM module using a structured pruning strategy, thereby effectively reducing the model's computational complexity and parameter count without significantly affecting segmentation accuracy. In some implementations, this pruning strategy is based on a hierarchical knowledge distillation framework, which introduces a teacher-student model structure to transfer key semantic segmentation capabilities from the original SAM model to the lightweight student model. Specifically, the teacher model uses a standard SAM architecture containing multiple multi-head self-attention (MHSA) layers, while the student model adopts a convolutional-attention hybrid structure, retaining only some key MHSA layers.
[0032] Structured pruning is performed within the attention module of the student model. By analyzing the activation values, gradient contributions, and feature map correlations of each attention head, redundant heads with minimal impact on overall segmentation performance are identified. At the parameter level, the Attention Head Importance Score (AHIS) is used as the evaluation metric during pruning. AHIS is calculated based on the feature reconstruction error and gradient sensitivity of each head, with a threshold of 0.15. Attention heads below this threshold are removed. Furthermore, the number of parameters in the pruned model can be reduced by approximately 35%, and FLOPs are reduced to below 60% of the original model, meeting the computational resource constraints of edge device deployment.
[0033] In some implementations, this step is applicable to real-time damage detection systems for RC structural members after earthquakes, offering significant deployment advantages, especially in resource-constrained environments such as mobile terminals or drone platforms. By preserving key feature extraction paths, the model maintains high-precision segmentation capabilities for typical damage types such as cracks, spalling, and exposed reinforcement, ensuring the quality of the damage map.
[0034] The technical effect of this step is that it significantly improves the inference efficiency of the model through structured pruning, while maintaining segmentation performance, providing high-quality segmentation map input for the subsequent damage understanding module, thereby enhancing the real-time performance and interpretability of the overall evaluation system.
[0035] S22 introduces a dynamic parameter sharing mechanism in the encoder, enabling feature maps of different scales to share some learnable parameters to improve the model's generalization ability.
[0036] Specifically, a dynamic parameter sharing mechanism is introduced into the encoder to improve the model's generalization ability and computational efficiency when processing feature maps of different scales by sharing some learnable parameters. The core principle of this mechanism lies in designing a parameter sharing strategy based on the semantic consistency and structural similarity of multi-scale feature maps. This allows feature extraction modules at different levels to reuse some weight parameters, thereby reducing model redundancy and enhancing the model's adaptability to damaged features. In some implementations, this mechanism can dynamically select shared convolutional kernel or attention module parameters based on the spatial resolution differences of the feature maps. For example, in the encoder of a lightweight SAM model, a cross-layer shared multi-head attention (MHA) weight matrix is used, enabling parameter reuse between high-level and low-level features in the semantic space.
[0037] Optionally, the dynamic parameter sharing mechanism introduces a learnable gating function or a shared weight controller to dynamically determine whether to share parameters based on the scale information of the current feature map (such as resolution, number of channels, spatial distribution entropy, etc.). Furthermore, parameter sharing can be applied to the query, key, and value projection matrices in convolutional layers, normalization layers, or attention mechanisms. For example, between layers 3 and 5 of the encoder, some weight parameters of the attention head can be shared, with the sharing ratio set to 50%-70% to balance model performance and parameter efficiency.
[0038] Specifically, the dimensions of shared parameters are typically 1. ,in For the hidden layer dimension of the model, The dimension of each attention head. The update method for shared parameters can employ a gradient synchronization strategy to ensure semantic consistency of shared parameters across feature maps at different scales. Furthermore, the introduction of the sharing mechanism needs to be combined with a structured pruning strategy to avoid performance degradation caused by parameter redundancy. Typically, when the proportion of shared parameters exceeds 60%, a compensatory feature enhancement module (such as 1×1 convolution or residual connections) needs to be introduced to maintain the model's expressive power.
[0039] This step is particularly useful in practical applications when earthquake damage images exhibit multi-scale damage features, such as the coexistence of small crack areas and large areas of spalling. Through dynamic parameter sharing, the model can maintain consistent semantic understanding across different scales, thereby improving the accuracy and stability of damage segmentation.
[0040] In terms of technical effectiveness, the dynamic parameter sharing mechanism significantly reduces the number of parameters and computational complexity of the model, while enhancing its adaptability to damage features at different scales. Experiments show that with a shared parameter ratio of 60%, the model's mIoU metric on the DamageNet dataset decreases by only 1.2%, while computational cost is reduced by approximately 35%, demonstrating a good balance between performance and efficiency. This mechanism provides high-quality, consistent semantic segmentation maps for subsequent damage understanding modules (such as LLaVA), and is a key foundation for achieving interpretability in damage assessment and description.
[0041] S3, perform non-rigid motion correction, cell segmentation and trace extraction on the calcium ion fluorescence signal to obtain the activity characteristics of each neuron at different stimulation stages.
[0042] Specifically, the constructed lightweight LLaVA damage understanding module achieves multimodal understanding of RC structure seismic damage images by integrating a spatial attention mechanism driven by damage segmentation maps, full-parameter fine-tuning, and knowledge distillation techniques. This module takes the original images of the damaged components and their corresponding damage segmentation maps as input, extracts image features through a visual encoder, and introduces spatial attention weight coefficients during feature extraction to guide the model in enhancing the features of key damage areas (such as cracks, spalling, and exposed reinforcement), thereby improving the semantic understanding and assessment accuracy of damage identification.
[0043] In some implementations, the spatial attention weight coefficients are generated from the output of a lightweight weight generation subnetwork, which performs a non-linear mapping based on the categorical statistical features of the damage segmentation map (such as the pixel proportion of each type of damage and the spatial distribution entropy). The LLaVA model introduces this weight matrix into the visual encoder, using element-wise multiplication to weight and enhance the image feature map, allowing the model to focus more on damage regions that significantly impact structural performance during processing. Furthermore, the model employs a full-parameter fine-tuning strategy, utilizing the multimodal image dataset constructed in step 1 to optimize visual-language alignment capabilities, ensuring the model can accurately understand damaged images and generate corresponding natural language descriptions.
[0044] The spatial attention weight coefficients typically have the same dimensionality as the spatial resolution of the image feature map (e.g., 14×14 or 28×28), with values ranging from [0,1] and normalized using the Sigmoid function. During model training, a weighted combination of the cross-entropy loss function and the language generation loss function is used, with a weighting ratio of 0.7:0.3, to balance the quality of damage index prediction and natural language description generation.
[0045] This module can be applied to scenarios such as post-earthquake emergency assessment, structural health monitoring systems, and automated damage report generation, and is particularly suitable for deployment on mobile or edge computing devices. Its technical value lies in significantly improving the transparency and credibility of the model output by introducing an interpretable attention mechanism. At the same time, by combining the semantic information of the damage segmentation map, it achieves the synchronous output of damage index and natural language description, providing engineers with intuitive and traceable assessment basis.
[0046] Furthermore, S3 includes: S31 transfers the complex visual-language alignment capabilities of the fine-tuned model to a simplified architecture through hierarchical knowledge distillation, while retaining key semantic association features.
[0047] Specifically, in this invention, the lightweight LLaVA damage understanding module achieves deep semantic understanding and efficient reasoning of structural damage by fusing a spatial attention mechanism driven by a damage segmentation map, full parameter fine-tuning, and hierarchical knowledge distillation. Specifically, this module first introduces a damage segmentation map guidance mechanism into the visual encoder, dynamically adjusting the spatial attention weights of the visual feature map by using the output semantic segmentation map as auxiliary input. In some implementations, this mechanism employs a weighted fusion strategy, concatenating the original image features and segmentation map features along the channel dimension, and weighting the features using a learnable attention weight matrix, thereby enhancing the model's ability to perceive key damage areas (such as steel buckling and concrete spalling).
[0048] Furthermore, the LLaVA model was fine-tuned with all parameters using a constructed multimodal image dataset. During fine-tuning, a joint optimization of the cross-entropy loss function and contrastive learning loss was employed to improve the model's performance in the vision-language alignment task. The model input consisted of the original seismic damage image and its corresponding damage segmentation map, and the output was a natural language description of the damage type, location, and severity. During the training phase, the number of model parameters was kept below 120 million, and the inference latency was less than 300ms, meeting the constraints of real-time performance and computational resources required for engineering deployment.
[0049] Finally, through hierarchical knowledge distillation, the visual-language alignment capability of the fine-tuned complex LLaVA model was transferred to a simplified architecture. The teacher model is a fully parameter-tuned LLaVA, while the student model is a lightweight version with structured pruning. During the distillation process, a combination of intermediate layer feature alignment and output layer probability distribution matching was used, with loss functions including KL divergence loss and feature consistency loss, with weight coefficients of 0.6 and 0.4, respectively. This step significantly reduced the computational complexity of the model while preserving key semantic features, improving the feasibility of deployment and the interpretability of the assessment results in real-world post-earthquake assessment scenarios.
[0050] S32 uses low-rank tensor decomposition technology to compress the high-dimensional weight tensor of the decoder, decomposing the weight matrix into a low-rank core tensor and a factor matrix to reduce storage requirements.
[0051] Specifically, the high-dimensional weight tensors of the decoder are compressed using low-rank tensor decomposition (LTD). The core principle of LTD is to approximate the high-dimensional tensor as a combination of several low-rank tensors and factor matrices, thereby significantly reducing the number of model parameters and storage overhead without significantly sacrificing model performance. In some implementations, this step can employ mainstream tensor decomposition methods such as CP decomposition (CANDECOMP / PARAFAC) or Tucker decomposition. Tucker decomposition is preferred due to its advantage in preserving tensor structural information.
[0052] In the specific implementation, firstly, the weight tensors of the fully connected or convolutional layers in the decoder are treated as three-dimensional or higher-dimensional tensors, for example, with the shape of... The convolution kernel tensor, where and These represent the number of input and output channels, respectively. Let be the kernel size. Then, Tucker decomposition is performed on this tensor, representing it as a low-rank kernel tensor. With multiple factor matrices , The product form. By setting an appropriate rank parameter. , This can effectively compress the storage space of the original tensor and reduce computational complexity.
[0053] This technology is integrated into the decoder module of a lightweight SAM model to compress high-dimensional feature mapping transformation operations during semantic segmentation. This step reduces the number of model parameters by more than 30% while maintaining segmentation accuracy with a decrease in mIoU of no more than 2%. This compression strategy significantly improves the feasibility of deploying the model on edge devices or mobile devices, providing fundamental support for the efficient operation of subsequent damage understanding modules.
[0054] S4. The SAM module and LLaVA module are jointly trained through a two-stage optimization mechanism, which includes a short-term freeze-and-fine-tuning stage and a long-term joint optimization stage, in order to achieve a balance between reducing the number of model parameters and improving evaluation accuracy.
[0055] Specifically, the dual-stage optimization mechanism proposed in this invention aims to achieve joint optimization of the SAM module and the LLaVA module through a phased training strategy, thereby improving evaluation accuracy while reducing the number of model parameters. At the technical implementation level, this mechanism is divided into a short-term freeze-fine-tuning stage and a long-term joint optimization stage. In the short-term freeze-fine-tuning stage, the parameters of the SAM module and the LLaVA backbone network are first fixed, and only a lightweight weight generation sub-network is trained. This sub-network consists of 1x1 convolutional layers and fully connected layers. The input is the statistical features of the damage segmentation map, including the pixel ratio and spatial entropy of various types of damage (such as cracks, spalling, and exposed reinforcement), and the output is the multi-scale spatial attention weight coefficients in the LLaVA module. Through training in this stage, the model initially establishes a nonlinear mapping relationship between damage type and attention intensity, laying the foundation for subsequent joint optimization.
[0056] In the long-term joint optimization phase, the SAM and LLaVA backbone networks are unfrozen, and an end-to-end joint training approach is adopted to enable the weight generation subnetwork and the backbone model parameters to be updated collaboratively. During training, a constructed multimodal image dataset is used, with damage index and natural language description as supervision signals, to optimize the overall loss function, including cross-entropy loss and attention consistency loss. Regarding parameter settings, the learning rate is typically set to 1e-3 during the frozen phase, and the optimizer is AdamW; during the joint optimization phase, the learning rate is reduced to 1e-4 to avoid drastic parameter perturbations. Through this two-stage mechanism, the model significantly improves its ability to perceive and evaluate key damage regions while maintaining lightweight design, enhancing the overall system's robustness and interpretability, and providing a solid foundation for subsequent engineering deployment.
[0057] Furthermore, S4 includes: S41 fixes the backbone network parameters during the short-term freeze-fine-tuning phase and trains only lightweight weight-generating subnetworks to achieve rapid convergence.
[0058] Specifically, in the short-term freeze-fine-tuning phase, fixing the backbone network parameters and training only lightweight weighted subnetworks is a key step in optimizing the training strategy of the damage understanding module in this invention. The technical principle behind this step is based on a collaborative mechanism of model parameter freezing and subnetwork fine-tuning, aiming to accelerate model convergence by reducing the number of training parameters while preserving the damage feature representation capabilities already learned by the backbone network.
[0059] In some implementations, the backbone network (such as the visual encoder of a lightweight LLaVA model) has already undergone pre-training and full parameter fine-tuning in the early stages, possessing strong capabilities for extracting damage image features. To avoid destroying existing feature representations during subsequent training, this invention sets the parameters of the backbone network to an untrainable state during the short-term freeze-tuning phase, optimizing only the newly added lightweight weight generation sub-network. This sub-network typically consists of 1×1 convolutional layers and fully connected layers. Its input is the categorical statistical features of the damage segmentation map, including quantitative indicators such as the pixel proportion and spatial distribution entropy of various types of damage (e.g., cracking, peeling, exposed tendons). The output is multi-scale spatial attention weight coefficients, used to guide the model to focus on key regions during damage understanding.
[0060] The input dimension of the subnetwork is typically twice the number of damage categories (pixel percentage and entropy value), while the output dimension corresponds to the number of channels or spatial scale of the attention mechanism in the LLaVA model. During training, the Adam optimizer is used with a learning rate set between 1e-4 and 5e-4 to ensure rapid convergence of the subnetwork without affecting the stability of the backbone network. The loss function can combine mean squared error (MSE) and cross-entropy loss to simultaneously optimize the numerical accuracy and semantic relevance of the weight coefficients.
[0061] In practical applications, this step is suitable for the rapid optimization phase before model deployment, especially in post-earthquake emergency assessment environments with limited computing resources or tight deadlines. By freezing the backbone network, training overhead can be significantly reduced while ensuring more precise attention allocation in key areas, thereby improving the accuracy of damage index prediction and the interpretability of assessment results. Technically, this strategy effectively balances model lightweighting and performance optimization, laying a stable foundation for subsequent joint optimization phases and serving as a crucial guarantee for the efficient deployment of end-to-end damage assessment systems.
[0062] S42, during the long-term joint optimization phase, unfreezes the backbone network parameters and simultaneously trains the sub-networks and backbone network through an alternating optimization strategy to improve overall performance.
[0063] Specifically, in the long-term joint optimization phase, unfreezing the backbone network parameters and simultaneously training the sub-networks and backbone network using an alternating optimization strategy are key technical steps in this invention to achieve lightweighting and performance improvement of the damage understanding module. This phase aims to significantly reduce computational and storage overhead while maintaining high accuracy by collaboratively optimizing the parameters of the backbone network and the weight generation sub-network.
[0064] In its implementation, this step first generates a lightweight weighted sub-network (composed of 1×1 convolutional layers and fully connected layers) based on the constructed sub-network. This sub-network receives statistical features of the damage segmentation map as input, including the pixel percentage of various damage types (e.g., crack percentage, spalling percentage, exposed reinforcement percentage) and quantitative indicators such as spatial distribution entropy. The sub-network outputs multi-scale spatial attention weight coefficients, which guide the visual encoder of the LLaVA model to focus on key damage regions at different scales. During the long-term joint optimization phase, the backbone network parameters are unfrozen and participate in training together with the sub-network parameters. An alternating optimization strategy is adopted: in each round of training, the sub-network parameters are fixed first to optimize the backbone network, then the backbone network parameters are fixed to optimize the sub-network, iterating until convergence.
[0065] Specifically, the number of parameters in the backbone network is typically controlled to be less than 30% of that in the original LLaVA model to meet the deployment requirements of edge devices. The output dimension of the sub-networks is consistent with the number of channels in the attention module of the backbone network to ensure the compatibility of weight mapping. During training, a joint optimization is performed using the cross-entropy loss function and the KL divergence loss. The KL divergence is used to measure the knowledge consistency between the sub-network output and the fully parameterized model. The loss weight coefficient is set to 0.3 to balance the training objectives of aligning the segmentation graph guidance with the language model.
[0066] In practical applications, this step is suitable for rapid assessment systems of RC structures after earthquakes, especially in edge computing environments deployed on mobile terminals or drone platforms. Through an alternating optimization strategy, the model can achieve high-precision damage index prediction and interpretable text output with limited computing power, significantly improving assessment efficiency and decision reliability.
[0067] The technical effect of this step is that by unfreezing the backbone network and introducing an alternating optimization mechanism, parameter coordination between the backbone network and sub-networks is achieved, enhancing the model's ability to perceive key damage features. At the same time, it improves the model's generalization performance and inference efficiency, providing a solid foundation for subsequent damage description and engineering decision-making.
[0068] Furthermore, S1 includes: S5, perform multimodal training and annotation on the original damaged component image, including: Specifically, performing multimodal training and annotation on the original images of the damaged components is a key step in constructing a high-quality training dataset in the method of this invention. This step aims to provide structured and semantically rich input information for subsequent damage segmentation and multimodal evaluation models, thereby improving the model's perceptual accuracy and evaluation interpretability.
[0069] In some implementations, this annotation process includes two main subtasks: pixel-level damage segmentation annotation and text description annotation. First, semantic segmentation tools (such as LabelMe, VIA, or professional building structure annotation platforms) are used to annotate the damaged areas of the RC components in the image pixel-by-pixel. Damage categories include, but are not limited to, "cracking," "stripping," "exposed rebar," and "rebar buckling," with each type of damage requiring a separate label map for differentiation. Second, text annotation is performed on each image, covering the component's Damage Index (DI) and a natural language description of the damage phenomenon, such as "A longitudinal crack appears at the bottom of the component, approximately 0.3 mm wide, with no obvious spalling observed." The text annotation must adhere to a unified semantic template to ensure consistency and parsability of the description.
[0070] Specifically, the damage index (DI) is typically quantified according to the damage level standards defined in the "Code for Seismic Design of Building Structures" (GB 50011) or the "Code for Durability Design of Concrete Structures" (GB / T 50476), with a value ranging from 0 to 1, corresponding to no damage and complete failure, respectively. Pixel-level annotations must meet an IoU (Intersection over Union) threshold of no less than 0.85 to ensure the accuracy of the segmentation map. The spatial distribution entropy value is calculated using the Shannon entropy formula and is used to measure the dispersion of various types of damage in the image. Its value ranges from 0 to 1, with a higher entropy value indicating a more irregular damage distribution.
[0071] This annotation process is applicable to image acquisition and processing of RC structural components after an earthquake. Image sources include drone aerial photography, ground cameras, and mobile phone photography. An image resolution of at least 1024×768 is recommended to ensure the discernibility of damage details. After annotation, the dataset will be used to train the lightweight SAM damage segmentation module and the LLaVA damage understanding module, providing the model with dual visual and semantic supervision signals.
[0072] The technical effect of this step is that it establishes a strong correlation between damage images and structural assessment indicators through multimodal annotation, providing a foundation for introducing spatial attention mechanisms and knowledge distillation strategies into subsequent models, and significantly improving the model's generalization ability and the interpretability of assessment results in complex seismic damage scenarios.
[0073] S51 provides pixel-level annotations for different damage categories on damaged components in the image, covering typical damage types such as cracking, spalling, and exposed reinforcement.
[0074] Specifically, this step aims to perform pixel-level annotation of different damage categories on RC structural components in the image, covering typical damage types such as cracking, spalling, and exposed rebar. It is a fundamental step in constructing a damage segmentation map in a multimodal assessment system. Technically, this annotation process typically employs semantic segmentation technology, using deep learning models (such as U-Net, MaskR-CNN, or improved SAM models) to classify each pixel in the image, achieving accurate identification and localization of damaged areas. During annotation, damage must be divided into multiple semantic categories, such as: minor cracks, moderate cracks, severe cracks, localized spalling, large-area spalling, and exposed rebar, ensuring that the damage type definitions conform to the classification standards in the "Technical Specification for Inspection and Evaluation of Building Structures" (GB / T50344).
[0075] In some implementations, annotation accuracy is typically evaluated using Intersection over Union (IoU) and Mean Average Precision (mAP), requiring an overall IoU value of no less than 0.85 to ensure the reliability of subsequent model training. Furthermore, during annotation, statistical characteristics such as the pixel percentage and spatial distribution entropy of each type of damage need to be recorded as input parameters for subsequent attention weight generation. For example, when the crack pixel percentage exceeds 30%, it may trigger a significant increase in the overall damage index of the component.
[0076] This step is widely used in rapid assessment systems for reinforced concrete (RC) structures after earthquakes. Images acquired through drone aerial photography, ground inspection cameras, and other equipment are pixel-level annotated to generate damage segmentation maps, providing structured input for subsequent damage index calculations and natural language descriptions. These maps can be directly used to guide on-site engineers in determining whether components require immediate reinforcement or replacement.
[0077] The technical value of this step lies in the fact that, through refined pixel-level annotation, it not only improves the accuracy of damage identification, but also introduces an interpretability mechanism into the model, enabling the damage assessment results to be quantified and described based on the distribution and severity of specific damage areas, thereby achieving a technological leap from "being able to see" to "being able to explain".
[0078] S52, synchronously generate a text description containing the component damage index and typical damage characteristics, the text description specifically explaining key parameters such as the concrete spalling range, longitudinal reinforcement buckling angle and distribution density.
[0079] Specifically, the constructed lightweight LLaVA damage understanding module achieves semantic understanding and interpretable output of structural damage by fusing a spatial attention mechanism driven by damage segmentation maps, full parameter fine-tuning, and knowledge distillation techniques. Specifically, this module introduces a damage segmentation map guidance mechanism into the visual encoder. During the image feature extraction stage, the output semantic segmentation map is used as auxiliary input, and the spatial attention weights are dynamically adjusted through weighted fusion. In some implementations, this mechanism employs a learnable attention fusion layer (such as the SE Block or CBAM module) to assign differentiated importance weights to regions of different damage categories (such as spalling, cracks, exposed reinforcement, etc.) in the segmentation map, thereby guiding the model to focus more on key damage regions that have a greater impact on structural performance.
[0080] The visual encoder of the LLaVA model is typically based on the ViT (Vision Transformer) architecture, with an input image size of 224×224 pixels and 3 channels. During fine-tuning, a full-parameter fine-tuning strategy is employed, with a learning rate set to 1e-5, an AdamW optimizer selected, a weight decay coefficient of 0.05, and a training batch size of 16. Furthermore, to improve the model's interpretability and inference efficiency, a hierarchical knowledge distillation method is used to transfer the capabilities of the fine-tuned full model to a simplified architecture, such as ViT-B / 16 or the smaller ViT-Tiny model. This reduces the number of model parameters to less than 30% of the original model, meeting the deployment requirements of edge devices.
[0081] In practical applications, this module receives the original seismic damage image and its corresponding damage segmentation map, and outputs a textual description including the component damage index and typical damage characteristics. For example, for a concrete column component, the system can output "The concrete spalling area accounts for 18% of the component surface area, the maximum buckling angle of the longitudinal reinforcement is 25°, and the buckling area distribution density is 0.6 units / square meter", and provide a qualitative analysis of the impact on structural performance in combination with the damage location and morphology.
[0082] This step plays a crucial role in the overall technical solution, not only transforming image recognition results into interpretable text information, but also enhancing the model's ability to perceive key damage features through spatial attention mechanisms, thereby significantly improving the accuracy and practicality of earthquake damage assessment.
[0083] Example 2 A multimodal assessment method for seismic damage of RC structures driven by damage segmentation maps, such as Figure 2 As shown, the main steps include: S10, construct a brand-new multimodal image dataset of seismically damaged RC components; Image data of earthquake-damaged reinforced concrete (RC) components were collected from existing experimental data, publicly available online resources, and earthquake damage reports. Multimodal training and annotation were performed on the collected images. First, different damage categories on the damaged components in the images were annotated at the pixel level, including cracking, spalling, and exposed reinforcement. Then, text annotations were added to the images, including the degree of damage (damage index) and a textual description of the damage phenomena. The textual description included the overall damage degree of the component shown in the image and typical damage characteristics, such as whether concrete spalling occurred, whether buckling of the longitudinal reinforcement was observed, and the degree of buckling.
[0084] S20 constructs a lightweight SAM (Segment Anything Model) damage segmentation module that integrates knowledge distillation-driven structured pruning, dynamic parameter sharing, and low-rank tensor decomposition. First, hierarchical knowledge distillation is used to transfer the semantic segmentation capabilities of the original backbone to a lightweight convolutional-attention hybrid architecture, while simultaneously implementing structured pruning to remove redundant multi-head attention layers. Second, a dynamic parameter sharing mechanism is introduced into the encoder. Finally, block decomposition is performed on the decoder, compressing the high-dimensional weight tensor into a low-rank core tensor and factor matrix, significantly reducing storage and computational complexity. The established module is lightweight and easy to deploy in practical engineering. It accepts original images of earthquake-damaged components as input and can output semantic segmentation maps of different types of structural damage on the components.
[0085] S30 constructs a lightweight LLaVA (Large Language-and-Vision Assistant) damage understanding module that integrates a damage segmentation map-driven spatial attention mechanism, full-parameter fine-tuning, and knowledge distillation. First, a unique damage segmentation map-guided mechanism is introduced into the visual encoder, dynamically assigning spatial attention weights based on different categories of structural damage regions on the input damage segmentation map. Then, the LLaVA model is fine-tuned using a multimodal image dataset of seismically damaged RC components established in S10. Finally, hierarchical knowledge distillation transfers the complex vision-language alignment capabilities of the fine-tuned model to a simplified architecture, enabling efficient inference and end-to-end deployment in engineering. The proposed module possesses the characteristics of deep understanding of damage semantics and lightweight design. The module accepts the original seismically damaged image and its corresponding damage segmentation map as input and can output a natural language description of the type, location, and severity of structural damage in the image.
[0086] S40 constructs a lightweight weight generation subnetwork (1x1 convolutional layer + fully connected layer), receiving the category statistical features of the damage segmentation map (pixel proportion of each type of damage, spatial distribution entropy value) as input. The subnetwork's function is to output the multi-scale spatial attention weight coefficients proposed in S30. During training, a two-stage optimization mechanism is adopted, namely short-term freeze fine-tuning (fixing the backbone and training only the weight generator) and long-term joint optimization (unfreezing the backbone for full-parameter co-training), ultimately establishing a non-linear mapping relationship between damage type and attention intensity, so that key damage regions are strengthened with weights.
[0087] S50 organically integrates the algorithm modules from S20 to S40 to form a brand-new technical solution, realizing the entire process of structural damage perception, identification, segmentation, evaluation and description output.
[0088] Example 3 To achieve the above embodiments, such as Figure 3 As shown, this embodiment also provides a damage segmentation map-driven seismic damage assessment device 10 for RC structures, comprising: The lightweight SAM damage segmentation module 100 is used to perform semantic segmentation on the original image of the damaged component and output a pixel-level segmentation map containing different types of structural damage. The multi-scale spatial attention weight construction module 200 is used to construct multi-scale spatial attention weight coefficients based on the category statistical features of the segmentation map. The category statistical features include the pixel proportion and spatial distribution entropy value of various types of damage. A nonlinear mapping relationship between damage type and attention intensity is established through a lightweight weight generation sub-network. The lightweight LLaVA damage understanding module 300 is used to input the original image of the damaged component and the segmentation map into the lightweight LLaVA damage understanding module, use the spatial attention weight coefficient to guide the visual encoder to perform feature enhancement on the key damage area, and simultaneously output the damage index and natural language description corresponding to the damage features. The dual-stage optimization joint training module 400 is used to jointly train the SAM module and the LLaVA module through a dual-stage optimization mechanism, which includes a short-term freeze-fine-tuning stage and a long-term joint optimization stage, in order to achieve a balance between reducing the number of model parameters and improving evaluation accuracy.
[0089] Furthermore, the scale-space attention weight construction module 200 is also used for: The redundant multi-head attention layer in the lightweight SAM module is removed by a structured pruning strategy, while retaining the key feature extraction path to reduce computational complexity. A dynamic parameter sharing mechanism is introduced into the encoder, which allows feature maps of different scales to share some learnable parameters to improve the model's generalization ability.
[0090] Furthermore, the lightweight LLaVA damage understanding module 300 is also used for: By using hierarchical knowledge distillation, the complex visual-language alignment capabilities of the fine-tuned model are transferred to a simplified architecture, while retaining key semantic association features; The high-dimensional weight tensor of the decoder is compressed using low-rank tensor decomposition technology, which decomposes the weight matrix into a low-rank core tensor and a factor matrix to reduce storage requirements.
[0091] Furthermore, the two-stage optimization joint training module 400 is also used for: During the short-term freeze-fine-tuning phase, the parameters of the backbone network are fixed, and only lightweight weights are trained to generate subnetworks for rapid convergence. During the long-term joint optimization phase, the backbone network parameters are unfrozen, and the sub-networks and backbone networks are trained synchronously through an alternating optimization strategy to improve overall performance.
[0092] Furthermore, it also includes: A multimodal training and annotation module is used to perform multimodal training and annotation on the original images of earthquake-damaged components, including: Pixel-level annotations are performed on different damage categories on damaged components in the image, covering typical damage types such as cracking, spalling, and exposed reinforcement. Simultaneously generate textual descriptions containing component damage indices and typical damage characteristics, with the textual descriptions specifically explaining key parameters such as concrete spalling range, longitudinal reinforcement buckling angle, and distribution density.
[0093] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0094] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.
Claims
1. A method for earthquake damage assessment of RC structures based on damage segmentation map, characterized in that, Comprise: S1, the original seismic damage component image is carried out semantic segmentation by lightweight SAM damage segmentation module, and the pixel level segmentation atlas containing different category structure damage is output; S2, the multi-scale space attention weight coefficient is constructed based on the category statistical characteristics of the segmentation atlas, the category statistical characteristics include the pixel proportion and the spatial distribution entropy value of each category damage, and the nonlinear mapping relationship between the damage type and the attention intensity is established by the lightweight weight generation subnetwork; S3, the original seismic damage component image and the segmentation atlas are input into the lightweight LLaVA damage understanding module, the spatial attention weight coefficient is used to guide the visual encoder to strengthen the characteristics of the key damage area, and the damage index and the natural language description corresponding to the damage characteristics are output; S4, the SAM module and the LLaVA module are trained by a two-stage optimization mechanism, the two-stage optimization mechanism includes a short-term freezing fine-tuning stage and a long-term joint optimization stage, so that the balance between reducing the model parameter quantity and improving the evaluation accuracy is realized.
2. The method of claim 1, wherein, S2, comprising: S21, the redundant multi-head attention layer in the lightweight SAM module is removed by a structured pruning strategy, and the key feature extraction path is reserved to reduce the calculation complexity; S22, a dynamic parameter sharing mechanism is introduced in the encoder, so that the feature maps of different scales share part of the learnable parameters to improve the model generalization ability.
3. The method of claim 1, wherein, S3, comprising: S31, the complex visual-linguistic alignment ability of the fine-tuned model is migrated to the simplified architecture by hierarchical knowledge distillation, and the key semantic correlation features are reserved; S32, the high-dimensional weight tensor of the decoder is compressed by using low-rank tensor decomposition technology, and the weight matrix is decomposed into a low-rank core tensor and a factor matrix to reduce the storage requirement.
4. The method of claim 1, wherein, S4, comprising: S41, in the short-term freezing fine-tuning stage, the backbone network parameters are fixed, and only the lightweight weight generation subnetwork is trained to quickly converge; S42, in the long-term joint optimization stage, the backbone network parameters are unfrozen, and the subnetwork and the backbone network are trained synchronously by an alternating optimization strategy to improve the overall performance.
5. The method of claim 1, wherein, Further comprising: S5, the original seismic damage component image is trained and labeled in multiple modes, comprising: S51, the different damage categories on the damaged components in the image are pixel-level labeled, covering typical damage types such as cracking, spalling and exposed reinforcement; S52, the text description containing the component damage index and the typical damage characteristics is generated synchronously, and the text description specifically describes the key parameters such as the spalling range of concrete, the bending angle and distribution density of longitudinal reinforcement.
6. A device for evaluating the seismic damage of RC structures based on a damage segmentation map, characterized by Comprise: A lightweight SAM damage segmentation module is used for semantic segmentation of the original seismic damage component image, and a pixel level segmentation atlas containing different category structure damage is output; A multi-scale space attention weight construction module is used for constructing a multi-scale space attention weight coefficient based on the category statistical characteristics of the segmentation atlas, the category statistical characteristics include the pixel proportion and the spatial distribution entropy value of each category damage, and the nonlinear mapping relationship between the damage type and the attention intensity is established by the lightweight weight generation subnetwork; The lightweight LLaVA damage understanding module is configured to input the original damage component image and the segmentation atlas into the lightweight LLaVA damage understanding module, guide a visual encoder to perform feature enhancement on a key damage area by using the spatial attention weight coefficient, and synchronously output a damage index and a natural language description corresponding to a damage feature; The two-stage optimization joint training module is configured to perform joint training on the SAM module and the LLaVA module by using a two-stage optimization mechanism, the two-stage optimization mechanism includes a short-term frozen fine-tuning stage and a long-term joint optimization stage, so as to balance reduction of model parameter quantity and improvement of evaluation accuracy.
7. The apparatus of claim 6, wherein, The multi-scale spatial attention weight construction module is further configured to: Remove redundant multi-head attention layers in the lightweight SAM module by using a structured pruning strategy, and reserve a key feature extraction path to reduce computational complexity; Introduce a dynamic parameter sharing mechanism in the encoder, so that feature maps of different scales share part of learnable parameters to improve model generalization ability.
8. The apparatus of claim 6, wherein, The lightweight LLaVA damage understanding module is further configured to: Migrate complex visual-language alignment capability of the fine-tuned model to a simplified architecture by using hierarchical knowledge distillation, and reserve key semantic correlation features; Compress a high-dimensional weight tensor of the decoder by using a low-rank tensor decomposition technology, decompose the weight matrix into a low-rank core tensor and a factor matrix to reduce storage requirements.
9. The apparatus of claim 6, wherein, The two-stage optimization joint training module is further configured to: Fix the backbone network parameters in the short-term frozen fine-tuning stage, and only train the lightweight weight generation subnetwork to quickly converge; Unfreeze the backbone network parameters in the long-term joint optimization stage, and synchronously train the subnetwork and the backbone network by using an alternating optimization strategy to improve overall performance.
10. The apparatus of claim 6, wherein, Further comprising: A multi-modal training labeling module is configured to perform multi-modal training labeling on the original damage component image, including: Performing pixel-level labeling on different damage classes on the damaged component in the image, covering typical damage types such as cracking, spalling, and exposed reinforcement; Synchronously generating a text description containing a component damage index and typical damage features, the text description specifically describes key parameters such as concrete spalling range, longitudinal reinforcement buckling angle, and distribution density.