A method for locating damage to a post-fire concrete structure

Abstract

The application discloses a method for locating damage of a post-fire concrete structure, and belongs to the technical field of post-fire structure damage detection.The method comprises the following steps: (1) constructing a multi-source data set based on an RGB image, a depth image and point cloud information; (2) constructing a semantic segmentation network based on YOLOv10-SimAM and a point cloud fusion network based on a parallel structure; (3) network training and performance evaluation; and (4) locating damage of a post-fire concrete structure through the network.The application firstly extracts point cloud information of a single damage by using two-dimensional segmentation, and then performs minimum bounding box regression on the point cloud information of the damage area, so that the calculation efficiency is effectively improved.The application proposes a method for fusing multi-view point cloud information, solves the limitation of single-view detection and positioning, and realizes global damage positioning of the point cloud information.

Description

Technical Field

[0001] This invention belongs to the field of post-fire structural damage detection technology, specifically relating to a method for locating damage to concrete structures after a fire. Background Technology

[0002] Concrete structures, as a major form of modern construction, are frequently subjected to adverse factors such as fire and sustained loads, resulting in irregular damage such as cracks and spalling. Traditional damage detection mainly relies on contact sensors or manual visual inspection, but these instruments are easily damaged in high-temperature fire environments, and the detection efficiency is low. In recent years, computer vision inspection technology based on deep learning has been widely used due to its non-destructive nature and high efficiency, enabling automated identification of cracks and spalling. However, existing two-dimensional vision inspection methods are highly dependent on lighting conditions, and due to the lack of spatial depth information, they cannot accurately quantify key three-dimensional geometric features such as the volume of the damaged area and the depth of spalling.

[0003] While 3D point cloud information can effectively address the planar limitations of 2D damage detection, it still has significant shortcomings in fire-related engineering scenarios. Firstly, existing 3D point cloud information typically processes visual and geometric features separately, lacking an effective two-way interaction mechanism. This leads to significant noise and data gaps in harsh environments with dust or low visibility, affecting the accuracy of damage detection and localization. Secondly, 3D point cloud data is characterized by its massive volume; directly extracting damage from large-scale point clouds results in a surge in computational costs, making it difficult to meet the demands of automated detection and localization. Therefore, developing a rapid 3D damage localization method adapted to harsh environments is urgently needed. Summary of the Invention

[0004] This invention discloses a method for locating damage to concrete structures after a fire, aiming to solve the following technical problems: 1. Currently, 3D damage detection uses deep learning to directly perform damage detection and point cloud segmentation, which is computationally expensive and has low detection efficiency. This invention effectively decomposes the two tasks. First, it uses 2D segmentation to extract individual damage point cloud information, and then performs minimum bounding box regression on the point cloud information of the damage area, which effectively improves computational efficiency.

[0005] 2. Existing 3D damage detection technology relies on acquiring complete point cloud information. However, in complex engineering contexts, it is impossible to effectively acquire complete and valid point clouds. This invention proposes a multi-view point cloud information fusion method, which simultaneously solves the limitations of single-view detection and localization, and achieves global damage localization based on point cloud information.

[0006] 3. Current detection technologies are susceptible to various interferences such as lighting, noise, point clouds, and occlusion in complex environments, making it impossible to guarantee high-precision detection in degraded environments. Regarding applicability to engineering scenarios, existing detection technologies can only perform detection in the deployed environment and cannot achieve effective detection accuracy in other similar environments. This invention, through technological improvements, can effectively improve detection accuracy.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows: A method for locating damage to concrete structures after a fire includes the following steps: (1) Construct a multi-source dataset based on RGB images, depth images, and point cloud information; (2) Construct a semantic segmentation network based on YOLOv10-SimAM and a point cloud fusion network based on parallel structure; (3) Train the semantic segmentation network based on YOLOv10-SimAM constructed in step (2), and evaluate the performance of the semantic segmentation network based on YOLOv10-SimAM and the point cloud fusion network based on parallel structure. (4) Damage location of concrete structures after fire is achieved by using a semantic segmentation network based on YOLOv10-SimAM and a point cloud fusion network based on parallel structure.

[0008] Preferably, in step (1), the acquired RGB images are used for two-dimensional semantic segmentation feature extraction of concrete surface cracks and spalling damage under fire, and also serve as the data source for subsequent fusion of two-dimensional and multi-domain three-dimensional information; in the data preprocessing stage, the original RGB images are subjected to resolution unification and scale normalization processing, and combined with data augmentation strategies; wherein, the depth image data is acquired in two different ways, the first method is based on the structure self-motion algorithm, and depth estimation is performed through multi-view RGB images; the second method is to directly acquire depth images through a depth camera; wherein, the point cloud data is acquired using a depth camera and a lidar scanning device, and the acquisition methods include: one is a fixed-point shooting method, and the other is a continuous video recording shooting method.

[0009] Preferably, in step (2), while maintaining the efficient reasoning capability of YOLOv10, the SimAM attention mechanism is introduced to adaptively enhance the features of the damaged area; specifically: The YOLOv10-SimAM model consists of three parts: Backbone, Neck, and Head. The Backbone uses a multi-layer CBS and C3K2SimAM module for feature extraction. The C3K2SimAM module is composed of a parameterless SimAM attention module embedded in the YOLOv10 backbone network. The Neck part achieves multi-scale information interaction through a feature fusion structure, thereby improving the model's detection performance for damage targets at different scales. The Head part adopts a semantic segmentation head structure and outputs pixel-level damage segmentation results. The SimSPPF module is introduced to process high-level semantic features through multi-scale pooling operations, which enhances the ability to represent complex damage morphologies while ensuring computational efficiency, and finally outputs a two-dimensional semantic segmentation mask.

[0010] Preferably, in step (2), the method for constructing the point cloud fusion network based on the parallel structure is as follows: a fusion-aware feature extraction module is constructed based on the Transformer self-attention mechanism, and efficient feature learning is achieved by combining it with the PointNeXt encoder. Before feature extraction, a standardized preprocessing is performed, including point cloud centering and unit scale normalization. In the multi-view point cloud registration, feature fusion is first performed on the damage region segmentation point cloud under a single modality to construct an initial point cloud representation. Then, the point cloud is finely registered by the pose-constrained iterative nearest point algorithm to gradually form a globally consistent three-dimensional damage feature representation.

[0011] Preferably, in step (2), the point cloud fusion network based on the parallel structure adopts a local point cloud extraction strategy algorithm based on anchor frames.

[0012] The beneficial effects of the method for locating damage to concrete structures after a fire, as described in this invention, are as follows: 1. This invention constructs a parallel interactive fusion architecture of two-dimensional semantic segmentation and three-dimensional point cloud information, and utilizes the YOLOv10-SimAM network and Transformer sub-attention mechanism to realize bidirectional mapping of semantic features to geometric sampling, effectively solving the problem of damage feature failure caused by smoke interference, drastic changes in lighting and structural occlusion at fire scenes. 2. This invention adopts a local point cloud extraction strategy algorithm based on anchor frames, which avoids the high computational cost brought about by full-scale point cloud processing. While ensuring a real-time processing speed of 28 FPS, it significantly improves the average compactness of the three-dimensional damage area and realizes the anchor frame regression and lightweight processing method for irregular tilted damage areas. 3. This invention breaks through the limitation of traditional 2D vision, which can only obtain the damaged area. It calculates the depth profile, curvature characteristics and three-dimensional volume data of the damaged area through three-dimensional spatial coordinates, which facilitates the assessment of the residual bearing capacity of concrete structures after fire and the study of existing damaged structures. Attached Figure Description

[0013] Figure 1 Damage annotation using Labelme; Figure 2 RGB images; Figure 3 In COLMAP, the SfM algorithm is used to estimate the depth of images. Figure 4 Depth map acquired by L515 depth sensor; Figure 5 YOLOv10-SimAM network architecture design; Figure 6 A schematic diagram of the feature extraction module for heterogeneous point cloud fusion; Figure 7 Semantic segmentation performance metrics of the YOLOv10-SimAM network; Figure 8 Training and validation curves of the YOLOv10-SimAM network; Figure 9 , two Visualization of 3D damage semantic segmentation results; Figure 10 Visual comparison of point cloud fusion effects before and after; Figure 11 The fused 3D damage point cloud; Figure 12 , and the detection results of the three-dimensional anchor frame on the fused damage point cloud; Figure 13 A comparison chart of the segmentation performance of different models; Figure 14 A comparison of the segmentation performance of the model under different noise levels. Detailed Implementation

[0014] The following description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0015] Example 1: A method for locating damage to concrete structures after a fire includes the following steps: (1) Construct a multi-source dataset based on RGB images, depth images, and point cloud information; (2) Construct a semantic segmentation network based on YOLOv10-SimAM and a point cloud fusion network based on parallel structure; (3) Train the semantic segmentation network based on YOLOv10-SimAM constructed in step (2), and evaluate the performance of the semantic segmentation network based on YOLOv10-SimAM and the point cloud fusion network based on parallel structure. (4) Damage location of concrete structures after fire is achieved by using a semantic segmentation network based on YOLOv10-SimAM and a point cloud fusion network based on parallel structure.

[0016] Example 2: Based on Example 1, in step (1), the constructed dataset consists of three types of data: RGB images, depth images, and point cloud information. Preliminary tests are conducted on concrete cubes and rectangular test blocks of two different sizes. The ratio of cracks to spalling in the damage dataset is controlled at 4:5, and the training set:test set:validation set = 6:2:2. Data acquisition uses an R8SLAM laser scanning device, an L515 depth camera sensor, and a HUAWEI P40 mobile phone for close-range photography.

[0017] To mitigate the risk of overfitting, geometric transformations and color enhancement were employed to augment the data. Geometric transformations included random rotation, flipping, scaling, and translation to simulate changes in the target's position, orientation, and size. Subsequently, the Labelme tool was used to annotate the enhanced images, delineating damaged areas as closed shapes, and the corresponding annotation information was saved in ".txt" format. Damage illustrations for cracks and bursts after annotation are shown below. Figure 1 As shown. The processed dataset is finally converted into a format suitable for YOLO model training.

[0018] The RGB image data was acquired through close-up photography using a HUAWEI P40 smartphone as the image acquisition device. Multiple viewing angles were employed for the same test block. The dataset is available here. Figure 2 To minimize the impact of environmental factors on image quality, uniform and stable lighting conditions were maintained as much as possible during data acquisition, avoiding strong reflections and significant shadow interference. The acquired RGB images were primarily used for two-dimensional semantic segmentation feature extraction of concrete surface cracks and spalling damage under fire conditions, and also served as an important data source for subsequent fusion of two-dimensional and multi-domain three-dimensional information. In the data preprocessing stage, the original RGB images underwent resolution unification and scale normalization, and necessary data augmentation strategies were combined to improve the model's adaptability to different shooting angles, scale variations, and damage morphology diversity.

[0019] The depth map data is acquired using two different methods. The first method is based on the Structure from Motion (SfM) algorithm, which estimates depth using multi-view RGB images. This method utilizes camera pose information from different shooting angles to perform 3D reconstruction of the scene, thereby obtaining the corresponding depth information. The second method directly acquires depth images using a depth camera. This invention uses a RealSense L515 depth camera sensor to perform close-range scanning of the test block to directly acquire high-precision depth data. The depth information acquired by both methods is shown in [link to documentation]. Figure 3 , Figure 4 .

[0020] The two depth acquisition methods are complementary in terms of data sources and accuracy characteristics. The SfM-based depth estimation does not require additional hardware and is suitable for 3D information recovery under multi-view image conditions, while the depth camera-based method can provide more stable and dense depth measurement results, providing reliable data support for subsequent point cloud construction and multimodal information fusion.

[0021] The point cloud data was acquired using a RealSense L515 depth camera and a SLAM-R8 LiDAR scanner, as detailed in Table 1. Two main acquisition methods were used: fixed-point shooting and continuous video recording. The video sequences were segmented into frames to extract continuous, stable frames as data samples for depth maps and point cloud information.

[0022] Table 1 Comparison of Data Acquisition Equipment Models and Sensor Configurations: .

[0023] Example 3: Based on embodiments 1 and 2, the method for constructing the YOLOv10-SimAM semantic segmentation network architecture in step (2) includes the following: Global localization of damage point clouds is achieved by fusing 2D semantic mask information with 3D point cloud data. First, a YOLOv10-SimAM network is used to perform 2D damage detection and semantic segmentation on the acquired RGB images. Then, a binary mask generated from the segmentation results is used to spatially constrain the 3D point cloud. Through precise alignment between the 2D pixel coordinates and 3D spatial coordinates in the multi-source point cloud data, saliently characterized damage point clouds can be effectively extracted, thereby achieving 3D localization of concrete structure damage.

[0024] The overall performance of damage detection based on a 2D-3D fusion strategy largely depends on the performance of the 2D detection model. Therefore, YOLOv10 is selected as the benchmark and improved to enhance accuracy and computational efficiency. To achieve high-precision identification of concrete surface cracks and spalling damage under fire conditions, this invention constructs a semantic segmentation network based on YOLOv10-SimAM, as detailed below. Figure 5 While maintaining the efficient inference capabilities of YOLOv10, this network introduces the SimAM attention mechanism to adaptively enhance the features of damaged regions, thereby improving segmentation performance under complex background conditions.

[0025] In terms of network structure design, the model consists of three parts: Backbone, Neck, and Head. The Backbone employs multi-layer CBS and C3K2SimAM modules for feature extraction, embedding SimAM attention modules in key layers to enhance the representation of fine-scale damage features such as cracks and spalling. The Neck part achieves multi-scale information interaction through a feature fusion structure, thereby improving the model's detection performance for damage targets at different scales. The Head part adopts a semantic segmentation head structure, outputting pixel-level damage segmentation results. To further expand the network's receptive field, a SimSPPF module is introduced, processing high-level semantic features through multi-scale pooling operations, enhancing the representation ability of complex damage morphologies while ensuring computational efficiency. Finally, a high-quality two-dimensional semantic segmentation mask is output, providing reliable semantic constraints for subsequent two-dimensional and three-dimensional information fusion and damage point cloud extraction.

[0026] The construction method for converged network architecture design is as follows: Fusion networks are used to match 3D feature information from multiple sources and perspectives. Existing multi-source data fusion methods mainly include autonomous camera enhancement methods, dual-channel parallel fusion frameworks, and alternating interactive fusion schemes. However, in the process of heterogeneous point cloud fusion, problems such as differences in geometric distribution and inconsistencies in semantic representation seriously affect the stability and accuracy of the fusion results. To address these challenges, this invention focuses on the efficient fusion of multi-domain point clouds.

[0027] This invention designs a point cloud fusion network based on a parallel structure, as detailed below. Figure 6As shown, this method constructs a fusion-based perceptual feature extraction module based on the Transformer self-attention mechanism and combines it with a PointNeXt encoder to achieve efficient feature learning. In this framework, high-information-density point cloud features that are highly correlated with the damaged region are given higher attention weights, thereby effectively suppressing non-critical information regions and reducing information loss caused by invalid downsampling during feature extraction. Considering the common differences in point density distribution and coordinate systems among heterogeneous point cloud data, this invention performs standardization preprocessing before feature extraction, including point cloud centering and unit scale normalization. Through the above processing, the network can more effectively capture local geometric features within a unified scale and coordinate space, improving the stability and consistency of feature learning.

[0028] For multi-view point cloud registration, feature fusion is first performed on the point cloud segmented by the damage region under a single modality to construct an initial point cloud representation. Then, the point cloud is finely registered using the pose-constrained Iterative Closest Point (ICP) algorithm to gradually form a globally consistent three-dimensional damage feature representation, providing a foundation for subsequent damage localization and quantitative analysis.

[0029] Example 4: Based on embodiments 1, 2, and 3, step (3) is as follows: 4.1 Performance of the damage segmentation network: This invention focuses on two typical three-dimensional damage forms: cracks and spalling. Therefore, a two-dimensional semantic segmentation model is constructed to identify and classify these two types of damage. The correspondence between RGB pixel coordinates (u, v) and 3D point cloud coordinates (x, y, z) is established according to formula (1), where D(u, v) represents the depth value of each pixel, and K is the camera intrinsic parameter matrix. Through this mapping, the 3D points corresponding to the mask value of 1 are extracted to form a subset of the damage region, which is then registered as a global 3D damage point cloud.

[0030] (1); The YOLOv10-SimAM network is trained using the Adam optimizer to enhance its robustness in identifying damage features. To improve the stability of the training process and avoid overfitting, a dynamic learning rate adjustment strategy and an early stopping mechanism are introduced during training. The initial learning rate is set to 0.01 and decreases by orders of magnitude every 10 epochs during training, with a minimum threshold of 0.0001. The batch size is set to 500. The model performance is comprehensively evaluated using multiple metrics, including recall, precision, F1 score, and mAP@50, the specific definitions of which are given in formulas (2), (3), (4), and (5). In addition, to further evaluate the spatial consistency and segmentation accuracy of the semantic segmentation results, the mean Intersection over Union (mIoU) metric is used. Here, TP, FP, and FN represent the number of true positives, false positives, and false negatives, respectively, while N represents the number of classes, and AP represents the average precision of each class.

[0031] (2); (3); (4); (5); The performance metrics of the YOLOv10-SimAM model are as follows: Figure 7 As shown. Due to the poor quality of the dataset, the validation loss curve maintains a stable overall convergence trend. The training loss and validation loss are as follows: Figure 8 As shown, the improvements introduced in feature perception and information fusion significantly enhance the segmentation capability of the YOLOv10-SimAM model. Results show that the model achieves an mAP@50 of 0.806, with average accuracies of 0.853 and 0.759 for cracks and spalling, respectively. The F1 score curves show that the F1 score reaches its highest value of 0.73 at a confidence threshold of 0.299. These results confirm that the YOLOv10-SimAM network exhibits excellent performance in crack and spalling detection, thus providing reliable support for subsequent global 3D damage localization.

[0032] This invention quantizes the segmented damaged region by calculating the pixel area of ​​the binary mask region, and converts the pixel area into the actual surface area by combining the spatial resolution expressed in "pixels / millimeter". The calculation is performed according to formulas (6) and (7), where i represents the area of ​​the i-th segmented region, T is the area threshold, and R is the image resolution. Then, the 2D quantized area is compared with the 3D point cloud coordinate quantized value to analyze the size difference.

[0033] (6); (7).

[0034] To further enhance the interpretability of damage detection results, the binary semantic segmentation mask is converted into a heatmap representation, mapping the geometric features of the segmented region to a spatial intensity distribution. The generated color heatmap can intuitively reflect the spatial extent of the damage and its relative severity, with darker areas corresponding to higher damage levels, thus providing a verification basis for the three-dimensional damage quantification results. Figure 9 A representative visualization example of this method is provided.

[0035] 4.2 Blended point cloud effect: The damage concern point cloud results calculated by this invention are as follows: Figure 10 As shown in the figure. Based on this, heterogeneous point cloud data are fused and registered. First, the SfM algorithm is used to estimate the poses of multiple-view cameras, and preliminary registration of the point clouds is completed based on the camera pose information; then, the ICP algorithm is introduced to perform fine registration of the point clouds to further improve spatial alignment accuracy. Within this framework, the raw point cloud features from the RealSense L515 depth camera are interactively fused with the point cloud features generated from RGB-D data through a local attention mechanism. This process significantly enhances the feature representation ability of local damaged areas, allowing key geometric structural information to be more fully preserved. Finally, by fusing point cloud features from multiple frames and multiple views, a complete 3D reconstruction result of the damaged area is constructed, as shown in the figure. Figure 11 As shown.

[0036] 4.3 Three-dimensional detection of anchor frame positioning accuracy: Given that subsequent damage detection tasks require generating 3D anchor boxes for the damaged area and embedding their information into the overall point cloud representation, this invention introduces the aspect ratio of the anchor box as an evaluation metric. This parameter is defined as the ratio of the maximum to the minimum value of the Oriented Bounding Box (OBB) in the three principal axis directions, providing a quantitative description of the spatial scale and morphological characteristics of the damaged area. Compared to the traditional Axis-Aligned Bounding Box (AABB), the OBB can be rotated and aligned according to the actual spatial orientation of the local damaged area, offering significant advantages in terms of compactness and positioning accuracy. It is particularly suitable for damaged areas with irregular geometry or tilted distribution, providing more accurate 3D positioning and spatial representation capabilities.

[0037] To further improve the detection efficiency of downstream damage detection and quantification tasks, a clustering method is used to aggregate discrete crack point clouds into several connected regions. For each cluster, its corresponding bounding box is calculated and a 3D anchor box is automatically generated, such as... Figure 12As shown in the figure. Based on this, a preliminary three-dimensional quantitative analysis of the damaged area is performed. Open3D is used to reconstruct the damaged point cloud into a triangular mesh, and the α-shape algorithm is introduced to perform high-fidelity fitting of the complex geometry of the damaged surface. Finally, by accumulating the areas of each facet in the triangular mesh, the actual surface area of ​​each damaged area in three-dimensional space is obtained.

[0038] 4.4 3D point cloud quantization results: This invention verifies the effectiveness of the aforementioned semantic segmentation and 3D reconstruction results by comparing the 2D and 3D quantization results of the exfoliated damage region. Because the 2D segmentation mask is limited by a single planar viewpoint, its measurement results cannot reflect the true spatial morphology of the damage; therefore, the obtained area is usually smaller than the actual damage and the point cloud quantization results. As shown in Table 2, the difference between the 2D and 3D quantization results gradually increases with the increase of exfoliation depth. This is mainly because the change in the depth direction of the damage region introduces a more significant viewpoint bias.

[0039] Meanwhile, based on the quality evaluation indicators such as coverage, density uniformity and noise level defined by formulas (8), (9), (10), (11) and (12), the accuracy of the three-dimensional anchor frame generation results was further evaluated.

[0040] The coverage rate of a point cloud is defined as the proportion of points in the real point cloud that can be matched by corresponding points in the generated point cloud, given a distance threshold φ. Its calculation is shown in Equation (8). The higher the coverage rate, the better the reconstructed point cloud approximates the real point cloud in terms of overall distribution and density characteristics.

[0041] (8); Among them, P gen P represents the generation of point clouds. gt Let represent the true point cloud, ε represent the distance threshold, || represent the Euclidean distance, and || represent the number of points in the set.

[0042] Density uniformity is used to evaluate the spatial distribution characteristics of point clouds, focusing on analyzing whether there are localized over-clustering or sparse regions. This index is measured by calculating the squared Euclidean distance from each point in the point cloud to the global centroid, as shown in the following formula. A smaller value indicates a more uniform point cloud distribution and a more compact structure.

[0043] (9); Where |P| represents the number of points in the point set, and ||| represents the squared Euclidean distance from the point to the centroid of the point cloud.

[0044] Besides coverage and compactness, the overall quality of a point cloud is also closely related to the noise level in the reconstruction results. Noise is quantified by calculating the proportion of outliers in the point cloud. Specifically, the distance from each point to its local neighborhood fitting plane is first calculated, and then the Z-score method is used to statistically analyze these distance deviations. The calculation process is shown in the following formula: (10); (11); (12); in, Point The distance to its local neighborhood centroid, and d i Specifically, μ is defined as the average distance between point i and its K nearest neighbors. d and σ d Let Z represent the mean and standard deviation of this distance distribution in the entire point cloud, respectively. The normalized bias Z is calculated using the Z-score method. i When |Z i When the value is greater than 2, the point is identified as a noise point (outlier). The noise level of the point cloud is ultimately measured by the ratio of the number of noise points to the total number of points in the point cloud.

[0045] The results in Table 3 show that the generated anchor frames perform well in terms of compactness and coverage, and can effectively fit the actual damage area. Compared with the point cloud data without fusion processing, the fused point cloud significantly reduces the noise level without reducing the compactness and positioning accuracy of the anchor frames, verifying the effectiveness of the proposed fusion strategy in 3D damage detection and quantization tasks.

[0046] Table 2. Comparison of 2D and 3D quantization results of point cloud of exfoliation damage: .

[0047] Table 3. Performance evaluation indicators for 3D anchor frame detection: .

[0048] 4.5 Ablation Test: 4.5.1 Segmentation model ablation test: To evaluate segmentation performance, baseline networks, UNet, SegNet, and other representative architectures were applied to RGB data from the same source. Accuracy, recall, mAP@50, and mIoU were used as evaluation metrics.

[0049] Table 4 shows the segmentation results, indicating that YOLOv10n-SimAM achieved the best performance across all evaluation metrics, with accuracy, recall, mIoU, and mAP@50 of 82.4%, 80.1%, 80.6%, and 78.3%, respectively, while maintaining a frame processing speed of 28 FPS. In contrast, SE showed lower competitiveness, with scores of 71.0%, 68.5%, 69.5%, and 66.8% for the aforementioned metrics. Among all evaluated methods, TransUNet performed the worst, with accuracy, recall, mIoU, and mAP@50 of 62.0%, 59.3%, 58.5%, and 56.0%, respectively.

[0050] Table 4. Comparison of segmentation results from different semantic segmentation models: .

[0051] 4.5.2 Fusion module ablation test: In ablation experiments of the fusion module, the reliability of the proposed fusion method was verified using the public datasets 3DMatch, 3DLoMatch, and ScanNet++. The experimental results are shown in Table 5. Compared with Trans Fusion combined with ResNet-50, the proposed method improves the mAP@50 metric by 1.33%. This performance gain is attributed to the enhanced feature sampling process and the design of the parallel interaction mechanism. By reducing redundancy and mitigating interference in the feature transfer process, this strategy preserves the structural integrity of the image and point cloud representations, maintaining modal independence while ensuring efficient collaboration. Compared with Deep Interaction, which also employs a parallel interaction design, the proposed model achieves an additional 1.82% improvement in average accuracy. This advantage is mainly due to the introduction of the sub-attention mechanism, which removes unnecessary information in locally highly damaged regions, thereby reducing noise and stabilizing the training process. In summary, these results confirm the effectiveness of the proposed method in providing excellent feature fusion performance.

[0052] Table 5. Performance comparison of different fusion strategies on benchmark datasets: ; Note: "-" indicates that the modality was not used or that detailed information about the backbone network was not provided in the original paper.

[0053] 4.6 Comparison of segmentation effects of different networks: To evaluate segmentation performance, we employed several advanced methods, including YOLO variants, UNet, SegNet, and other representative architectures, and tested them on RGB data from the same source. Results were evaluated using four metrics: accuracy, recall, mIoU, and mAP@50.

[0054] Detection results are as follows Figure 13 As shown in Table 6, the quantitative results indicate that YOLOv10n-SimAM performed best across all evaluation metrics, with accuracy, recall, mIoU, and mAP@50 values ​​of 82.4%, 80.1%, 80.6%, and 78.3%, respectively, while maintaining a frame processing speed of 28 frames per second. In contrast, SE was less competitive, with values ​​of 71.0%, 68.5%, 69.5%, and 66.8% for the aforementioned metrics. TransUNet performed the worst among the evaluated methods, with recorded accuracy, recall, mIoU, and mAP@50 values ​​of 62.0%, 59.3%, 58.5%, and 56.0%, respectively. These results demonstrate that YOLOv10n-SimAM exhibits significant advantages in both segmentation accuracy and computational efficiency, highlighting its applicability for subsequent 3D damage detection tasks.

[0055] Table 6 Quantitative results of various segmentation models: .

[0056] 4.6 Public Dataset Experiment: For damage detection of concrete components, robustness to multi-source data is crucial. To evaluate the model's generalization ability, the detection performance was assessed on three public datasets: Crack500, CFD, and CrackLS315. Before training, the dataset was preprocessed using edge-symmetric padding and center cropping to obtain uniform pixel sizes. The corresponding annotation information was also simultaneously cropped, scaled, and binarized to ensure that the image content remained consistent with the marked damage features.

[0057] The training results are summarized in Table 7. On all three datasets, the mAP@50 value exceeded 70%, indicating that the YOLOv10-SimAM network still performs well in detection, confirming the robustness and reliability of this method in crack detection scenarios.

[0058] Table 7 Performance of YOLOv10-SimAM on public benchmark datasets: .

[0059] To validate the generalizability of the proposed method, further evaluation was conducted using public datasets related to earthquakes and structural damage, including PEER Hub ImageNet, Quake City, and SDNET2018. These datasets encompass various damage morphologies caused by earthquakes, such as oblique shear cracks and surface spalling. Quantitative results show that the YOLOv10-SimAM model performs consistently on earthquake datasets, achieving an mAP@50 of 73.6% on PEER Hub ImageNet and an average accuracy of 75.4% on SDNET2018. Even on the Quake City dataset, where environmental noise and debris are more prominent, the mAP@50 reaches 68.9%. This indicates that the feature extraction logic of the framework focuses on general structural discontinuities rather than damage caused by specific factors, thus demonstrating the potential of this method in broader applications such as structural health monitoring and multi-hazard response.

[0060] Table 8 shows the detection performance of YOLOv10-SimAM on the seismic dataset: .

[0061] 4.7 Robustness analysis of YOLOv10n-SimAM under harsh environmental conditions: To evaluate the reliability of the proposed segmentation model for damage detection in fire-affected scenarios, we conducted an evaluation under extreme environmental conditions. Fire scenes typically feature dust, structural obstruction, and variable lighting conditions. Therefore, three quantitative metrics were used to simulate the degradation of the original dataset: noise induced by point clouds (N... p ), shading rate (η) and light change rate (γ).

[0062] 4.7.1 Quantitative Indicators of Environmental Deterioration: This dataset employs three degradation methods: point cloud noise, occlusion, and variable lighting. The quantization criteria for these three degradation modes are determined by the following formula. Point cloud noise is used to simulate dust and smoke, achieved by injecting Gaussian noise into the image to simulate visual artifacts caused by suspended particles and sensor measurement errors. The noise level is defined as: (13); Where I(x, y) represents the original pixel intensity, and σ represents the noise standard deviation, which is directly related to the simulated dust density.

[0063] To simulate the situation where some structural components are obscured due to structural collapse after a fire, a random obscuration strategy was adopted. The obscuration rate (η) was calculated as follows: (14); Among them, Mi This represents the area of ​​a randomly generated black mask covering the damaged area, while A total This is the total area of ​​the identified damaged areas.

[0064] Because fire significantly reduces structural visibility, gamma correction was used to adjust light intensity to simulate low visibility conditions in a fire environment. The expression for its light attenuation model is defined as follows: (15); Where c is a constant and γ is a correction coefficient. When γ is greater than 1, it simulates a low-light environment, while when γ is less than 1, it simulates high-intensity lighting.

[0065] 4.7.2 Results and Analysis of Deterioration Experiment: To evaluate the specific impact of each factor, we conducted three independent sets of experiments. First, the segmentation performance of the proposed YOLOv10n-SimAM was compared with the baseline YOLOv10n, UNet, and SegNet at four different noise levels (σ=0.01, 0.04, 0.07, and 0.10). Under the minimum noise condition (σ=0.01), all models exhibited strong feature extraction capabilities; specifically, YOLOv10n-SimAM achieved an mAP of 80.9% and an mIoU of 76.1% at a 50% threshold, with a performance gap of only 1.6% compared to the baseline YOLOv10n. This indicates that the mainstream architecture can effectively capture the texture features of concrete damage under near-ideal environmental conditions. However, as the noise intensity increases, the performance of the baseline models significantly degrades due to high-frequency pixel fluctuations. At extreme noise levels (σ=0.10), the mIoU of traditional UNet and SegNet drops to 24.2% and 19.1%, respectively, leading to fragmented artifacts within the segmentation mask. Simultaneously, the mAP@50 of the benchmark YOLOv10n decreases to 38.2%. The significant contraction in recall observed in the radar plot indicates that random noise interferes with the fine geometric features of the cracks, resulting in severe missed detections. In contrast, YOLOv10n-SimAM demonstrates strong adaptability under degraded environmental conditions. Even under extreme interference of σ=0.10, this model maintains a detection performance of 54.5% mAP@50 and 48.2% mIoU.

[0066] To address the performance degradation typically caused by occlusion, a random occlusion strategy is employed to simulate complex real-world environments. This strategy occludes 10% to 50% of the total damaged area. Although monocular 2D computer vision has limitations in inferring the missing geometry of occluded regions, the proposed multi-view fusion module effectively overcomes this limitation. By stitching together the damaged point cloud through a spatial feature fusion mechanism, the framework reconstructs the entire damaged topology, effectively addressing the information constraints imposed by occlusion on traditional 2D vision tasks.

[0067] Finally, the impact of different light intensities on model performance was systematically analyzed. In fire-damaged environments, extreme lighting conditions severely weaken the contrast between concrete cracks and the background, leading to the disappearance of features in traditional segmentation architectures. As shown in Table 9, four lighting levels were established using gamma correction (γ=1.0, 2.0, 3.5, and 4.0), where γ=4.0 represents an extremely low-light environment.

[0068] Table 9 Comparison of segmentation performance under different light intensities: .

[0069] As the correction factor γ increases, the performance of the baseline model drops sharply. In the extreme case where γ reaches 4.0, the mIoU of UNet and SegNet drops to 18.6% and 12.4%, respectively, because the models cannot distinguish fine damage textures under low contrast conditions. While the baseline YOLOv10 also shows a significant decline, with its mAP@50 dropping to 25.1%, YOLOv10-SimAM exhibits superior robustness. Even in the darkest conditions, the proposed model maintains an mAP@50 of 51.3% and an mIoU of 49.5%. This stability is attributed to the SimAM attention mechanism, which effectively enhances relevant feature channels even under severe global pixel intensity suppression. As lighting conditions worsen, the performance gap between YOLOv10-SimAM and the baseline model widens from 1.4% to 26.2%.

Claims

1. A method for locating damage to concrete structures after a fire, characterized in that, Includes the following steps: (1) Construct a multi-source dataset based on RGB images, depth images, and point cloud information; (2) Construct a semantic segmentation network based on YOLOv10-SimAM and a point cloud fusion network based on parallel structure; (3) Train the semantic segmentation network based on YOLOv10-SimAM constructed in step (2), and evaluate the performance of the semantic segmentation network based on YOLOv10-SimAM and the point cloud fusion network based on parallel structure. (4) Damage location of concrete structures after fire is achieved by using a semantic segmentation network based on YOLOv10-SimAM and a point cloud fusion network based on parallel structure.

2. The method for locating damage to concrete structures after a fire as described in claim 1, characterized in that, In step (1), the acquired RGB images are used for two-dimensional semantic segmentation feature extraction of concrete surface cracks and spalling damage under fire, and also serve as the data source for subsequent fusion of two-dimensional and multi-domain three-dimensional information. In the data preprocessing stage, the original RGB images are subjected to resolution unification and scale normalization, and combined with data augmentation strategies. Among them, the depth image data is acquired in two different ways. The first method is based on the structure self-motion algorithm, which estimates the depth through multi-view RGB images. The second method is to directly acquire depth images through a depth camera. The point cloud data is acquired using a depth camera and a LiDAR scanning device. The acquisition methods include fixed-point shooting and continuous video recording.

3. The method for locating damage to concrete structures after a fire as described in claim 1, characterized in that, In step (2), while maintaining the efficient reasoning capabilities of YOLOv10, the SimAM attention mechanism is introduced to adaptively enhance the features of the damaged area; specifically: The YOLOv10-SimAM model consists of three parts: Backbone, Neck, and Head. The Backbone uses a multi-layer CBS and C3K2SimAM module for feature extraction. The C3K2SimAM module is composed of a parameterless SimAM attention module embedded in the YOLOv10 backbone network. The Neck part achieves multi-scale information interaction through a feature fusion structure, thereby improving the model's detection performance for damage targets at different scales. The Head part adopts a semantic segmentation head structure and outputs pixel-level damage segmentation results. The SimSPPF module is introduced to process high-level semantic features through multi-scale pooling operations, which enhances the ability to represent complex damage morphologies while ensuring computational efficiency, and finally outputs a two-dimensional semantic segmentation mask.

4. The method for locating damage to concrete structures after a fire as described in claim 1, characterized in that, In step (2), the construction method of the point cloud fusion network based on parallel structure is as follows: a fusion perception feature extraction module is constructed based on the Transformer self-attention mechanism, and efficient feature learning is achieved by combining the PointNeXt encoder. Before feature extraction, standardization preprocessing is performed, including point cloud centering and unit scale normalization. For multi-view point cloud registration, feature fusion is first performed on the point cloud segmentation of the damage region under a single modality to construct an initial point cloud representation; then, the point cloud is finely registered through the pose-constrained iterative nearest point algorithm to gradually form a globally consistent three-dimensional damage feature representation.

5. The method for locating damage to concrete structures after a fire as described in claim 1, characterized in that, In step (2), the point cloud fusion network based on parallel structure adopts a local point cloud extraction strategy algorithm based on anchor frames.

Images