A building damage rapid identification method and system based on end-cloud cooperation

Abstract

The application discloses a kind of based on end cloud cooperation's building damage fast identification method and system, method includes: obtaining historical building damage image dataset, based on the historical building damage image dataset, construct damage type identification model and crack fine segmentation model;Collect the field damage image of building component to be detected;The field damage image of the building component to be detected is respectively input into the damage type identification model and crack fine segmentation model, obtains damage area mask and crack binary mask;Based on the damage area mask and crack binary mask, obtain damage identification result.The application combines the side perception ability of intelligent mobile phone with deep learning model, can realize the full-process automation from damage type identification, crack pixel-level segmentation to geometric dimension measurement, improve the efficiency and precision of building damage detection.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary fields of intelligent construction, structural engineering and computer vision, and in particular relates to a method and system for rapid identification of building damage based on edge-cloud collaboration. Background Technology

[0002] In recent years, damage assessment of building structures during natural disasters has been a key issue in the field of building safety management. Especially after disasters such as earthquakes, floods, and storms, the ability to quickly and accurately assess building damage has become an urgent need to ensure the safety of people and property and to effectively carry out post-disaster reconstruction. Traditional building damage detection mainly relies on manual inspections, which is not only time-consuming and labor-intensive but also highly subjective, making it difficult to quantify the results and failing to meet the needs of rapid post-disaster assessment. To address this issue, in recent years, with the development of intelligent construction technology, technologies such as computer vision, image processing, and deep learning have been gradually applied to building damage detection, particularly in crack detection and geometric dimension measurement.

[0003] Traditional methods for building damage detection include visual inspection, sensor monitoring, and image processing-based automated detection. However, all these methods have certain limitations. Visual inspection relies on human judgment, resulting in low detection efficiency and highly subjective results. While sensor-based monitoring methods can provide more accurate real-time data, they typically cannot comprehensively cover all damaged areas of a building, and they are costly to install and difficult to maintain. Image processing-based automated detection methods often rely on target references or control points, but in complex environments, such as those affected by changes in lighting, cluttered backgrounds, and varying shooting angles, image stability is poor, making it difficult to perform stable geometric dimensional measurements.

[0004] Existing image processing and computer vision technologies still face multiple challenges in the field of building damage identification. First, the scarcity of high-quality datasets constitutes a research bottleneck. Current research focuses primarily on target detection, while studies on refined image segmentation are relatively limited; especially in crack detection, existing results are mostly concentrated on traffic roads, with insufficient application to complex building structures. Second, the generalization performance of existing models is limited, making it difficult to effectively cope with the diversity of building types and complex environmental interference, resulting in significant limitations in the rapid assessment of large-scale post-disaster building damage. Therefore, existing technical solutions still cannot fully meet the urgent needs of actual earthquake relief and disaster recovery in terms of timeliness and detection accuracy.

[0005] With the continuous improvement of smartphone hardware performance, especially breakthroughs in image processing, sensor technology (such as gyroscopes, accelerometers, GPS, etc.) and computing power, new technological paths have been provided for rapid post-disaster building inspection. Smartphones have become indispensable tools in people's daily lives, and their powerful hardware performance and widespread availability make them an important means of rapid post-disaster assessment. Utilizing deep learning models built into smartphones, functions such as crack detection, damage identification, and geometric measurement can be achieved on mobile devices, demonstrating strong environmental adaptability and application flexibility.

[0006] However, existing research largely focuses on damage identification and extraction, which falls under the "data acquisition" stage. From obtaining damage geometric parameters to generating the final "structural safety assessment report," a significant amount of manual calculation and professional logical judgment is still required. The inability to translate detection results into decision-making recommendations in real time leads to a clear technological gap between detection and assessment, resulting in insufficient level of intelligence. Summary of the Invention

[0007] To address the aforementioned technical challenges, this invention proposes a rapid building damage identification method and system based on edge-cloud collaboration. This invention combines the edge-side perception capabilities of smartphones with deep learning models, enabling fully automated processes from damage type identification and pixel-level crack segmentation to geometric dimension measurement, thus improving the efficiency and accuracy of building damage detection. Simultaneously, with the support of cloud computing and large language models, real-time data upload and structural safety assessment are achieved, forming a complete collaborative working mode of rapid edge-side detection and intelligent cloud-based assessment.

[0008] To achieve the above objectives, this invention provides a method for rapid building damage identification based on edge-cloud collaboration, comprising: Obtain a dataset of images showing damage to historical buildings, and based on this dataset, construct a damage type identification model and a fine crack segmentation model. Collect on-site damage images of the building components to be inspected; The on-site damage images of the building components to be detected are input into the damage type recognition model and the crack fine segmentation model respectively to obtain the damage area mask and the crack binary mask; Based on the damage area mask and the crack binary mask, the damage identification result is obtained.

[0009] Optionally, acquiring a dataset of historical building damage images, and constructing a damage type identification model and a fine crack segmentation model based on the historical building damage image dataset, includes: Acquire a dataset of historical building damage images, the dataset including images of surface damage to building components and their corresponding damage annotation files, the damage annotation files including at least one damage category annotation among cracks, spalling, exposed rebar, and breakage; The dataset of images showing damage to historical buildings is preprocessed. The preprocessed damage image is used as input, and the multi-class damage annotation map is used as the supervision label to train the initial damage type recognition model. The crack binary mask is used as the supervision label to train the initial crack fine segmentation model. The trained damage type identification model and crack fine segmentation model are lightweighted to obtain the damage type identification model and crack fine segmentation model.

[0010] Optionally, acquiring on-site damage images of the building components to be inspected also includes: Component information is recorded synchronously, and a reference straight line of known actual length is placed in the on-site damage image; The conversion ratio between pixel size and actual physical size is calculated based on the actual length of the reference line and the pixel length.

[0011] Optionally, the on-site damage image of the building component to be detected is input into the damage type recognition model and the crack fine segmentation model, respectively, to obtain the damage area mask and the crack binary mask, including: The on-site damage image is adjusted to the preset input size of the model and pixel normalization is performed. The damage type recognition model is used to infer the normalized image and output the probability of each pixel belonging to each damage category. The category with the highest probability is taken as the predicted damage category of the corresponding pixel, and a damage region mask under a preset input size is obtained. The crack fine segmentation model is used to infer the normalized image and output a crack binary prediction mask. The damaged area mask and the crack binary mask are inversely transformed according to the size adjustment parameters, and the mask is restored to the original image size by nearest neighbor interpolation.

[0012] Optionally, the identification method further includes: Based on the damage identification results, a retrieval query is constructed, and the damage assessment knowledge base is invoked to perform similar text retrieval and rule matching to obtain a knowledge text set and an assessment rule set. Based on the set of evaluation rules, rule matching and threshold judgment are performed to generate preliminary damage assessment results. The preliminary damage assessment results, knowledge text set and preset output format are then input into the large language model to generate a damage assessment report. The damage assessment report is displayed in conjunction with the on-site damage images, damage area masks, crack binary masks, and damage quantification results.

[0013] Optionally, generating a damage assessment report includes: The damage identification results are processed to form a damage assessment feature vector that includes component type, damage category, damage location, crack width, crack length, damage area, and model identification confidence. Based on the damage assessment feature vector, a retrieval query information is generated, and the damage assessment knowledge base is called to perform similar text retrieval and rule matching to obtain a set of knowledge texts and a set of assessment rules corresponding to the current component type, damage category and damage quantification index. Based on the damage assessment feature vector and the set of assessment rules, rule matching and threshold judgment are performed to obtain the preliminary damage assessment result; The preliminary damage assessment results, knowledge text set, and preset output format are used to construct the report generation input information. This information is then input into a large language model for text organization, clause summarization, and report formatting to generate a damage assessment report. The large language model does not independently change the damage level, damage quantification indicators, and manual review markers obtained from rule matching and threshold judgment.

[0014] Optionally, rule matching and threshold determination based on the damage assessment feature vector and assessment rule set include: When the model identification confidence is not lower than the preset confidence threshold and the evaluation rule matching degree is not lower than the preset matching threshold, a preliminary damage assessment result is generated, which includes damage level, main damage type, damage quantification index, rule matching basis, risk warning, manual review suggestion and repair and reinforcement suggestion. When the model's confidence level is lower than the preset confidence threshold or the evaluation rule's matching degree is lower than the preset matching threshold, a mark indicating manual review is required and corresponding review suggestions are output.

[0015] This invention also provides a rapid building damage identification system based on edge-cloud collaboration, comprising: a model building module, an image acquisition module, an image processing module, and a result output module; The model building module is used to acquire a dataset of historical building damage images and, based on the dataset, construct a damage type identification model and a crack fine segmentation model. The image acquisition module is used to acquire on-site damage images of the building components to be inspected; The image processing module is used to input the on-site damage image of the building component to be detected into the damage type recognition model and the crack fine segmentation model respectively, and obtain the damage area mask and the crack binary mask. The result output module is used to obtain damage identification results based on the damage area mask and the crack binary mask.

[0016] Optionally, the system further includes: a damage assessment module and a damage visualization module; The damage assessment module is used to construct retrieval query information based on the damage identification results, call the damage assessment knowledge base to perform similar text retrieval and rule matching to obtain a knowledge text set and an assessment rule set; perform rule matching and threshold judgment based on the assessment rule set to generate a preliminary damage assessment result, and input the preliminary damage assessment result, the knowledge text set and the preset output format into the large language model to generate a damage assessment report; The damage visualization module is used to associate and display the damage assessment report with the on-site damage image, damage area mask, crack binary mask, and damage quantification results.

[0017] Compared with the prior art, the present invention has the following advantages and technical effects: First, mobile devices can complete damage identification, crack segmentation, and geometric parameter quantification on-site, no longer relying solely on the cloud to complete all image recognition tasks, thus improving on-site response speed. Second, the system converts pixel-level recognition results into actual physical dimensions through size calibration parameters, and can output quantitative parameters such as crack length, crack width, and damage area, making the detection results more suitable for engineering evaluation. Third, the cloud-based damage assessment center combines the damage assessment knowledge base and the big language model to generate damage levels, risk warnings, and repair and reinforcement suggestions, enabling the image recognition results to be further transformed into assessment conclusions with engineering significance. Fourth, the system forms a closed-loop process of "mobile terminal on-site data collection - terminal-side identification and quantification - cloud-based knowledge enhancement assessment - mobile terminal result display", which can be used for rapid screening of post-disaster building damage and emergency decision support. Attached Figure Description

[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a method for rapid building damage identification based on edge-cloud collaboration according to an embodiment of the present invention. Detailed Implementation

[0019] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0021] This embodiment proposes a rapid building damage identification method based on edge-cloud collaboration, such as... Figure 1 As shown, the specific steps include: Obtain a dataset of images showing damage to historical buildings, and based on this dataset, construct a damage type identification model and a fine crack segmentation model. Collect on-site damage images of the building components to be inspected; The on-site damage images of the building components to be detected are input into the damage type recognition model and the crack fine segmentation model respectively to obtain the damage area mask and the crack binary mask; Based on the damage area mask and the crack binary mask, the damage identification result is obtained.

[0022] Furthermore, a dataset of historical building damage images is obtained, and based on this dataset, a damage type identification model and a fine crack segmentation model are constructed, including: Acquire a dataset of historical building damage images, the dataset including images of surface damage to building components and their corresponding damage annotation files, the damage annotation files including at least one damage category annotation among cracks, spalling, exposed rebar, and breakage; The dataset of images showing damage to historical buildings is preprocessed. The preprocessed damage image is used as input, and the multi-class damage annotation map is used as the supervision label to train the initial damage type recognition model. The crack binary mask is used as the supervision label to train the initial crack fine segmentation model. The trained damage type identification model and crack fine segmentation model are lightweighted to obtain the damage type identification model and crack fine segmentation model.

[0023] Specifically, the model training and update center acquires existing building damage image datasets, which include surface damage images of building components and their corresponding damage annotation files. The damage annotation files include at least one or more damage categories such as cracks, spalling, exposed rebar, and breakage.

[0024] The dataset undergoes format standardization, size adjustment, pixel normalization, and category mapping. The size adjustment involves uniformly resizing the input images to the model's preset input size. Let the original image size be... The model's preset input size is ,in The size can be selected as 128, 256, 512, or other preset values ​​depending on the model structure and mobile computing resources. The size adjustment process is represented as follows: ; If direct scaling is used, the horizontal and vertical scaling factors are: ; The pixel coordinate mapping relationship is as follows: ; in, For the original image, For the adjusted image, These are the pixel coordinates of the original image. These are the adjusted pixel coordinates. The original damage image is resized using bilinear interpolation to maintain the continuity of image grayscale or color information; the damage annotation mask is resized using nearest neighbor interpolation to avoid changes in category numbers during scaling.

[0025] The resized image is then subjected to pixel normalization, mapping the RGB three-channel pixel values ​​from [0, 255] to [0, 1]. The calculation method is as follows: ; in, These are the pixel values ​​of the resized image. These are the normalized pixel values.

[0026] The category mapping unifies the damage names in different datasets to preset damage category numbers and generates a multi-category damage annotation map for training the first model. Let the damage category set be: ; Where 0 represents the background category, and 1 to These represent damage categories such as cracks, spalling, exposed reinforcement, and breakage. For any pixel in the image... Its multi-category damage annotation values ​​are expressed as follows: ; in, Represents pixels Damage category label value at the location, Indicates the first The pixel region corresponding to the type of damage.

[0027] The first model takes the original damage image as input and multi-class damage annotation map as input. The labels are used as supervisory labels during training to output the damage category corresponding to each pixel. During model inference, the first model outputs the probability that each pixel belongs to each damage category, and the final predicted category is represented as: ; in, Represents pixels Belongs to the The probability of damage type, This represents the damage category predicted by the first model under the model input dimensions.

[0028] The first model is used for multi-class damage recognition, and its loss function adopts multi-class cross-entropy loss, expressed as: ; in, This represents the loss function of the first model. Represents a set of damage categories. Represents pixels Belongs to the Predicted probability of damage type This indicates the corresponding true category label.

[0029] For fine-grained crack segmentation tasks, crack annotations are converted into binary masks: ; in, Represents pixels The binary annotation value of the crack at that location, This indicates the cracked area.

[0030] The second model takes a damage image containing cracks or a crack candidate region as input, and uses a crack binary mask. The labels are used as supervisory labels for training, which are then used to output a binary crack prediction mask. .

[0031] The second model is used for fine-grained crack segmentation. Its loss function is a combination of binary cross-entropy loss and Dice loss, expressed as: ; in, This represents the loss function of the second model. Used to constrain the classification results of cracked pixels and non-cracked pixels. Used to improve the segmentation continuity of slender crack regions. These are the weighting coefficients.

[0032] The model input image is of the original size Adjust to preset input size The damage mask size output by the model is also... To ensure that the recognition results correspond to the original image locations, the system performs an inverse transformation on the output mask after inference based on the size adjustment parameters, restoring the damage area mask output by the first model and the crack binary mask output by the second model to the original image size. The process is represented as follows: ; ; in, and These are the preset-size damage mask and crack binary mask output by the model, respectively. and These are the restored original-size damage mask and the binary crack mask, respectively. This indicates an inverse size transformation operation. Nearest neighbor interpolation is used during mask restoration to maintain the category number.

[0033] After the model training is completed, the Model Training and Update Center performs lightweight processing on the first and second models to obtain the lightweight first model. Second Model The process is represented as follows: ; in, This represents the damage type identification model after training. This represents the fine-grained crack segmentation model after training. and These represent the lightweighted damage type identification model and the fine crack segmentation model, respectively. This indicates a lightweight processing operation, which includes at least one of parameter quantization, model pruning, model distillation, input size compression, or network structure simplification.

[0034] Furthermore, acquiring on-site damage images of the building components to be inspected also includes: Component information is recorded synchronously, and a reference straight line of known actual length is placed in the on-site damage image; The conversion ratio between pixel size and actual physical size is calculated based on the actual length of the reference line and the pixel length.

[0035] Specifically, the mobile damage acquisition and recognition terminal acquires on-site damage images of the building components to be inspected, and simultaneously records the building number, floor, component type, component location, and textual description information. The on-site damage image is a visible light image containing the damage area to be inspected, and the damage area to be inspected includes at least one of cracks, spalling, exposed reinforcement, and breakage.

[0036] To achieve the conversion between image pixel size and actual physical size, a reference straight line of known actual length is placed near the area of ​​damage to be detected during image acquisition. The reference straight line can be a ruler, measuring rod, or other linear reference object with a known length, and it and the area of ​​damage to be detected are located in the same imaging plane to reduce the impact of perspective distortion on the size conversion results.

[0037] Let the actual length of the reference line be... The pixel coordinates of its two endpoints in the on-site damage image are respectively and Then refer to the pixel length of the line in the image. Represented as: ; Based on the actual length of the reference line and pixel length Calculate the conversion ratio between image pixel size and actual physical size. : ; in, This indicates the actual length of a single pixel, expressed in mm / pixel or cm / pixel.

[0038] For any pixel distance in the image Its corresponding actual length Represented as: ; For the image containing The actual area corresponding to the damage region composed of pixels. Represented as: ; Thus, the mobile damage acquisition and identification terminal obtains the size calibration parameters. The dimensional calibration parameters are saved together with the on-site damage images and component information for subsequent calculations of crack length, crack width, spalling area, and exposed rebar area.

[0039] Further, the on-site damage image of the building component to be detected is input into the damage type recognition model and the crack fine segmentation model, respectively, to obtain the damage area mask and the crack binary mask, including: The on-site damage image is adjusted to the preset input size of the model and pixel normalization is performed. The damage type recognition model is used to infer the normalized image and output the probability of each pixel belonging to each damage category. The category with the highest probability is taken as the predicted damage category of the corresponding pixel, and a damage region mask under a preset input size is obtained. The crack fine segmentation model is used to infer the normalized image and output a crack binary prediction mask. The damaged area mask and the crack binary mask are inversely transformed according to the size adjustment parameters, and the mask is restored to the original image size by nearest neighbor interpolation.

[0040] Specifically, the mobile damage acquisition and recognition terminal acquires the damage area mask output from the first model and restores it to the original image size. And the second model output and the crack binary mask restored to the original image size. .in, Pixel regions used to represent different damage categories such as cracks, spalling, exposed reinforcement, and breakage. The pixel location used to represent the crack area.

[0041] For fine-grained crack segmentation results, the system is based on a crack binary mask. Skeleton extraction was performed to obtain a set of crack skeletons. The crack skeleton set is used to represent the location of the crack centerline. Let the pixel distance between adjacent skeleton pixels in the crack skeleton be... Then the crack pixel length Represented as: ; in, This represents the number of pixels in the crack skeleton. For the first The skeleton pixel point and the first The pixel distance between skeleton pixels; when adjacent pixels are horizontally or vertically adjacent. When adjacent pixels are diagonally adjacent, .

[0042] Based on the conversion ratio coefficient between pixel size and actual physical size obtained in step 2 Actual length of the crack Represented as: ; in, This is the actual length of the crack. This represents the actual length corresponding to a single pixel.

[0043] For crack width, the system is based on a crack binary mask. Perform boundary detection and calculate the distance from the skeleton pixel to the crack boundary. Let the... The pixel distance from each crack skeleton pixel to the crack boundary is The width of the crack at that location is... Represented as: ; Average crack width and maximum width They are represented as follows: ; ; in, The average width of the crack. This represents the maximum width of the crack.

[0044] For the crack area, the system statistically analyzes the binary crack mask. Number of pixels in the crack. Let the number of pixels in the crack region be... The actual area of ​​the crack Represented as: ; in, This represents the actual area of ​​the crack. This represents the number of pixels in the crack area.

[0045] For planar damage such as peeling, exposed reinforcement, and breakage, the system uses a mask to represent the damaged area. Extract the pixel region corresponding to the category. Let the first... The set of pixels in the damage-like region is Its pixel count is Then the first Actual area of ​​the damage Represented as: ; in, , For a preset set of damage categories, For the first The actual area of ​​the damage.

[0046] The system further performs connectivity analysis on various types of damage regions, dividing multiple independent damage regions of the same damage category into several connected components. Let the first... Class of damage The connected components are Its pixel count is Then the actual area of ​​the connected component is... Represented as: ; For the Class of damage From a connected region, the system can also extract its perimeter, circumscribed rectangle size, and shape factor. Let the actual area of ​​this connected region be... The actual perimeter is Then the shape factor Represented as: ; in, Used to characterize the regularity of the shape of the damaged area.

[0047] After the above processing, the mobile damage acquisition and identification terminal generates a structured damage quantification result, which is then encapsulated together with the on-site damage image, component information, and dimensional calibration parameters into a structured damage data packet. The structured damage data packet can be represented as follows: ; in, Represents structured corrupted data packets. Indicates the image of the damage at the scene or the image number. Indicates basic building information, Indicates component information, Indicates the dimensional calibration parameters. This indicates the damage identification results. This indicates the results of damage quantification.

[0048] The damage identification results The damage quantification result includes at least one of the following: damage category, damage location, and model confidence level. The data package includes at least one of the following: crack length, average crack width, maximum crack width, crack area, spalling area, exposed rebar area, damaged area, number of damaged regions, connected region area, connected region perimeter, and shape factor. The structured damage data package is uploaded to the cloud-based damage assessment center via an edge-cloud communication collaboration module, serving as input data for subsequent standard knowledge retrieval, large language model evaluation, and report generation.

[0049] Furthermore, the identification method also includes: Based on the damage identification results, a retrieval query is constructed, and the damage assessment knowledge base is invoked to perform similar text retrieval and rule matching to obtain a knowledge text set and an assessment rule set. Based on the set of evaluation rules, rule matching and threshold judgment are performed to generate preliminary damage assessment results. The preliminary damage assessment results, knowledge text set and preset output format are then input into the large language model to generate a damage assessment report. The damage assessment report is displayed in conjunction with the on-site damage images, damage area masks, crack binary masks, and damage quantification results.

[0050] Specifically, the mobile damage acquisition and identification terminal uploads the structured damage data package obtained in step 3 to the cloud-based damage assessment center via the end-to-cloud communication collaboration module. The structured damage data package includes at least one of the following: on-site damage image or image number, building information, component information, dimensional calibration parameters, damage identification results, and damage quantification results, and can be represented as: ; in, Represents structured corrupted data packets. Indicates the image of the damage at the scene or the image number. Indicates building information, Indicates component information, Indicates the dimensional calibration parameters. This indicates the damage identification results. This indicates the results of damage quantification.

[0051] After receiving the structured damage data packet, the cloud-based damage assessment center first processes the damage identification and quantification results to form a damage assessment feature vector: ; in, Indicates the component type. Indicates the type of damage. Indicates the location of the injury. Indicates the crack width or damage feature size. Indicates the crack length. Indicates the area of ​​damage. This represents the confidence level of the model's recognition.

[0052] The cloud-based damage assessment center generates retrieval query information based on the damage assessment feature vector: ; in, This represents the retrieval query information generated from the damage assessment feature vector. This indicates an operation to build query information.

[0053] The cloud-based damage assessment center accesses the damage assessment knowledge base to retrieve information. Similar text retrieval and rule matching are performed to obtain a set of knowledge texts and a set of evaluation rules corresponding to the current component type, damage category, and damage quantification index. The set of knowledge texts is represented as follows: ; in, Representing and retrieving query information Related knowledge text collections, Indicates the first The retrieved text fragments.

[0054] The set of evaluation rules is represented as follows: ; in, This represents the set of evaluation rules that match the current damage state. Indicates the first The assessment rules include at least one of the following: component type conditions, damage category conditions, damage quantification index thresholds, risk level mapping relationships, repair and reinforcement recommendations, and corresponding basis.

[0055] The cloud-based damage assessment center uses damage assessment feature vectors. and evaluation rule set Perform rule matching and threshold judgment to generate preliminary damage assessment results: ; in, This indicates a damage assessment operation based on rule matching and threshold judgment. This indicates the preliminary damage assessment results.

[0056] The preliminary damage assessment results It includes at least one of the following: damage level, main damage type, damage quantification index, rule matching basis, risk warning, manual review suggestion, and repair and reinforcement suggestion.

[0057] When the model's confidence level is lower than a preset confidence threshold, or when the matching degree between the retrieved set of assessment rules and the current damage state is lower than a preset matching threshold, the cloud-based damage assessment center does not output a definitive damage level. Instead, it outputs a "pending manual review" label and corresponding review suggestions. This process can be represented as: ; in, This indicates a pre-set confidence threshold. Indicates the degree of matching of the evaluation rules. This indicates the preset matching threshold. This indicates that the result is pending manual review.

[0058] Subsequently, the cloud-based damage assessment center will provide the preliminary damage assessment results. Knowledge text collection Together with the preset output format, they constitute the input information for report generation: ; in, This indicates the input information for generating the report. This indicates a preset output format. The preset output format includes at least one of the following: damage level, damage quantification index, applicable clauses, risk warning, review suggestions, and repair and reinforcement suggestions.

[0059] The large language model generates input information based on the report. The preliminary damage assessment results are organized into text, summarized, and formatted into a report to generate a damage assessment report. ; in, This indicates the report generation operation. This refers to the damage assessment report generated in the cloud. The large language model does not independently change the damage level, damage quantification indicators, and manual review markers obtained from rule matching and threshold judgment.

[0060] The cloud-based damage assessment center transmits the damage assessment report back to the mobile damage acquisition and identification terminal via the end-to-cloud communication collaboration module. The mobile terminal displays the report in at least one of the following formats: text report, parameter table, risk level, review prompt, and image overlay results.

[0061] Furthermore, generating a damage assessment report includes: The damage identification results are processed to form a damage assessment feature vector that includes component type, damage category, damage location, crack width, crack length, damage area, and model identification confidence. Based on the damage assessment feature vector, a retrieval query information is generated, and the damage assessment knowledge base is called to perform similar text retrieval and rule matching to obtain a set of knowledge texts and a set of assessment rules corresponding to the current component type, damage category and damage quantification index. Based on the damage assessment feature vector and the set of assessment rules, rule matching and threshold judgment are performed to obtain the preliminary damage assessment result; The preliminary damage assessment results, knowledge text set, and preset output format are used to construct the report generation input information. This information is then input into a large language model for text organization, clause summarization, and report formatting to generate a damage assessment report. The large language model does not independently change the damage level, damage quantification indicators, and manual review markers obtained from rule matching and threshold judgment.

[0062] Furthermore, the rule matching and threshold determination based on the damage assessment feature vector and assessment rule set include: When the model identification confidence is not lower than the preset confidence threshold and the evaluation rule matching degree is not lower than the preset matching threshold, a preliminary damage assessment result is generated, which includes damage level, main damage type, damage quantification index, rule matching basis, risk warning, manual review suggestion and repair and reinforcement suggestion. When the model's confidence level is lower than the preset confidence threshold or the evaluation rule's matching degree is lower than the preset matching threshold, a mark indicating manual review is required and corresponding review suggestions are output.

[0063] Specifically, the cloud-based damage assessment center generates damage assessment results. The damage assessment results are then transmitted back to the mobile damage acquisition and identification terminal via the edge-cloud communication collaboration module. It includes at least one of the following: damage level, main damage type, damage quantification indicators, risk warning, manual review recommendations, repair and reinforcement recommendations, and report text.

[0064] After receiving the damage assessment results, the mobile damage acquisition and recognition terminal associates and displays them with the on-site damage image, damage area mask, size calibration parameters, and damage quantification results. The association and display methods include at least one of the following: image mask overlay display, damage parameter table display, risk level prompts, text assessment report display, and inspection record archiving.

[0065] In one implementation, a mobile damage acquisition and identification terminal generates a standardized building damage assessment report based on on-site damage images, component information, dimensional calibration parameters, damage identification results, damage quantification results, and cloud-based damage assessment results. The standardized building damage assessment report includes at least one of the following: basic building information, component information, damage location, damage category, crack length, crack width, damage area, damage level, risk description, and treatment recommendations.

[0066] Through the above processing, the system achieves a closed loop for rapid building damage assessment, consisting of "mobile-side on-site data collection, end-side damage identification, damage parameter quantification, cloud-based knowledge-enhanced assessment, and mobile-side result display."

[0067] Furthermore, the model training and update center updates and trains the damage type identification model and the crack fine segmentation model based on newly added building damage images, manual review results, or historical assessment data. After being lightweighted, the updated model is distributed to the mobile damage acquisition and identification terminal via the edge-cloud communication collaboration module to improve the adaptability of the mobile model to different building types, shooting environments, and damage morphologies.

[0068] Furthermore, the edge-cloud communication collaboration module allocates tasks between the mobile device and the cloud based on the mobile device's computing power, network status, and the complexity of the assessment task. When the mobile device meets the local inference conditions, damage identification, crack segmentation, size calibration, and parameter quantization are completed on the mobile device. When the mobile device's computing power is insufficient, the network status is good, or the assessment task is complex, the structured damage data is uploaded to the cloud-based damage assessment center for supplementary analysis and report generation. This ensures the real-time nature of on-site detection while improving the completeness and stability of the damage assessment results.

[0069] Furthermore, the model lightweighting process includes at least one of model pruning, parameter quantization, model distillation, input size compression, and network structure simplification, which are used to reduce the number of model parameters and inference computation, enabling the mobile damage acquisition and identification terminal to complete rapid on-site identification in the field environment.

[0070] This invention also provides a rapid building damage identification system based on edge-cloud collaboration, comprising: a model building module, an image acquisition module, an image processing module, and a result output module; The model building module is used to acquire a dataset of historical building damage images and, based on the dataset, construct a damage type identification model and a crack fine segmentation model. The image acquisition module is used to acquire on-site damage images of the building components to be inspected; The image processing module is used to input the on-site damage image of the building component to be detected into the damage type recognition model and the crack fine segmentation model respectively, and obtain the damage area mask and the crack binary mask. The result output module is used to obtain damage identification results based on the damage area mask and the crack binary mask.

[0071] Furthermore, the system also includes: a damage assessment module and a damage visualization module; The damage assessment module is used to construct retrieval query information based on the damage identification results, call the damage assessment knowledge base to perform similar text retrieval and rule matching to obtain a knowledge text set and an assessment rule set; perform rule matching and threshold judgment based on the assessment rule set to generate a preliminary damage assessment result, and input the preliminary damage assessment result, the knowledge text set and the preset output format into the large language model to generate a damage assessment report; The damage visualization module is used to associate and display the damage assessment report with the on-site damage image, damage area mask, crack binary mask, and damage quantification results.

[0072] Specifically, the system completes the acquisition of building damage images, damage identification, size calibration and geometric parameter quantification through mobile devices, and completes the retrieval of damage assessment knowledge base, large language model-assisted assessment and assessment report generation through the cloud, thereby realizing rapid detection, quantitative assessment and result feedback of building damage.

[0073] The system includes a model training and update center, a mobile damage acquisition and identification terminal, an edge-cloud communication collaboration module, and a cloud-based damage assessment center. The model training and update center, the mobile damage acquisition and identification terminal, and the cloud-based damage assessment center are all connected via the edge-cloud communication collaboration module.

[0074] The model training and update center is used to train the damage type recognition model and the crack fine segmentation model, and then distributes the lightweight model to the mobile damage acquisition and recognition terminal. The mobile damage acquisition and recognition terminal collects building damage images and component information, calls the lightweight model to complete damage identification, crack segmentation, size calibration, and damage parameter quantification, and generates structured damage data. The cloud-based damage assessment center receives the structured damage data and generates damage assessment results by combining the damage assessment knowledge base and a large language model. The end-to-cloud communication and collaboration module is used to realize model distribution, data upload, task invocation, and result feedback.

[0075] Through the above system architecture, the mobile terminal is responsible for on-site damage collection, identification, and quantification, while the cloud terminal is responsible for model updates, knowledge retrieval, comprehensive evaluation, and report generation, thus forming a closed loop of end-to-cloud collaboration: "cloud model update - mobile terminal on-site identification - cloud terminal knowledge enhancement evaluation - mobile terminal result display".

[0076] This embodiment uses the rapid assessment of wall damage in residential buildings after an earthquake as an example to illustrate in detail: 1. System Deployment: In this embodiment, the rapid building damage assessment system based on edge-cloud collaboration includes a model training and update center, a mobile damage acquisition and identification terminal, an edge-cloud communication collaboration module, and a cloud-based damage assessment center.

[0077] The model training and update center is deployed on a cloud server to acquire existing building damage image datasets and train damage type recognition models and crack fine segmentation models. After training, both models are lightweighted to obtain a lightweight damage type recognition model. and fine segmentation model of cracks It is then transmitted to the mobile damage acquisition and identification terminal via the end-to-cloud communication collaboration module.

[0078] The mobile damage acquisition and identification terminal is deployed on a smartphone or tablet to acquire on-site damage images, input component information, call a lightweight model for end-side identification, and calculate quantitative parameters such as crack length, crack width, and damage area.

[0079] The cloud-based damage assessment center is deployed on a cloud server and includes a damage assessment knowledge base, a large language model, and an assessment result generation module. The damage assessment knowledge base pre-stores building damage assessment standards, post-disaster building safety assessment standards, repair and reinforcement technical documents, and engineering experience texts. The cloud-based damage assessment center receives structured damage data packets uploaded from mobile devices and combines the knowledge base retrieval results with the large language model to generate damage levels, risk warnings, and repair and reinforcement recommendations.

[0080] The edge-cloud communication and collaboration module is used to complete the distribution of lightweight models, the uploading of structured damage data, the invocation of cloud assessment tasks, and the feedback of assessment results.

[0081] Among them, APP is short for Application, which means an application installed on a mobile terminal; LLM is short for Large Language Model; RAG is short for Retrieval-Augmented Generation; JSON is short for JavaScript Object Notation; and RGB represents a three-channel color image of red, green, and blue.

[0082] 2. On-site image acquisition and dimension calibration: During post-earthquake inspections, inspectors carried smartphones equipped with the aforementioned mobile damage acquisition and identification terminal into the buildings to be inspected. Taking the third-floor infill wall of a residential building as an example, the inspectors selected the inspection task on their mobile devices and entered the building number, floor, component type, component location, and textual description information.

[0083] For example, the detection records include: Building Number: B-01; Floor: 3F; Component type: Infill wall; Component location: East wall of the third-floor corridor; Textual description: The wall surface has diagonal cracks and is accompanied by localized peeling.

[0084] The mobile device uses the camera to capture images of the damage at the scene. To complete the dimensional calibration, the inspectors place a device with an actual length of [missing information] near the area of ​​damage to be inspected. Use a ruler and ensure that the ruler and the damaged area of ​​the wall are on the same imaging plane as much as possible.

[0085] The mobile device identifies the pixel coordinates of the two endpoints of a ruler in the image. Assume the pixel coordinates of the two endpoints are... and Then the pixel length of the scale in the image is: ; Based on the actual length of the ruler and the pixel length, the conversion ratio between the pixel size and the actual physical size is obtained: ; This proportionality coefficient Used for subsequent conversion of actual dimensions of crack length, crack width, and damaged area.

[0086] 3. End-side damage identification and crack segmentation: The mobile device adjusts the on-site damage image to the model's preset input size, for example... The image is then normalized by mapping the RGB three-channel pixel values ​​from [0, 255] to [0, 1].

[0087] Subsequently, the mobile device calls the lightweight first model. The system identifies damage types in on-site damage images and outputs damage categories such as cracks, spalling, exposed rebar, and breakage, along with their corresponding damage area masks. For identified crack areas, the mobile device further calls a lightweight second model. Perform fine segmentation of the cracks and output a binary crack mask.

[0088] The mask size output by the model is the same as the model input size. The mobile device adjusts the parameters according to the image size to perform an inverse transformation on the output mask, restoring the damaged area mask and the crack binary mask to the original image size, so that the recognition result maintains a spatial correspondence with the original on-site damage image.

[0089] In this embodiment, the first model identifies two types of damage to the wall: cracks and spalling. The second model further generates a binary mask of the cracks. The mobile device displays the identification results as a mask overlay on the on-site image for inspection personnel to view.

[0090] 4. Damage parameter quantification: The mobile device extracts the crack skeleton based on a binary mask restored to the original image size, obtaining the crack centerline. Let the crack skeleton pixel length be... In this embodiment, the crack skeleton pixel length is: ; Combined with the size conversion ratio obtained in step 2 Calculate the actual length of the crack: ; For the crack width, the mobile device calculates the distance from each crack skeleton point to the crack boundary and obtains the crack width corresponding to each skeleton point. In this embodiment, the calculated average crack width is: ; The maximum width of the crack is: ; For the crack area, the system counts the number of crack pixels in the binary crack mask. Assume the number of pixels in the crack region is... The actual area of ​​the crack is: ; For the peeling area, the system counts the number of peeled pixels based on the damaged area mask. Assume the number of pixels in the peeling area is... The actual area of ​​peeling is: ; The mobile app can also perform connected component analysis on the damaged region to obtain parameters such as the number of damaged regions, the area of ​​the connected component, the perimeter of the connected component, and the shape factor.

[0091] After the above processing, the mobile device generates a structured damaged data packet: ; in, Indicates the image of the damage at the scene or the image number. Indicates basic building information, Indicates component information, Indicates the dimensional calibration parameters. This indicates the damage identification results. This indicates the results of damage quantification.

[0092] In this embodiment, the structured damage data package includes at least: building number, floor, component type, component location, damage category, crack length, average crack width, maximum crack width, crack area, spalling area, and model confidence level.

[0093] 5. Cloud-based knowledge-enhanced impairment assessment: The mobile device transmits structured damaged data packets through the end-to-end cloud communication collaboration module. The data is uploaded to the cloud-based damage assessment center. After receiving the data, the cloud-based damage assessment center constructs a search query based on fields such as component type, damage category, crack width, crack length, and spalling area. ; In this embodiment, retrieving query information may include: Component type: Infill wall; Damage types: cracks, spalling; Maximum crack width: 2.8mm; Crack length: 860mm; Peeling area: 6080mm².

[0094] Based on the above search query information, the cloud-based damage assessment center retrieves relevant standard texts, assessment clauses, and repair and reinforcement knowledge texts from the damage assessment knowledge base, resulting in a collection of knowledge texts: ; Subsequently, the cloud will transmit the structured damage data packets. The retrieved knowledge text collection and preset output format Together, they form the input information for a large language model: ; Large language models based on input information Generate damage assessment results: ; In this embodiment, the damage assessment results include: Damage level: Moderate damage; Main damage types: Wall cracks, localized spalling; Risk warning: The cracks are relatively wide, and localized spalling may affect the durability and safety of the wall, requiring manual verification; Repair and reinforcement recommendations: Seal or grout the cracks, clean, repair and protect the spalled areas; Verification recommendation: It is recommended that professionals further inspect the connection between the wall and adjacent components.

[0095] 6. Evaluation results feedback and report generation: The cloud-based damage assessment center transmits damage assessment results through an edge-cloud communication collaboration module. The data is transmitted back to the mobile damage acquisition and identification terminal. After receiving the results, the mobile terminal associates and displays the assessment results with the on-site damage image, damage area mask, size calibration parameters, and damage quantification parameters.

[0096] The mobile app displays image mask overlay results, damage parameter tables, risk level alerts, repair and reinforcement recommendations, and text assessment reports. Inspectors can view the component's damage location, damage type, crack length, crack width, damage area, and cloud-based assessment conclusions on their mobile devices.

[0097] In one implementation, the mobile device generates a standardized building damage assessment report based on on-site damage images, component information, dimensional calibration parameters, damage identification results, damage quantification results, and cloud-based damage assessment results. The report includes basic building information, component information, damage images, damage location, damage category, quantification parameters, damage level, risk description, and remediation recommendations.

[0098] 7. Model update and edge-cloud collaboration optimization: After completing multiple inspections, the system can send the manual review results, historical inspection records, and newly added building damage images back to the model training and update center. The model training and update center uses the new data to update and train the damage type identification model and the crack fine segmentation model, performs another lightweight processing, and then distributes the updated model to mobile devices.

[0099] Furthermore, the edge-cloud communication collaboration module can adjust the allocation of tasks between the mobile device and the cloud based on the mobile device's computing power, network status, and task complexity. When the mobile device meets the local inference conditions, it completes damage identification, crack segmentation, size calibration, and parameter quantization. When the mobile device's computing power is insufficient, the network status is good, or the evaluation task is complex, the mobile device uploads structured damage data to the cloud for supplementary analysis and report generation.

[0100] Through the above methods, the present invention can improve the integrity and stability of building damage assessment results while ensuring the real-time nature of on-site detection.

[0101] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for rapid building damage identification based on edge-cloud collaboration, characterized in that, include: Obtain a dataset of images showing damage to historical buildings, and based on this dataset, construct a damage type identification model and a fine crack segmentation model. Collect on-site damage images of the building components to be inspected; The on-site damage images of the building components to be detected are input into the damage type recognition model and the crack fine segmentation model respectively to obtain the damage area mask and the crack binary mask; Based on the damage area mask and the crack binary mask, the damage identification result is obtained.

2. The method for rapid building damage identification based on edge-cloud collaboration according to claim 1, characterized in that, Obtaining a dataset of historical building damage images, and constructing a damage type identification model and a fine crack segmentation model based on the historical building damage image dataset, includes: Acquire a dataset of historical building damage images, the dataset including images of surface damage to building components and their corresponding damage annotation files, the damage annotation files including at least one damage category annotation among cracks, spalling, and exposed rebar; The dataset of images showing damage to historical buildings is preprocessed. The preprocessed damage image is used as input, and the multi-class damage annotation map is used as the supervision label to train the initial damage type recognition model. The crack binary mask is used as the supervision label to train the initial crack fine segmentation model. The trained damage type identification model and crack fine segmentation model are lightweighted to obtain the damage type identification model and crack fine segmentation model.

3. The method for rapid building damage identification based on edge-cloud collaboration according to claim 1, characterized in that, Acquiring on-site damage images of the building components to be inspected also includes: Component information is recorded synchronously, and a reference straight line of known actual length is placed in the on-site damage image; The conversion ratio between pixel size and actual physical size is calculated based on the actual length of the reference line and the pixel length.

4. The method for rapid building damage identification based on edge-cloud collaboration according to claim 1, characterized in that, The on-site damage images of the building components to be detected are input into the damage type recognition model and the crack fine segmentation model, respectively, to obtain the damage area mask and the crack binary mask, including: The on-site damage image is adjusted to the preset input size of the model and pixel normalization is performed. The damage type recognition model is used to infer the normalized image and output the probability of each pixel belonging to each damage category. The category with the highest probability is taken as the predicted damage category of the corresponding pixel, and a damage region mask under a preset input size is obtained. The crack fine segmentation model is used to infer the normalized image and output a crack binary prediction mask. The damaged area mask and the crack binary mask are inversely transformed according to the size adjustment parameters, and the mask is restored to the original image size by nearest neighbor interpolation.

5. The method for rapid building damage identification based on edge-cloud collaboration according to claim 1, characterized in that, The identification method further includes: Based on the damage identification results, a retrieval query is constructed, and the damage assessment knowledge base is invoked to perform similar text retrieval and rule matching to obtain a knowledge text set and an assessment rule set. Based on the set of evaluation rules, rule matching and threshold judgment are performed to generate preliminary damage assessment results. The preliminary damage assessment results, knowledge text set and preset output format are then input into the large language model to generate a damage assessment report. The damage assessment report is displayed in conjunction with the on-site damage images, damage area masks, crack binary masks, and damage quantification results.

6. The method for rapid building damage identification based on edge-cloud collaboration according to claim 5, characterized in that, The generated damage assessment report includes: The damage identification results are processed to form a damage assessment feature vector that includes component type, damage category, damage location, crack width, crack length, damage area, and model identification confidence. Based on the damage assessment feature vector, a retrieval query information is generated, and the damage assessment knowledge base is called to perform similar text retrieval and rule matching to obtain a set of knowledge texts and a set of assessment rules corresponding to the current component type, damage category and damage quantification index. Based on the damage assessment feature vector and the set of assessment rules, rule matching and threshold judgment are performed to obtain the preliminary damage assessment result; The preliminary damage assessment results, knowledge text set, and preset output format are used to construct the report generation input information. This information is then input into a large language model for text organization, clause summarization, and report formatting to generate a damage assessment report. The large language model does not independently change the damage level, damage quantification indicators, and manual review markers obtained from rule matching and threshold judgment.

7. The method for rapid building damage identification based on edge-cloud collaboration according to claim 6, characterized in that, The rule matching and threshold determination based on the damage assessment feature vector and assessment rule set include: When the model identification confidence is not lower than the preset confidence threshold and the evaluation rule matching degree is not lower than the preset matching threshold, a preliminary damage assessment result is generated, which includes damage level, main damage type, damage quantification index, rule matching basis, risk warning, manual review suggestion and repair and reinforcement suggestion. When the model's confidence level is lower than the preset confidence threshold or the evaluation rule's matching degree is lower than the preset matching threshold, a mark indicating manual review is required and corresponding review suggestions are output.

8. A rapid building damage identification system based on edge-cloud collaboration, used to implement the method as described in any one of claims 1-7, characterized in that, include: The system includes a model building module, an image acquisition module, an image processing module, and a result output module. The model building module is used to acquire a dataset of historical building damage images and, based on the dataset, construct a damage type identification model and a crack fine segmentation model. The image acquisition module is used to acquire on-site damage images of the building components to be inspected; The image processing module is used to input the on-site damage image of the building component to be detected into the damage type recognition model and the crack fine segmentation model respectively, and obtain the damage area mask and the crack binary mask. The result output module is used to obtain damage identification results based on the damage area mask and the crack binary mask.

9. A rapid building damage identification system based on edge-cloud collaboration according to claim 8, characterized in that, The system also includes: a damage assessment module and a damage visualization module; The damage assessment module is used to construct retrieval query information based on the damage identification results, call the damage assessment knowledge base to perform similar text retrieval and rule matching to obtain a knowledge text set and an assessment rule set; perform rule matching and threshold judgment based on the assessment rule set to generate a preliminary damage assessment result, and input the preliminary damage assessment result, the knowledge text set and the preset output format into the large language model to generate a damage assessment report; The damage visualization module is used to associate and display the damage assessment report with the on-site damage image, damage area mask, crack binary mask, and damage quantification results.

Images