Intelligent management method for diagnosis and prevention of ancient building wood components based on fusion of digital twinning

Abstract

The application belongs to the technical field of intelligent management of ancient buildings, and provides an ancient building wood component diagnosis and prevention intelligent management method combined with digital twinning, comprising: processing the main diagnosis surface image of the wood component by using a damage diagnosis model to obtain damage state recognition results; integrating the static attribute data of the wood component, and environment exposure data, damage state recognition results and maintenance data to obtain a structured digital twinning archive; obtaining the feature vector of the wood component at each historical time point based on the structured digital twinning archive, and stacking them in time sequence into a three-dimensional feature tensor; inputting the three-dimensional feature tensor into a multi-task time series prediction model to obtain the future damage prediction result of the wood component and the damage influence weight of the historical time point; calculating the SHAP value of the index; generating damage cause description; matching the damage cause description with a decision rule library to obtain the maintenance decision instruction of the wood component and sending it. The accuracy and forward-looking nature of the future damage prediction result of the wood component are improved, and a decision closed loop is formed.

Description

Technical Field

[0001] This invention relates to the field of intelligent management technology for ancient buildings, and in particular to an intelligent management method for the diagnosis and prevention of wooden components in ancient buildings that integrates digital twins. Background Technology

[0002] Most related technologies for structural health monitoring, damage identification, and maintenance decision-making for ancient building wooden components focus on single data sources or localized optimization of specific aspects. While progress has been made in data governance, coupled prediction, and decision-making processes, they suffer from the following drawbacks when addressing the ultimate goal of preventative protection of ancient building wooden components: Some solutions rely heavily on environmental sensor data (temperature and humidity) and wood constitutive parameters for damage prediction, resulting in inaccurate predictions that cannot accurately correspond to specific, visible damage morphology evolution. Furthermore, some solutions base maintenance decisions for ancient buildings on superficial criteria, often based on macroscopic damage levels or abstract assessment scores, failing to identify key causes of damage. This leads to insufficiently targeted preventative maintenance measures, potentially resulting in resource waste or inadequate intervention. Summary of the Invention

[0003] This application aims to at least address the technical problems existing in the prior art and provide an intelligent management method for the diagnosis and prevention of ancient building wooden components that integrates digital twins.

[0004] This application provides an intelligent management method for the diagnosis and prevention of ancient building wooden components integrating digital twins. The method includes: periodically acquiring main diagnostic surface images, environmental exposure data, and maintenance data of the wooden components; processing the main diagnostic surface images of the wooden components using a pre-trained damage diagnosis model to obtain damage status identification results of the wooden components; integrating the static attribute data of the wooden components, as well as environmental exposure data, damage status identification results, and maintenance data from multiple historical time points to obtain a structured digital twin archive of the wooden components; obtaining feature vectors of the wooden components at each historical time point based on the structured digital twin archive of the wooden components, the feature vectors including multiple indicators; stacking the feature vectors of all historical time points within the historical time window in chronological order into a three-dimensional feature tensor; inputting the three-dimensional feature tensor into a pre-trained multi-task temporal prediction model to obtain future damage prediction results of the wooden components and damage influence weights of more than one historical time point; and calculating the SHAP value of more than one indicator based on the future damage prediction results. By combining the future damage prediction results of the wooden components, the damage impact weight of more than one historical time point, and the SHAP value of more than one indicator, a damage cause description of the wooden components is generated; the damage cause description of the wooden components is matched with the decision rule base to obtain the maintenance decision instructions for the wooden components; and the maintenance decision instructions for the wooden components are sent to the management end.

[0005] The beneficial technical effects of this application are as follows: For each wooden component in ancient buildings, damage diagnosis, future damage prediction, damage cause analysis, and maintenance decision instructions are performed individually. The damage diagnosis model directly processes the main diagnostic surface image of each wooden component, making damage judgment no longer dependent on environmental data and the wood itself, and enabling more accurate and intuitive identification of the damage status of the wooden components. A structured digital twin archive is constructed for each wooden component, forming a traceable, multi-dimensional, and time-series database of the wooden component's life cycle. A multi-task time-series prediction model evolves damage based on the fusion of multi-source data, including static attribute data, environmental exposure data from multiple historical time points, damage status identification results, and maintenance data, to obtain future damage prediction results for the wooden components, improving the accuracy and foresight of the prediction. Damage cause analysis is performed based on the damage impact weights and SHAP values ​​of indicators at historical time points to obtain interpretable damage cause descriptions of the components. These damage cause descriptions are then matched with a decision rule base to obtain targeted maintenance decision instructions for the wooden components, forming a closed-loop decision system encompassing damage diagnosis, future damage prediction, and maintenance decision instructions. Attached Figure Description

[0006] Figure 1 This is a flowchart illustrating a preferred embodiment of the present invention: an intelligent management method for the diagnosis and prevention of ancient building wooden components that integrates digital twins. Figure 2 This is a flowchart of a structured digital twin archive component for wooden components, as shown in an example of the present invention. Figure 3 This is a flowchart of the training and testing process of a damage diagnosis model in an example of the present invention; Figure 4 This is a flowchart of the training and testing process for a multi-task time-series prediction model in one example of the present invention; Figure 5 This is a schematic diagram illustrating the generation of maintenance decision instructions in one example of the present invention; Figure 6 This is a schematic diagram of the construction process of a visual interactive platform in one example of the present invention; Figure 7 This is a schematic diagram of the structure of a damage diagnosis model in a preferred embodiment of the present invention. Detailed Implementation

[0007] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0008] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0009] The intelligent management method for diagnosis and prevention of ancient building wooden components integrating digital twins provided by this invention includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, or a computer. In other words, the intelligent management method for diagnosis and prevention of ancient building wooden components integrating digital twins can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0010] This invention provides an intelligent management method for the diagnosis and prevention of ancient building wooden components that integrates digital twins. In a preferred embodiment, such as... Figure 1 As shown, the method includes: Step S1: Periodically acquire main diagnostic surface images, environmental exposure data, and maintenance data of the wooden components.

[0011] In this embodiment, the acquisition cycles for the main diagnostic image, environmental exposure data, and maintenance data can be different.

[0012] For the main diagnostic surface images, wooden components with different risk levels are sampled according to different sampling periods or sampling frequencies. For example, main diagnostic surface images are collected every six months for wooden components in the high-frequency monitoring group, and once a year for wooden components in the regular monitoring group. Supplementary collection can also be triggered after extreme events such as severe lightning, storms, and torrential rain.

[0013] Specifically, the main diagnostic surface of a wooden component refers to the surface on which damage is most evident, identified based on structural mechanics and disease principles (e.g., the bottom surface of a beam, the shaded surface of a column). This surface is where damage is most apparent. Manual data acquisition can be performed using a portable full-frame high-definition camera (resolution no less than 24 megapixels, equipped with a standard color chart and scale) + tripod + positioning target, ensuring repeatability. Uniform overcast lighting or diffused artificial light sources should be prioritized to eliminate shadows and reflections. Non-destructive cleaning should be performed before shooting. Camera acquisition posture: The camera lens axis should be as perpendicular as possible to the component surface, and the shooting distance should be fixed at 1.0 ± 0.1 meters to ensure consistent image scale for each acquisition. After acquiring the original main cross-sectional image, the following preprocessing is performed: white balance correction, geometric distortion correction, uniform scaling to a fixed resolution (e.g., 1024x1024 pixels), and saving in a non-destructive format. This ensures the consistency of color and scale in the image data input to the damage diagnosis model.

[0014] In this embodiment, the collection cycle for environmental exposure data is not limited to daily or every 6 hours. Environmental exposure data is acquired through a combination of wireless sensor network deployment and on-site investigation, and is used to quantify the microenvironmental stress of the wooden components, which is an external factor driving damage evolution. Environmental exposure data includes: (1) Humidity and humidity load: Temperature and humidity sensors are deployed near the wooden components to continuously monitor and record time-series data of temperature and relative humidity. Key derived indicators based on this time-series data include the estimated annual average moisture content (which can be estimated using empirical formulas) and the duration of high humidity (the number of hours with an annual air humidity level of RH > 80%).

[0015] (2) Solar radiation load: Record the sunshine duration and solar radiation intensity on the surface of the component. The annual cumulative solar radiation of the wooden component can be obtained by analyzing the three-dimensional model of the building orientation and surrounding obstructions, combined with local meteorological data (EPW file).

[0016] (3) Ventilation and humidity index: The local ventilation conditions and the risk of contact with liquid water are comprehensively assessed and divided into discrete levels through on-site investigation, such as: good ventilation / dry (level 1), average ventilation / occasionally damp (level 2), poor ventilation / long-term damp (level 3).

[0017] In this embodiment, the maintenance data of wooden components can be historical intervention records, which record the time, maintenance type, materials and processes used for each repair in a timestamp sequence.

[0018] Step S2: Process the main diagnostic surface image of the wooden component using a pre-trained damage diagnosis model to obtain the damage status identification result of the wooden component.

[0019] In this embodiment, the damage diagnosis model preferably uses, but is not limited to, the existing YOLO series of target detection networks. The damage state identification results are not limited to including damage type and damage degree.

[0020] Step S3: Integrate the static attribute data of the wooden components, as well as environmental exposure data, damage status identification results and maintenance data from multiple historical time points, to obtain a structured digital twin archive of the wooden components.

[0021] In this embodiment, the static property data of the wooden components do not change over time, and include: (1) Component code, which is unique and equivalent to an identifier, is abbreviated as ID. It can be generated according to the rule of "building number-region-type-serial number", such as SWC-01-main beam-012.

[0022] (2) Spatial location, specifically including the coordinate set of key feature points of the wooden component (such as the coordinates of the center of gravity of the wooden component, the center point of the end, the tenon and mortise positioning point, etc.). The point cloud model of the wooden component can be obtained by three-dimensional laser scanning, and two types of key coordinate data can be extracted from it to accurately describe the geometric shape, installation position and spatial relationship with adjacent wooden components. All coordinates are unified under the same absolute or relative global coordinate system.

[0023] (3) Topological connection relationship of timber components: Create a topological connection relationship table for each timber component. This table records all timber components directly connected to it and their connection attributes in a structured manner. The topological connection relationship table is the core data structure for constructing the overall timber frame relationship diagram, analyzing load transfer paths, and simulating damage chain effects.

[0024] Each connection record contains: a. Connecting Component ID: A unique identifier for the wooden component being connected.

[0025] b. Connection type: Based on the standardized classification of traditional materials such as the "Yingzao Fashi" (e.g., dovetail tenon, hoop tenon, through tenon, etc.).

[0026] c. Connection location: The key feature point ID or local coordinates on the main wooden component and the connecting wooden component.

[0027] (4) Mechanical role: Based on the stress state classification, it can be quantified into discrete variables, such as: main load-bearing column (MR=3, such as column, beam), secondary load-bearing beam (MR=2, such as purlin), and non-load-bearing connecting member (MR=1). This parameter directly affects the critical threshold of component failure.

[0028] (5) Material properties: including wood species (such as fir and cypress), standard air-dry density ρ, initial modulus of elasticity, etc. For ancient buildings without measured data, empirical reference values ​​can be assigned based on tree species and age, referring to the reference "Test Methods for Mechanical Properties of Ancient Building Timber Structure Materials".

[0029] In this embodiment, a structured digital twin profile is created for each wooden component, which is essentially a multidimensional time-series data object that evolves over time and has embedded relationships. A structured digital twin profile can be stored in a database using a single table. To ensure a complete depiction of the wooden component's lifecycle, its primary key is a composite primary key: wooden component ID + timestamp. Each record corresponds to a complete snapshot of the wooden component's state at a specific moment. The complete state includes static attribute data at that specific moment, as well as environmental exposure data, damage status identification results, and maintenance data from multiple historical time points. Figure 2 This is a flowchart of a structured digital twin archive component for wooden components, as shown in an example of the present invention.

[0030] Step S4 involves obtaining the feature vector of the wooden component at each historical time point based on the structured digital twin archive of the wooden component. The feature vector includes multiple indicators. Preferably, step S4 includes: Static attribute sub-vectors are generated based on the static attribute data of the wooden components at this historical time point. The static attribute sub-vectors include component codes, spatial locations, mechanical roles (e.g., primary load-bearing = 3, secondary = 2, non-load-bearing = 1), wood species (which can be represented by unique thermal codes), initial material strength, and also include a topological connection table; this part remains unchanged throughout the time series. A dynamic environmental subvector is generated based on the environmental exposure data of wooden components at this historical time point. Environmental exposure data from archives, dynamic environmental subvectors include at least one of the following: average environmental temperature, average relative humidity, cumulative rainfall, cumulative solar radiation, and estimated average moisture content of timber at that historical point in time (or within the period collected or estimated). Based on the damage state identification results of the wooden components at this historical time point, a damage state sub-vector is generated. The damage status identification results are derived from the archives. The damage status sub-vector includes whether there is damage, damage type, and damage degree. Damage type includes crack and decay. Damage degree is the quantified severity value of the damage type. The damage status sub-vector also includes the confidence of the damage diagnosis model. The damage status sub-vector is a direct quantitative expression of the health status of the wooden component. The whether there is damage can be represented by 0 or 1, where 1 indicates that there is damage and 0 indicates that there is no damage. Maintenance intervention sub-vectors are generated based on the maintenance data of the wooden components at this historical time point. The maintenance data comes from the archives. The maintenance intervention sub-vector includes a maintenance flag and a maintenance type. The maintenance flag can be represented by 0 or 1. 1 indicates that maintenance (such as reinforcement or replacement) has occurred at or near this historical time point, and 0 indicates that no maintenance has been performed at this historical time point. By concatenating the static attribute sub-vectors, dynamic environment sub-vectors, damage state sub-vectors, and maintenance intervention sub-vectors of the spliced ​​wooden component at that historical time point, a feature vector of the wooden component at that historical time point is obtained. Multiple indicators of the feature vector include the aforementioned spatial location, mechanical role, wood species, initial material strength, average environmental temperature, average environmental relative humidity, cumulative rainfall, cumulative solar radiation, estimated average wood moisture content, whether it is damaged, damage type, damage degree, whether it requires maintenance, and maintenance type.

[0031] Step S5 involves stacking the feature vectors of all historical time points within the historical time window into a three-dimensional feature tensor in chronological order. Specifically, this involves constructing... At historical junctures eigenvectors Represented as: .

[0032] Feature vector Stacked in chronological order to form a three-dimensional feature tensor ,in The number of time steps within the historical time window. This represents the total number of indicators. This tensor fully encapsulates the evolution trajectory of the component's health status within a historical time window.

[0033] Step S6 involves inputting the three-dimensional feature tensor into a pre-trained multi-task temporal prediction model to obtain the future damage prediction results for the wooden components and the damage influence weights at more than one historical time point. The multi-task temporal prediction model can be an existing long short-term memory neural network.

[0034] Step S7: Calculate the SHAP value of one or more indicators based on the future damage prediction results.

[0035] Specifically, the SHAP value is used as a post-hoc attribution tool. Based on cooperative game theory, the SHAP value calculates a fair contribution value for each indicator in the feature vector.

[0036] 1) Analysis process: With the current prediction task (such as the probability of damage transition risk) as the target, the Kernel SHAP algorithm is used to calculate the SHAP value of all input indicators.

[0037] 2) Attribution and Ranking: The system analyzes the results and generates a ranked list of feature contributions. Positive contribution values ​​indicate that the indicator increases the probability of damage transition risk, while negative contribution values ​​indicate that it decreases the probability of damage transition risk. For example, the quantification results might show: Positive driving factors: Mechanical role (primary load-bearing) SHAP value = +0.15; Average humidity over the past 24 months: SHAP value = +0.12; Time since last structural maintenance: SHAP value = +0.09.

[0038] Negative mitigation factor: Local ventilation index: SHAP value = -0.05.

[0039] This allows us to clearly identify which specific indicators, and to what extent, dominate the level of damage transition risk.

[0040] Step S8: Combine the future damage prediction results of the wooden component, the damage impact weights of more than one historical time point, and the SHAP values ​​of more than one indicator to generate a damage causation description for the wooden component. Specifically, select the historical time points corresponding to the largest y1 damage impact weights from more than one historical time point (e.g., all historical time points). Select the indicators corresponding to the largest y2 SHAP values ​​from more than one indicator. Integrate the future damage prediction results, the selected historical time points, and the indicators to generate a damage causation description.

[0041] Example: High risk of deterioration of the wooden component within the next 18 months ( =0.72), mainly driven by the coupled effect of its load-bearing capacity and long-term high-temperature and humid environment, and lacking mid-term maintenance intervention. Furthermore, the extreme wet damage during the 2021 rainy season was a significant starting point for deterioration, while the surge in damage level in the summer of 2023 further confirmed the worsening trend. Primary factor (structure-environment coupling): The combined effect of the "primary load-bearing" role (contribution +0.15) and the "long-term high-humidity environment" (contribution +0.12) constitutes the main driving force for the damage to wooden components. Secondary factor (lack of intervention): "More than 10 years without structural maintenance" (contribution +0.09) prevented early micro-damage from being contained.

[0042] Mitigating factor: "Relatively good ventilation" (contribution -0.05) partially slowed the rate of deterioration, but was insufficient to offset the main risks.

[0043] Step S9: Match the description of the damage cause of the wooden component with the decision rule base to obtain the maintenance decision instruction for the wooden component; send the maintenance decision instruction for the wooden component to the management terminal.

[0044] In this implementation, the production rule base and case base form the foundation of the decision rule base. The rules are derived from cultural relic restoration specifications, structural safety standards, materials science knowledge, and domain expert experience, and are encoded into computer-processable IF-THEN logical statements. Each rule is associated with multiple dimensions of objective constraints, which are as follows: Safety maximization goal: Reduce the degree of damage to below the safety threshold.

[0045] Minimize intervention objective: Prioritize the process that causes the least interference to the artifact itself and its historical information.

[0046] Cost minimization objective: This includes minimizing material costs, labor costs, and downtime losses.

[0047] Maximum durability goal: to achieve the longest expected maintenance durability cycle.

[0048] Constraints include: construction season, material availability, and on-site operating conditions.

[0049] The condition section of the rule not only incorporates the type and extent of damage, but also delves deeper into interpretable attribution results (i.e., the SHAP value of the indicator and the damage state identification result). For example, an advanced rule might be defined as: "IF The primary risk attribution is 'long-term high humidity' and the damage type is 'non-structural decay' THEN The core strategy should include 'environmental control' and recommend 'hydrophobic and corrosion-inhibiting materials'." In this way, the decision-making basis delves from surface phenomena to the disease mechanism.

[0050] In a preferred embodiment, preferably, the process of matching the damage cause description with the decision rule base in step S9 employs multi-stage reasoning, including: (1) Initial strategy screening: The engine matches the features of the wooden components with the decision rule base, triggers all rules that meet the conditions, and generates an initial set of feasible maintenance strategies. For example, for "moderate decay of load-bearing columns in high humidity environment", multiple strategies corresponding to rules such as "grouting reinforcement", "partial replacement", "carbon fiber cloth wrapping" and "environmental dehumidification" may be triggered at the same time.

[0051] (2) Quantitative evaluation and optimization: Each maintenance strategy will be scored on dimensions such as "safety", "intervention", "cost" and "durability". Then, the non-dominated sorting genetic algorithm II (NSGA-II) of the elite strategy can be used to find the Pareto optimal solution set. Through selection, crossover and mutation operations, the population evolves generation by generation, and finally outputs a set of optimal maintenance schemes (e.g., scheme A focuses on safety but has high cost, scheme B focuses on low intervention but requires high frequency monitoring) for decision-makers to choose from.

[0052] (3) Generating precise instructions: The system outputs the final precise maintenance instructions from the optimal solution set. The precise maintenance instructions are a structured document that explicitly includes: Target component and problem summary: Refer to the wooden component ID and briefly describe "the risk of decay deepening in the main load-bearing column due to long-term high humidity".

[0053] Recommended measures and processes: Specific solutions, such as "using low-pressure injection method for silicone resin anti-corrosion reinforcement and adding a capillary drainage isolation layer around the column base".

[0054] Materials and parameters: Specify the technical standards and key construction parameters for the materials.

[0055] Decision-making basis: Directly link and reference the risk values ​​in the forecast report. And the core conclusions of the SHAP attribution analysis (such as "based on humidity contribution = +0.12").

[0056] Construction priority and time window: Based on the risk level and seasonal impact, it is recommended to implement the project within three months before the next rainy season. Figure 5 This is a schematic diagram illustrating the generation of maintenance decision instructions in one example of the present invention.

[0057] In a preferred embodiment, the intelligent management method for diagnosis and prevention of ancient building wooden components integrating digital twins further includes displaying the geometric model of the wooden components, the structural topology diagram in the ancient building, the damage status identification results of the wooden components, and the future damage prediction results in a visual interactive platform.

[0058] In this embodiment, a user interface layer integrating 3D visualization, interactive analysis, simulation and collaborative decision-making is built on the visualization interaction platform. The diagnostic, prediction and decision results generated in the previous steps are presented in an intuitive and operable manner, and in-depth analysis tools and collaborative working environment are provided for users with multiple roles.

[0059] In this implementation, the visualization and interactive platform uses WebGL / Three.js technology to build a web-based 3D rendering engine, supporting smooth browsing of large ancient building complexes. It achieves layered detail rendering technology: at the macro level, it displays the overall risk distribution of the building, while at the micro level, it can focus on the surface texture and damage details of individual wooden components.

[0060] In this embodiment, the platform establishes a real-time connection with the backend service via WebSocket. When new monitoring data, diagnostic results, or prediction reports are generated, the status of the corresponding components in the 3D scene is automatically updated, and the status evolution process is displayed in the form of animation.

[0061] In this embodiment, and more preferably, an intelligent coloring system is also provided on the visual interactive platform. Based on the damage transition risk probability in the future damage prediction results output by the multi-task time-series prediction model, a dynamic color code is assigned to the surface of each wooden component, such as red: high risk (…). >0.7); Yellow: Medium risk (0.3 < ≤0.7); Green: Low risk ( ≤0.3).

[0062] In this embodiment, the visualization and interactive platform can also overlay the damage type probability map output by the damage diagnosis model onto the surface of the wooden component model in the form of a semi-transparent heat map. Figure 6 This is a schematic diagram illustrating the construction process of a visual interactive platform in an example.

[0063] In a preferred embodiment, please see Figure 7 As shown, the damage diagnosis model includes: A backbone network is used to extract features at multiple scales from the main diagnostic surface image of the wooden component. Preferably, the backbone network employs a deep convolutional neural network, such as the ResNet-50 neural network model. The original fully connected classification head of the ResNet-50 neural network model is removed, retaining the feature maps of its output at four stages, such as... Figure 7 As shown, these feature maps, denoted as C2, C3, C4, and C5, contain rich information ranging from low-level texture to high-level semantics.

[0064] The neck network, based on a feature pyramid fusion module, fuses features from multiple scales output by the backbone network to obtain multiple fused features. The neck network, through top-down and lateral connections, fuses features from different scales in the backbone network to generate a set of enhanced feature pyramid features with a uniform number of channels but different spatial resolutions, i.e., four fused features, denoted as P2, P3, P4, and P5. This solves the problem of large differences in damage scale, enabling the neck network to capture both large-area damage regions and focus on subtle texture changes.

[0065] The multi-task prediction head includes a damage type classification head, a damage severity regression head, and an aggregation unit; Among them, the damage type classification head generates a damage type probability map corresponding to each fusion feature; The damage level regression head generates a damage level map corresponding to each fused feature; The aggregation unit is used to aggregate the probability maps of damage types corresponding to multiple fusion features to obtain the damage type, and also to aggregate the damage degree maps corresponding to multiple fusion features to obtain the damage degree. The damage type and damage degree constitute the damage state identification result of the wooden component.

[0066] In this embodiment, the damage type classification head includes a Convolutional layers and the Sigmoid activation function The convolutional layer performs convolution on the features at each spatial location of each fused feature, outputting K channels. Then, it applies the sigmoid activation function to each channel to obtain the damage type probability at that spatial location. , This represents the probability that the Kth type of damage exists at this spatial location. The outputs of the fusion features from all levels are upsampled to the input image size using bilinear interpolation and then averaged and fused at the pixel level to generate the final damage type probability map for that level, which corresponds to the damage type probability map of the fusion feature.

[0067] In this embodiment, the damage degree regression head includes a The convolutional layer performs convolution on the feature values ​​of each spatial location of each fused feature to obtain a single scalar, which represents the severity of damage at that spatial location. The fused features from each layer are then fused to determine the severity of damage at each spatial location and upsampled to the input image size using bilinear interpolation, resulting in a continuous damage map of the same size as the input image.

[0068] Finally, through global average pooling or pooling operations targeting the damaged regions of the damage type probability map and damage severity map, the damage type and overall damage severity scalar (i.e., damage severity) of the entire image layer are aggregated from the aforementioned damage type probability map and damage severity map.

[0069] In this embodiment, the training process of the damage diagnosis model includes: Step A1: Prepare a sample image dataset. The sample image dataset includes main diagnostic surface images of different wooden components and damage labels set by experts for each main diagnostic surface image. The damage labels include the damage type and the degree of damage.

[0070] Step A2: Build the damage diagnosis model network.

[0071] Step A3: Divide the sample image dataset into a training set, a test set, and a validation set according to a preset partitioning ratio. The preset partitioning ratio is not limited to 8:1:1.

[0072] Step A4: Train the injury diagnosis model network using the training set. During training, calculate the loss function value based on the injury state recognition results and injury labels output by the injury diagnosis model network. Update the network parameters of the injury diagnosis model network using Stochastic Gradient Descent (SGD) based on the loss function value. When the training stopping condition is met, stop training and save a set of network parameter values ​​of the injury diagnosis model network when the loss function value is minimized. Load the saved network parameter values ​​into the injury diagnosis model network to obtain the trained injury diagnosis model. The training stopping condition is not limited to reaching the maximum preset number of training iterations or the loss function value being less than a preset loss threshold. In step A4, the formula for calculating the loss function value is: , This represents the binary cross-entropy loss calculated using the damage state identification results and the damage types in the damage labels; This represents the L1 loss calculated based on the damage state identification results and the degree of damage in the damage label. The loss weight represents the degree of damage.

[0073] Step A5: Test and validate the trained damage diagnosis model using the test set and validation set respectively. If the test and validation are passed, the damage diagnosis model network that has passed the test and validation is the final damage diagnosis model. If the test or validation is failed, use deep learning model parameter tuning methods such as changing the learning rate and changing the training optimizer, and then return to execute steps A4 and A5. Figure 3 This is a flowchart of the training and testing process for a damage diagnosis model in one example of the present invention.

[0074] In reality, there are far fewer diagnostic images of actual injuries compared to those of healthy individuals, resulting in a severe imbalance between positive and negative samples. This imbalance reduces training efficiency and the accuracy of the model's loss state identification. Therefore, in a preferred embodiment, the sample image dataset is an augmented image dataset, and the injury diagnosis model is trained using this augmented image dataset.

[0075] In this embodiment, the method for constructing an enhanced image dataset includes: Step B1: Obtain the original main diagnostic surface images of multiple wooden components of the ancient building; Step B2: Preprocess the original main diagnostic surface image to obtain the main diagnostic surface image; Step B3: Label the damage type on the main diagnostic image; Step B4: Select multiple real damage images from multiple main diagnostic images, and segment the real damage images into damage area images and healthy background images; specifically, this is not limited to manually marking and outlining the damage area in the main diagnostic image, then cropping the damage area image from the real damage image according to the marking, and using the remaining part as the healthy background image. Step B5: Extract physical feature labels from the damaged area image. These labels include crack main direction features, crack continuity features, decay texture level features, and decay diffusion morphology features. If the main diagnostic image of the wooden component only contains cracks, the decay texture level features and decay diffusion morphology features are all zero vectors. If the main diagnostic image of the wooden component only contains decay, the crack main direction features and crack continuity features are all zero vectors. If the main diagnostic image of the wooden component does not contain either cracks or decay, the crack main direction features, crack continuity features, decay texture level features, and decay diffusion morphology features are all zero vectors. Generally, cracks and / or decay are present in the main diagnostic image.

[0076] The principal direction feature of the crack is obtained by skeletonizing the crack region in the damage layer / real damage image and combining it with principal component analysis; alternatively, the principal direction feature is obtained by mapping the long axis of the crack region in the damage layer / real damage image. Crack continuity features can be characterized by the length of the crack's connected components or the number of pixels in the connected components. Decay texture level features can be obtained through gray-level co-occurrence matrix texture feature mapping of the decay region. Decay diffusion morphology features can be characterized by the spatial morphological parameters of the decay region.

[0077] Step B6: Using a pre-trained conditional generative adversarial network generator, a damage layer is generated based on the physical feature labels of the real damaged image, the healthy background image, and the damage type label. The damage type label of the real damaged image is then used as the damage type label of the damage layer.

[0078] Step B7: Combine multiple damage layers, multiple master diagnostic images, damage type labels corresponding to the damage layers, and damage type labels corresponding to the master diagnostic images to form an enhanced image dataset.

[0079] In this embodiment, the physical feature labels of the real damage image, the healthy background image, and the damage type label are used as conditions to generate the generator of the adversarial network. This guides the generator to generate a damage layer that conforms to the pattern of the real damage image. Specific attributes of the image (such as damage type, degree, and location) can be specified, and multiple attribute dimensions (texture, color, shape, and layout) can be controlled simultaneously. This gives the damage layer the characteristics of high resolution and rich detail, making it approximate the real damage image.

[0080] In this embodiment, preferably, the conditional generative adversarial network includes: The conditional encoder encodes the physical feature labels and damage type labels of the real damage image respectively and concatenates the encoding results to obtain conditional features. It extracts background features from the healthy background image and concatenates the conditional features and background features to obtain input features. A generator that produces damage layers based on input features; The discriminator calculates the generation loss based on the damage layer and the real damage image. During the training of the conditional generative adversarial network, the network parameters of the conditional encoder and generator are adjusted based on the generation loss.

[0081] In this embodiment, the conditional encoder includes a first multilayer perceptron, an embedding layer, a second multilayer perceptron, a convolutional neural network (CNN), a first concatenation unit, a second concatenation unit, and a fully connected fusion layer. The fully connected fusion layer includes multiple cascaded fully connected layers and a ReLU activation function connected to each fully connected layer. The first multilayer perceptron processes physical feature labels to obtain physical feature encodings. The embedding layer embeds damage type labels into vectors, and the resulting vectors are input to the second multilayer perceptron to obtain damage type encodings. The CNN processes healthy background images to obtain background feature encodings. The first concatenation unit concatenates the physical feature encodings and damage type encodings to obtain conditional features, and the second concatenation unit concatenates the conditional features with the background feature encodings to obtain input features.

[0082] In this embodiment, the generator is preferably an existing U-Net network.

[0083] In this embodiment, the discriminator calculates and generates a loss based on the damage layer and the real damage image, including: Step C1, Calculate the damage layer Compared with real damage images The L2 norm of the pixel difference image is calculated, and this L2 norm is used as the adversarial loss term. . .

[0084] Step C2: Segment the damaged area image from the damaged layer. From the image of the damaged area Extract physical feature labels ; These represent the images of the damaged regions segmented from the damaged layer. The characteristics of the main direction of the crack, the continuity of the crack, the grade of decay texture, and the morphology of decay diffusion. Indicates the transpose operator. Actual damage image. The image of the segmented damaged area is From the image of the damaged area Extract physical feature labels ; These represent the actual damage images. Image of the segmented damaged area The characteristics of the main direction of the crack, the continuity of the crack, the grade of decay texture, and the morphology of decay diffusion.

[0085] Step C3: Calculate the L1 norm of the deviation between the physical feature labels of the damage layer and the physical feature labels of the real damage image, and use the L1 norm of the deviation as the physical label consistency loss term. .

[0086] Step C4: Combine the adversarial loss term and the physical label consistency loss term to obtain the generation loss. . , This represents the weight of the physical label consistency loss term.

[0087] In this embodiment, the training process of the conditional generation network includes: Step D1 involves acquiring multiple real damage images of wooden components. From each real damage image, the damaged area image and the healthy background image are segmented. Physical feature labels are extracted from the damaged area image, and the damage type is manually determined and a damage type label is generated. Each real damage image, along with its corresponding healthy background image, physical feature label, and damage type label, constitutes a real damage image sample, forming a real damage image sample set. The healthy background image, physical feature label, and damage type label serve as input to the conditional encoder.

[0088] Step D2: Build a conditional generative adversarial network.

[0089] Step D3: Divide the real damage image sample set into a training set, a test set, and a validation set according to a preset division ratio. The preset division ratio is not limited to 8:1:1.

[0090] Step D4: Train the Conditional Generative Adversarial Network (GAN) using the training set. The input to the conditional encoder is a healthy background image, physical feature labels, and damage type labels. Its output features are fed into the generator, which synthesizes a damage layer. The discriminator calculates the generation loss based on the damage layer and the real damage image. The network parameters of the conditional encoder and generator in the GAN are updated using Stochastic Gradient Descent (SGD) based on the generation loss. Training stops when the training stopping condition is met. The training stopping condition is not limited to reaching the maximum preset number of training iterations.

[0091] Through the above training, the conditional generative adversarial network not only learns the appearance distribution features of real damage images, but also explicitly learns the physical evolution law of wooden component diseases, thereby generating synthetic damage layer samples that meet the requirements in both visual realism and physical rationality, improving its effectiveness and reliability in subsequent damage diagnosis model training.

[0092] In a preferred embodiment, the method for constructing the enhanced image dataset further includes: performing spatial transformation and / or spectral transformation on the main diagnostic surface image to obtain a transformed enhanced image; using the damage type label of the original main diagnostic surface image as the damage type label of the transformed enhanced image obtained by its spatial transformation and / or spectral transformation. Multiple transformed enhanced images, multiple damage layers, and multiple main diagnostic surface images, along with their corresponding damage type labels, constitute the enhanced image dataset.

[0093] In a preferred embodiment, the multi-task time-series prediction model includes: The encoder, including multi-layer LSTM units, extracts features from the feature vector of each historical time point in the three-dimensional feature tensor to obtain the hidden state of that historical time point. The hidden states of all historical time points within the historical time window form a hidden state sequence. The self-attention processing module uses a self-attention mechanism to obtain the attention weight of each hidden state in the hidden state sequence. The attention weight of the hidden state is used as the damage impact weight of the corresponding historical time point. The hidden state sequence is weighted and summed according to the attention weight of the hidden state to obtain the weighted hidden state sequence. The decoder obtains future damage prediction results based on the weighted latent state of the last historical time point in the weighted latent state sequence. The future damage prediction results include the probability of damage transition risk. Types of damage and degree of damage . , This represents the probability of a transition in damage level (e.g., from mild to moderate, or from no damage to mild damage) occurring within a future time period. The weighted implicit state at the last historical time point is considered a comprehensive code of the historical state of the wooden component. The future time period can be an acquisition cycle of the main diagnostic surface image, such as one year or six months.

[0094] In this embodiment, the decoder includes three fully connected networks. The first fully connected network performs a mapping transformation on the weighted latent state at the last historical time point to obtain the probability of damage transition risk. , which is a scalar between 0 and 1. The second fully connected network maps and transforms the weighted hidden state at the last historical time point into... 3D damage type probability vector , Represents the number of damage types, output vector The damage type corresponds to the probability of the maximum damage type. The third fully connected network performs a mapping transformation on the weighted latent state at the last historical time point to obtain a regression value, which is used to predict the possible future damage level. .

[0095] In this embodiment, a tensor sample dataset is constructed. The tensor sample dataset includes a set of tensor samples and a true predicted label for each tensor sample. Based on structured digital twin archives of multiple wooden components, multiple three-dimensional feature tensors with historical time windows are generated. Each three-dimensional feature tensor is treated as a tensor sample, and the damage state identification result at the first historical time point after the historical time window is used as the true predicted label for that tensor sample, thus completing the construction of the tensor sample dataset. A network structure for a multi-task temporal prediction model is then constructed, and the multi-task temporal prediction model is trained using the tensor sample dataset. Figure 4 This is a flowchart of the training and testing process for a multi-task time-series prediction model in one example of the present invention.

[0096] Loss function during training of multi-task temporal prediction model for: in, , , These represent the first weight, the second weight, and the third weight, respectively. Represents the binary cross-entropy function; Represents the cross-entropy function; Represents the mean square error function; , These represent the damage transition risk probability output by the multi-task time series prediction model and the label of whether a damage transition has occurred in the actual prediction label (1 or 0, where 1 indicates that a damage transition has occurred in the future time period and 0 indicates that a damage transition has not occurred in the future time period). , These represent the damage type output by the multi-task time series prediction model and the damage type output by the damage diagnosis model for future time periods in the true prediction labels, respectively. , These represent the damage level output by the multi-task time-series prediction model and the damage level output by the damage diagnosis model for future time periods in the true prediction labels, respectively.

[0097] In the description of this specification, the references to terms such as "an embodiment," "some embodiments," "example," "specific example," "a implementation," "a preferred implementation," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0098] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for intelligent management of diagnosis and prevention of ancient architectural wooden components integrating digital twins, characterized in that: The method includes: Regularly acquire diagnostic images, environmental exposure data, and maintenance data for timber components; The damage status identification results of the wooden components are obtained by processing the main diagnostic surface image of the wooden components using a pre-trained damage diagnosis model. By integrating the static attribute data of wooden components with environmental exposure data, damage status identification results and maintenance data from multiple historical time points, a structured digital twin archive of wooden components is obtained. Based on the structured digital twin archive of wooden components, feature vectors of wooden components are obtained at each historical point in time. The feature vectors include multiple indicators. Stack the feature vectors of all historical time points within the historical time window into a three-dimensional feature tensor in chronological order. Input the three-dimensional feature tensor into a pre-trained multi-task time series prediction model to obtain the future damage prediction results of wooden components and the damage influence weights of more than one historical time point. Calculate the SHAP value of one or more indicators based on the future damage prediction results; A description of the causes of damage to wooden components is generated by combining the future damage prediction results of wooden components, the damage impact weight of more than one historical time point, and the SHAP value of more than one indicator. The damage cause description of the wooden component is matched with the decision rule base to obtain the maintenance decision instruction for the wooden component; the maintenance decision instruction for the wooden component is sent to the management terminal.

2. The intelligent management method for diagnosis and prevention of ancient building wooden components integrating digital twins as described in claim 1, characterized in that, Damage diagnosis models include: A backbone network is used to extract features at multiple scales from the main diagnostic surface image of wooden components; The neck network based on the feature pyramid fusion module fuses features from multiple scales output by the backbone network to obtain multiple fused features. The multi-task prediction head includes a damage type classification head, a damage severity regression head, and an aggregation unit; Among them, the damage type classification head generates a damage type probability map corresponding to each fusion feature; The damage level regression head generates a damage level map corresponding to each fused feature; The aggregation unit is used to aggregate the probability maps of damage types corresponding to multiple fusion features to obtain the damage type, and also to aggregate the damage degree maps corresponding to multiple fusion features to obtain the damage degree. The damage type and damage degree constitute the damage state identification result of the wooden component.

3. The intelligent management method for diagnosis and prevention of ancient building wooden components integrating digital twins as described in claim 1 or 2, characterized in that, The damage diagnosis model is obtained by training on an augmented image dataset, the method for constructing the augmented image dataset includes: Obtain original main diagnostic images of multiple wooden components of ancient buildings; The original main diagnostic surface image is preprocessed to obtain the main diagnostic surface image; Label the damage type on the main diagnostic image; Multiple real damage images are selected from multiple main diagnostic images, and the real damage images are segmented into damage area images and healthy background images. Physical feature labels are extracted from the damaged area image. These physical feature labels include crack main direction features, crack continuity features, decay texture level features, and decay diffusion morphology features. A pre-trained conditional generative adversarial network generator is used to generate a damage layer based on the physical feature labels of the real damaged image, the healthy background image, and the damage type label. The damage type label of the real damaged image is used as the damage type label of the damage layer. An augmented image dataset is composed of multiple damage layers, multiple master diagnostic images, and corresponding damage type labels.

4. The intelligent management method for diagnosis and prevention of ancient building wooden components integrating digital twins as described in claim 3, wherein the conditional generative adversarial network includes: The conditional encoder encodes the physical feature labels and damage type labels of the real damage image respectively and concatenates the encoding results to obtain conditional features. It extracts background features from the healthy background image and concatenates the conditional features and background features to obtain input features. A generator that produces damage layers based on input features; The discriminator calculates the generation loss based on the damage layer and the real damage image. During the training of the conditional generative adversarial network, the network parameters of the conditional encoder and generator are adjusted based on the generation loss.

5. The intelligent management method for diagnosis and prevention of ancient building wooden components integrating digital twins as described in claim 4, wherein the discriminator calculates and generates loss based on the damage layer and the actual damage image, including: Calculate the L2 norm of the pixel difference image between the damaged layer and the real damaged image, and use the L2 norm of the pixel difference image as the adversarial loss term; Segment the damaged region image from the damaged layer and extract physical feature labels from the damaged region image; Calculate the L1 norm of the deviation between the physical feature labels of the damage layer and the physical feature labels of the real damage image, and use the deviation as the physical label consistency loss term. The generation loss is obtained by combining the adversarial loss term and the physical label consistency loss term.

6. The intelligent management method for diagnosis and prevention of ancient building wooden components integrating digital twins as described in claim 3, characterized in that, The method for constructing the enhanced image dataset also includes: Perform spatial and / or spectral transformations on the main diagnostic image to obtain a transformation-enhanced image; The augmented image dataset is composed of multiple transformed enhanced images, multiple damage layers, multiple main diagnostic surface images, and corresponding damage type labels.

7. The intelligent management method for diagnosis and prevention of ancient building wooden components integrating digital twins as described in claim 1, 2, 4, 5, or 6, is characterized in that, The structured digital twin archive based on wooden components obtains the feature vectors of the wooden components at each historical time point, including: Based on the static attribute data of the wooden components at this historical time point, a static attribute sub-vector is generated. The static attribute sub-vector includes component code, spatial location, mechanical role, wood type, and initial material strength. A dynamic environmental subvector is generated based on the environmental exposure data of the wooden components at this historical time point. The dynamic environmental subvector includes at least one of the following: the average environmental temperature, average relative humidity, cumulative rainfall, cumulative solar radiation, and estimated average moisture content of the wood at this historical time point. Based on the damage status identification results of the wooden components at this historical time point, a damage status sub-vector is generated, which includes whether it is damaged, the damage type, and the degree of damage. Based on the maintenance data of the wooden components at this historical time point, a maintenance intervention sub-vector is generated. The maintenance intervention sub-vector includes whether maintenance is required and the maintenance type. By combining the static attribute sub-vectors, dynamic environment sub-vectors, damage state sub-vectors, and maintenance intervention sub-vectors of the spliced ​​wooden component at that historical time point, the feature vector of the wooden component at that historical time point is obtained.

8. The intelligent management method for diagnosis and prevention of ancient building wooden components integrating digital twins as described in claim 1, 2, 4, 5, or 6, is characterized in that, The multi-task time series prediction model includes: The encoder, including multi-layer LSTM units, extracts features from the feature vector of each historical time point in the three-dimensional feature tensor to obtain the hidden state of that historical time point. The hidden states of all historical time points within the historical time window form a hidden state sequence. The self-attention processing module uses a self-attention mechanism to obtain the attention weight of each hidden state in the hidden state sequence. The attention weight of the hidden state is used as the damage impact weight of the corresponding historical time point. The hidden state sequence is weighted and summed according to the attention weight of the hidden state to obtain the weighted hidden state sequence. The decoder obtains future damage prediction results based on the weighted latent state of the last historical time point in the weighted latent state sequence. The future damage prediction results include the probability of damage transition risk, damage type, and damage degree.

9. The intelligent management method for diagnosis and prevention of ancient building wooden components integrating digital twins as described in claim 8, characterized in that, The loss function during the training process of the multi-task temporal prediction model is: in, , , These represent the first weight, the second weight, and the third weight, respectively. Represents the binary cross-entropy function; Represents the cross-entropy function; Represents the mean square error function; , These represent the damage transition risk probability output by the multi-task time series prediction model and the label indicating whether a damage transition has occurred in the true prediction label, respectively. , These represent the damage type output by the multi-task time series prediction model and the damage type output by the damage diagnosis model for future time periods in the true prediction labels, respectively. , These represent the damage level output by the multi-task time-series prediction model and the damage level output by the damage diagnosis model for future time periods in the true prediction labels, respectively.

10. The intelligent management method for diagnosis and prevention of ancient building wooden components integrating digital twins as described in claim 1, 2, 4, 5, 6, or 9, characterized in that, The method also includes displaying the geometric model of the wooden components, the structural topology diagram in the ancient building, the damage status identification results of the wooden components, and the future damage prediction results in a visual interactive platform.

Images