House structure hidden danger intelligent detection method based on AI large model

Abstract

The application is specifically a housing structure hidden danger intelligent detection method based on an AI large model, relates to the technical field of building safety detection, and comprises: inputting the multi-modal spatiotemporal token sequence and the physical constraint parameter into a multi-modal large model to generate a hidden danger causal diagram in a self-recurrent manner, wherein the nodes of the hidden danger causal diagram are hidden danger events, and the edges are causal directions and confidence degrees. In the application, the multi-modal heterogeneous detection data is uniformly aligned to a component-level spatiotemporal coordinate and encoded into a structured token sequence, so that the multi-modal large model can perceive the accurate physical position and time attribute of each detection data, the technical problem that multi-source data lacks spatial correlation and thus cannot be used for cross-modal joint reasoning is solved, and the technical effect of supporting cross-modal joint reasoning of multi-modal data is achieved.

Description

Technical Field

[0001] This invention relates to the field of building safety inspection technology, and in particular to an intelligent detection method for potential structural hazards in buildings based on AI large-scale models. Background Technology

[0002] In the field of building safety inspection, with the development of artificial intelligence technology, image recognition-based building damage detection methods have been widely used. Existing methods mainly utilize convolutional neural networks or visual Transformers to perform end-to-end classification of building surface images, identifying apparent features such as the presence, type, and width of cracks.

[0003] However, the aforementioned prior art has the following drawbacks: Firstly, structural hazards in buildings have deep causal relationships and spatiotemporal evolution characteristics. Classifying and identifying them based solely on current surface images cannot trace the root cause and starting location of the damage. Secondly, existing methods output deterministic conclusions, lacking the quantification of uncertainties related to sensor noise and model cognitive limitations, making it difficult to provide reliable risk assessment basis for engineering maintenance decisions. These shortcomings prevent existing methods from meeting the engineering needs for reverse causal tracing, spatial progressive localization, and remaining lifespan prediction of building hazards. Summary of the Invention

[0004] The purpose of this invention is to propose an intelligent detection method for potential structural hazards in buildings based on an AI large model in order to solve the above-mentioned problems.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: Intelligent detection methods for potential structural hazards in buildings based on AI large-scale models include: Acquire multimodal heterogeneous detection data of the target building, associate the detection data of each modality with the same three-dimensional building information model, use a specialized encoder to generate feature vectors and attach modality type tokens and spatiotemporal location tokens, and map them to a unified semantic space to generate a multimodal spatiotemporal token sequence; Graph neural networks are used to embed entities and relationships in the building physics knowledge graph to generate physical constraint parameters. The multimodal spatiotemporal token sequence and the physical constraint parameters are input into the multimodal large model to generate a hazard causal graph in an autoregressive manner. The nodes of the hazard causal graph are hazard events, and the edges are causal directions and confidence levels. Uncertainty quantification is performed on each node in the hazard causal graph to generate confidence intervals for hazard locations and probability distributions for hazard severity. Based on the damage parameters and confidence intervals of the root cause of the hidden danger event, the remaining safe service life interval is output using a Gaussian process regression model.

[0006] Preferably, the generation of feature vectors using a specialized encoder includes: Z-score standardization was performed on the original data of each modality to eliminate the difference in dimensions. The standardized image data was processed by a visual Transformer encoder, the point cloud data was processed by a PointNet++ encoder, and the text data was processed by a pre-trained language encoder to generate feature vectors for each modality.

[0007] Preferably, the embedding calculation of entities and relationships in the building physics knowledge graph using a graph neural network includes: Using the feature vectors of entity nodes and relation edges in the building physics knowledge graph as input, and after multiple rounds of message passing aggregation, the embedding vectors of each entity node and the constraint weights of each relation edge are output. During training, a contrastive learning method is adopted, using known causal pairs in the building physics knowledge graph as positive samples and causal pairs that violate physical laws as negative samples, and using a triplet loss function to optimize the embedding space.

[0008] Preferably, after generating the physical constraint parameters, the process further includes: Acquire temperature, humidity, and carbon dioxide concentration data collected by on-site environmental sensors, adjust the material degradation rate parameter in the building physics knowledge graph based on the temperature, humidity, and carbon dioxide concentration data, and update the corresponding constraint weights in the physical constraint parameters.

[0009] Preferably, the step of generating the cause-effect graph of potential hazards in an autoregressive manner includes: The multimodal large model first identifies the most significant apparent damage as the end node of the causal chain, generating the end hidden event and its spatial coordinates; based on the end hidden event, it traces back layer by layer in the spatial direction from the outside to the inside, generating the preceding event node and its causal edge and confidence with the subsequent node in each layer, until the root hidden event is traced back.

[0010] Preferably, the training loss function of the multimodal large model is composed of a weighted sum of the cross-entropy loss generated by autoregression and a physical consistency penalty term; wherein, the physical consistency penalty term is the deviation metric between the attribute vector of the hazard event node output by the multimodal large model and the compliance attribute range specified by the corresponding causal rule in the building physics knowledge graph; when the attribute vector falls within the compliance range, the deviation metric is zero; when the attribute vector falls outside the compliance range, the deviation metric is the Euclidean distance from the attribute vector to the boundary of the compliance range; the average deviation metric of all nodes in the hazard causal graph is used as the physical consistency penalty term for the entire graph.

[0011] Preferably, the uncertainty quantification performed on each node in the hazard causal graph includes: In the multimodal large model inference stage, random discarding operations are enabled multiple times, forward computation is performed multiple times on the same input, and a set of hazard cause-effect graphs with multiple outputs is collected. The spatial coordinates and event attributes of each corresponding node in the hazard cause-effect graph set are statistically calculated, and the mean and variance of the spatial coordinates of each node are obtained as the location confidence interval, and the mean and variance of the event attribute values ​​of each node are obtained as the severity confidence interval.

[0012] Preferably, the step of using a Gaussian process regression model to output the remaining safe service life range includes: Extract the current damage parameter values ​​and their probability distribution of the root cause hazard event. Perform Z-score standardization on the damage parameter values, material corrosion rate parameters, and environmental factors respectively, and input them into the Gaussian process regression model to output the probability distribution curve of the damage parameter evolution over time. Based on the probability distribution curve, determine the time range within which the damage parameter reaches the safety threshold, which is taken as the remaining safe service life interval, and output the corresponding confidence level.

[0013] Preferably, the method further includes: The system receives corrective data submitted by structural engineers, including the deletion or addition of causal edges in the hazard causal graph and the relabeling of the root cause locations of hazards. The corrective data is then compared with the original output of the multimodal large model to form a comparison sample pair, which is stored in an incremental sample cache. When the number of samples in the incremental sample cache reaches a preset batch threshold, a low-rank adaptation method is used to update the parameters of the multimodal large model. During the update process, an experience replay method is used to synchronously input historical standard samples. The loss function for the parameter update is a weighted sum of the cross-entropy loss on the comparison sample pair and the knowledge distillation loss on the experience replay samples.

[0014] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention unifies the alignment of multimodal heterogeneous detection data to component-level spatiotemporal coordinates and encodes them into structured token sequences, enabling large multimodal models to perceive the precise physical location and temporal attributes of each detection data item. This solves the technical problem that the lack of spatial correlation among multi-source data prevents cross-modal joint reasoning, and achieves the technical effect of supporting cross-modal joint reasoning of multimodal data.

[0015] 2. This invention solves the technical problem that pure data-driven methods easily generate conclusions that violate physical logic by embedding structural mechanics and material degradation laws into the reasoning process in the form of differentiable constraints. It achieves the technical effect of constraining the output of large multimodal models to conform to physical laws. Through autoregressive causal tracing and spatiotemporal progressive positioning reasoning, it solves the technical problem that existing methods can only classify surface damage but cannot answer the questions of damage causes and initiation locations. Attached Figure Description

[0016] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0017] Several embodiments of this application will now be described in more detail with reference to the accompanying drawings to enable those skilled in the art to implement this application. This application may be embodied in many different forms and for various purposes and should not be limited to the embodiments set forth herein. These embodiments are provided to make this application thorough and complete, and to fully convey the scope of this application to those skilled in the art. The embodiments described do not limit this application.

[0018] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having a meaning consistent with their meaning in the relevant field and / or the context of this specification, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0019] Example 1 Its specific implementation method is combined with the appendix Figure 1 Please provide a detailed explanation.

[0020] In this embodiment, it includes: In a city-wide safety survey of existing residential buildings, the inspection team conducted a comprehensive safety inspection of a six-story masonry structure residential building (Building B7) built in the 1990s. Multiple visible cracks had appeared on the building's exterior walls, and some areas showed signs of water seepage. The inspection team used a drone equipped with a visible light camera and an infrared thermal imager to scan the facade. Simultaneously, crack gauges and strain gauges were installed on key components, and ultrasonic detectors were used to inspect the interior of the walls. The inspection took place in the autumn of 20XX, with an ambient temperature of 18 degrees Celsius, a relative humidity of 73%, and a carbon dioxide concentration of 520 ppm. The key component inspected was the south-facing exterior wall of the second floor (component code: W2-S).

[0021] The detection equipment collected six types of raw data from building B7, including visible light images, infrared thermal images, laser point clouds, ultrasonic echo signals, crack timing readings, and historical maintenance records. Taking component W2-S as an example, visual-inertial positioning technology was used to associate each modal data with a 3D building information model, obtaining precise 3D spatial coordinates and component identification for each data point. Before being input into the specialized encoder, the raw data for each modality underwent Z-score standardization to eliminate the influence of differences in the numerical ranges of each modality. The original amplitude range of the ultrasonic echo signal was approximately 0 to 4500 mV, and the image pixel value range was 0 to 255; after standardization, both were mapped to a distribution with a mean of 0 and a standard deviation of 1. After attaching modality type tokens and spatiotemporal location tokens to the feature vectors of each modality, they were mapped to a unified semantic space, forming a multimodal spatiotemporal token sequence.

[0022] Table 1 Overview of W2-S Multimodal Detection Data for Components A subgraph related to component W2-S was extracted from the building physics knowledge graph. This subgraph includes the topological adjacent components of W2-S (second-floor slab W2-FL, adjacent longitudinal walls W2-E and W2-W), the force flow transmission path (upper-floor load transmitted to W2-S via the slab and then to the foundation), and the causal chain of typical defects in the masonry structure's exterior wall. On-site environmental sensors measured a temperature of 18℃, relative humidity of 73%, and carbon dioxide concentration of 520ppm. Based on these measurements, the concrete carbonation rate parameter in the knowledge graph was adjusted from the baseline value of 0.18mm / year to 0.24mm / year, and the corresponding constraint weights were updated. After message passing calculations, the graph neural network outputs the embedding vectors of each entity node and the constraint weights of each relation edge as physical constraint parameters.

[0023] Table 2 Key Causal Chains of the W2-S Physical Knowledge Subgraph for Components The multimodal large model receives a concatenated sequence of multimodal spatiotemporal tokens and physical constraint parameters, and generates a hazard causal graph in an autoregressive manner. The decoder first identifies the most significant apparent damage: visible light images show a crack approximately 680 mm long and nearly vertically oriented on the W2-S surface, with spatial coordinates ranging from [missing information]. , The model generates the final node "Vertical Crack in External Wall (Node E1)". Subsequently, the model traces inward layer by layer, generating preceding event nodes. When generating "Reinforcement Corrosion (Node C2)", the attribute vector of this node is calculated. With the knowledge graph compliance scope center vector Deviation, inconsistent scores If the value is below the preset threshold of 0.15, the node is retained. When generating the initial version of "Foundation Settlement (Node C0 - Initial)", If the threshold is exceeded, the penalty term is backpropagated to the output distribution, and the model is corrected to "supercritical value of protective layer carbonization (node ​​C0)". This conforms to the laws of physics.

[0024] Based on the root cause node C0, the hazard development sequence was simulated in a positive direction. It was compared with the historical detection records of slight efflorescence in year A and microcracks in year B. The difference metric was 0.09, which was the minimum among the candidate paths. The corresponding causal graph was determined as the final hazard causal graph.

[0025] Table 3. Node information in the cause-and-effect diagram of potential hazards in component W2-S. Execute on the same input during the inference phase After randomly discarding the forward computation and introducing three model replicas with slightly different parameters, a total of 42 sets of hazard cause-effect graphs were obtained. Statistical calculations were performed on the spatial coordinates and event attributes of each node to obtain location confidence intervals and severity confidence intervals. Taking the root cause node C0 as an example, the mean carbonization depth attribute of the 30 randomly discarded outputs was 14.7 mm with a standard deviation of 1.3 mm; after fusing the model replica outputs, the mean was updated to 14.5 mm, and the standard deviation narrowed to 1.1 mm, corresponding to a 95% confidence interval of 12.4 mm to 16.6 mm.

[0026] Table 4. Quantification Results of Uncertainty at Key Nodes The mean carbonization depth of the root node C0 (14.5 mm) and its probability distribution were extracted. Combined with the environmentally adjusted carbonization rate parameter of 0.24 mm / year and environmental factors (temperature 18℃, relative humidity 73%), the data were standardized by Z-score and input into a Gaussian process regression model. The model was trained using time-series damage monitoring data of similar historical masonry structural members, and outputs the probability distribution curve of carbonization depth evolution over time.

[0027] Using a 20mm thick concrete cover as the safety threshold, the time range for reaching a carbonation depth of 20mm was determined based on the probability distribution curve. Gaussian process regression yielded the probability distribution of the remaining time for the damage parameters to reach the threshold, with a mean of 23.4 years and a 90% confidence interval of 18.1 to 29.2 years.

[0028] Table 5. Predicted Remaining Safe Service Life The multimodal large model simultaneously generates an attention heatmap and a key evidence index when outputting the causal graph of potential hazards. The attention heatmap shows that the ultrasonic echo token has the highest contribution weight to the inference of node C0 (0.41), followed by the infrared thermographic token to node C1 (0.35). The key evidence index lists the specific evidence supporting the causal edge C1→C2 as the temperature anomaly point at coordinates (3.20, 0.00, 4.85) in the infrared thermographic image and the characteristic of reduced wave velocity of the ultrasonic echo.

[0029] After review, the structural engineer determined that an intermediate node should be added between nodes C2 and E1, via a path of "accelerated corrosion due to moisture intrusion," and that the lower limit of the confidence interval for the carbonization depth of the root node C0 should be raised from 12.4 mm to 13.0 mm. This correction was formatted as a comparison sample pair and stored in the incremental sample cache. Once the cache accumulates to a preset batch threshold, a low-rank adaptation method is used to update the parameters, with an update loss of [missing information]. ,in To compare learning loss, For knowledge distillation loss, To balance the coefficients, historical standard samples were synchronously input during the experience replay. After the update, regression tests were performed on the standard test set. If the performance metrics did not fall below the pre-update level, the model version was officially retained.

[0030] Table 6 Incremental Learning Samples and Update Process Records Throughout the implementation process, data originates from six types of original detection signals. After spatiotemporal alignment and standardization, a multimodal spatiotemporal token sequence is formed (Step 1). This sequence is then combined with physical constraint parameters extracted from the knowledge graph and dynamically adjusted based on environmental data (Step 2). This drives the multimodal large model to autoregressively generate a hazard causal graph that has undergone physical consistency verification (Step 3). After multiple random discarding and model integration, the confidence intervals for each node are obtained (Step 4). Finally, the damage parameters and confidence intervals of the root node are input into the Gaussian process regression model to output the remaining safe service life interval (Step 5). Expert-corrected data is fed back into the model parameter update in the form of incremental samples (Step 6), forming a complete closed-loop data flow from original detection data to reliable risk assessment conclusions.

[0031] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0032] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0033] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for intelligent detection of structural hazards in buildings based on AI large-scale models, characterized in that, include: Acquire multimodal heterogeneous detection data of the target building, associate the detection data of each modality with the same three-dimensional building information model, use a specialized encoder to generate feature vectors and attach modality type tokens and spatiotemporal location tokens, and map them to a unified semantic space to generate a multimodal spatiotemporal token sequence; Graph neural networks are used to embed entities and relationships in the building physics knowledge graph to generate physical constraint parameters. The multimodal spatiotemporal token sequence and the physical constraint parameters are input into the multimodal large model to generate a hazard causal graph in an autoregressive manner. The nodes of the hazard causal graph are hazard events, and the edges are causal directions and confidence levels. Uncertainty quantification is performed on each node in the hazard causal graph to generate confidence intervals for hazard locations and probability distributions for hazard severity. Based on the damage parameters and confidence intervals of the root cause of the hidden danger event, the remaining safe service life interval is output using a Gaussian process regression model.

2. The intelligent detection method for hidden dangers in building structures based on AI large model according to claim 1, characterized in that, Generating feature vectors using a specialized encoder includes: Z-score standardization was performed on the original data of each modality to eliminate the difference in dimensions. The standardized image data was processed by a visual Transformer encoder, the point cloud data was processed by a PointNet++ encoder, and the text data was processed by a pre-trained language encoder to generate feature vectors for each modality.

3. The intelligent detection method for hidden dangers in building structures based on AI large model according to claim 1, characterized in that, Embedding computations of entities and relationships in a building physics knowledge graph using graph neural networks include: Using the feature vectors of entity nodes and relation edges in the building physics knowledge graph as input, and after multiple rounds of message passing aggregation, the embedding vectors of each entity node and the constraint weights of each relation edge are output. During training, a contrastive learning method is adopted, using known causal pairs in the building physics knowledge graph as positive samples and causal pairs that violate physical laws as negative samples, and using a triplet loss function to optimize the embedding space.

4. The intelligent detection method for hidden dangers in building structures based on AI large model according to claim 1, characterized in that, After generating the physical constraint parameters, the following is also included: Acquire temperature, humidity, and carbon dioxide concentration data collected by on-site environmental sensors, adjust the material degradation rate parameter in the building physics knowledge graph based on the temperature, humidity, and carbon dioxide concentration data, and update the corresponding constraint weights in the physical constraint parameters.

5. The intelligent detection method for hidden dangers in building structures based on AI large model according to claim 1, characterized in that, Generating hazard cause-effect graphs using an autoregressive approach includes: The multimodal large model first identifies the most significant apparent damage as the end node of the causal chain, generating the end hidden event and its spatial coordinates; based on the end hidden event, it traces back layer by layer in the spatial direction from the outside to the inside, generating the preceding event node and its causal edge and confidence with the subsequent node in each layer, until the root hidden event is traced back.

6. The intelligent detection method for hidden dangers in building structures based on AI large model according to claim 1, characterized in that, The training loss function of the multimodal large model consists of a weighted sum of the cross-entropy loss generated by autoregression and a physical consistency penalty term. The physical consistency penalty term is a deviation metric between the attribute vector of the hazard event node output by the multimodal large model and the compliance attribute range specified by the corresponding causal rule in the building physics knowledge graph. When the attribute vector falls within the compliance range, the deviation metric is zero; when the attribute vector falls outside the compliance range, the deviation metric is the Euclidean distance from the attribute vector to the boundary of the compliance range. The average deviation metric of all nodes in the hazard causal graph is used as the physical consistency penalty term for the entire graph.

7. The intelligent detection method for hidden dangers in building structures based on AI large model according to claim 1, characterized in that, Uncertainty quantification is performed on each node in the cause-and-effect diagram of potential hazards, including: In the multimodal large model inference stage, random discarding operations are enabled multiple times, forward computation is performed multiple times on the same input, and a set of hazard cause-effect graphs with multiple outputs is collected. The spatial coordinates and event attributes of each corresponding node in the hazard cause-effect graph set are statistically calculated, and the mean and variance of the spatial coordinates of each node are obtained as the location confidence interval, and the mean and variance of the event attribute values ​​of each node are obtained as the severity confidence interval.

8. The intelligent detection method for hidden dangers in building structures based on AI large model according to claim 1, characterized in that, The remaining safe service life range output using the Gaussian process regression model includes: Extract the current damage parameter values ​​and their probability distribution of the root cause hazard event. Perform Z-score standardization on the damage parameter values, material corrosion rate parameters, and environmental factors respectively, and input them into the Gaussian process regression model to output the probability distribution curve of the damage parameter evolution over time. Based on the probability distribution curve, determine the time range within which the damage parameter reaches the safety threshold, which is taken as the remaining safe service life interval, and output the corresponding confidence level.

9. The intelligent detection method for hidden dangers in building structures based on AI large model according to claim 1, characterized in that, Also includes: Receive the correction data submitted by the structural engineer, which includes the deletion or addition of causal edges in the cause-effect diagram of the hidden dangers and the re-labeling of the root cause location of the hidden dangers. The corrected data and the original output of the multimodal large model are used to form a comparison sample pair, which is stored in the incremental sample cache. When the number of samples in the incremental sample cache reaches a preset batch threshold, the low-rank adaptation method is used to update the parameters of the multimodal large model. During the update process, the experience replay method is used to synchronously input historical standard samples. The loss function of the parameter update is composed of the weighted sum of the cross-entropy loss on the comparison sample pair and the knowledge distillation loss on the experience replay samples.

Images