Building rectification monitoring method and device based on multi-modal perception and digital twinning

Abstract

This disclosure presents a method and apparatus for monitoring and correcting building deviations based on multimodal sensing and digital twins. One specific implementation of the method includes: acquiring multi-source sensing data of the building to be monitored; constructing a differential settlement model based on the multi-source sensing data and a preset building design model; extracting features from a preset building feature map to obtain a global feature map and a cognitive uncertainty map; performing semantic segmentation on the building structure map to obtain a geometric prior weight map; performing energy flow analysis on the differential settlement model to generate an energy flow density map; constructing a composite sampling guidance vector field based on the geometric prior weight map, the cognitive uncertainty map, and the energy flow density map; generating a local three-dimensional feature set based on the composite sampling guidance vector field; and performing defect detection on the local three-dimensional feature set and the global feature map to obtain a defect information set. This implementation can improve the efficiency of building deviation correction and ensure building stability.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of building foundation reinforcement and correction technology, specifically to a building correction monitoring method and apparatus based on multimodal perception and digital twins. Background Technology

[0002] In building construction, building correction is a key means to eliminate safety hazards and ensure the stability of the building. Currently, the common method used for monitoring building correction is the collaborative control and treatment method of differential settlement spectral density energy flow in high-rise buildings, which achieves deformation monitoring through real-time disturbance spectral density energy analysis.

[0003] However, when the above methods are used to monitor the deviation of buildings, a common technical problem is the lack of in-depth integration of multi-dimensional data such as groundwater level, soil pressure, and micro-cracks in welds. During the deviation correction process, it is difficult to accurately locate key areas with weak geometric structures and uncertain future deformations. This makes it difficult for the disturbance energy flow to accurately correspond to specific structural weak points, thereby affecting the deviation correction efficiency of the building and reducing the stability of the building structure. Summary of the Invention

[0004] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0005] Some embodiments of this disclosure propose a building correction monitoring method and apparatus based on multimodal perception and digital twins to solve the technical problems mentioned in the background section above.

[0006] In a first aspect, some embodiments of this disclosure provide a building correction monitoring method based on multimodal perception and digital twins. The method includes: acquiring multi-source perception data of the building to be monitored; constructing a differential settlement model based on the multi-source perception data and a preset building design model; extracting features from a preset building feature map to obtain a global feature map and a cognitive uncertainty map; performing semantic segmentation on the building structure map of the building to be monitored to obtain a geometric prior weight map; performing energy flow analysis on the differential settlement model to generate an energy flow density map; constructing a composite sampling guidance vector field based on the geometric prior weight map, the cognitive uncertainty map, and the energy flow density map; generating a local three-dimensional feature set based on the composite sampling guidance vector field; and performing defect detection on the local three-dimensional feature set and the global feature map to obtain a defect information set, wherein the defect information in the defect information set includes defect category and defect location.

[0007] Secondly, some embodiments of this disclosure provide a building correction monitoring device based on multimodal perception and digital twins. The device includes: an acquisition unit configured to acquire multi-source perception data of the building to be monitored, and to construct a differential settlement model based on the multi-source perception data and a preset building design model; a feature extraction unit configured to extract features from a preset building feature map to obtain a global feature map and a cognitive uncertainty map; a semantic segmentation unit configured to perform semantic segmentation on the building structure map of the building to be monitored to obtain a geometric prior weight map; an energy flow analysis unit configured to perform energy flow analysis on the differential settlement model to generate an energy flow density map; a construction unit configured to construct a composite sampling guidance vector field based on the geometric prior weight map, the cognitive uncertainty map, and the energy flow density map; a generation unit configured to generate a local three-dimensional feature set based on the composite sampling guidance vector field; and a defect detection unit configured to perform defect detection on the local three-dimensional feature set and the global feature map to obtain a defect information set, wherein the defect information in the defect information set includes defect category and defect location.

[0008] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0009] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0010] The above-described embodiments of this disclosure have the following beneficial effects: The building correction and monitoring method based on multimodal perception and digital twins, as described in some embodiments of this disclosure, can improve the stability of building structures. Specifically, the reason for insufficient building stability lies in the lack of deep integration of multi-dimensional data such as groundwater level, soil pressure, and micro-cracks in welds. During the correction process, it is difficult to accurately locate key areas with weak geometric structures and uncertain future deformation, resulting in the disturbance energy flow failing to accurately correspond to specific structural weak points, thus affecting the building's correction efficiency. Based on this, the building correction and monitoring method based on multimodal perception and digital twins, as described in some embodiments of this disclosure, firstly acquires multi-source perception data of the building to be monitored, and constructs a differential settlement model based on the multi-source perception data and a preset building design model. By combining the building's multi-source data with the building design model, the true state of the building can be dynamically reflected, reducing the impact of single-dimensional data on the accuracy of correction. Secondly, feature extraction is performed on the preset building feature map to obtain a global feature map and a cognitive uncertainty map. Feature extraction yields global features of the building and reduces the "uncertainty" of the convolutional network's predictions for a specific area, thus lowering the false negative rate in bias correction detection. Next, semantic segmentation is performed on the building structure diagram to obtain a geometric prior weight map. This allows the introduction of physical prior knowledge, increasing the weights of key structural points and geometrically weak points to improve bias correction efficiency. Then, energy flow analysis is performed on the differential settlement model to generate an energy flow density map. This allows the location of the most stress-concentrated parts of the building. Following this, a composite sampling guidance vector field is constructed based on the geometric prior weight map, the cognitive uncertainty map, and the energy flow density map. This allows the location of geometrically weak, stress-concentrated, or highly uncertain areas in the building, avoiding blind allocation of bias correction resources and difficulty in accurately identifying weak points. Then, a local three-dimensional feature set is generated based on the composite sampling guidance vector field. This allows focusing on the local three-dimensional features of important areas, improving bias correction efficiency and accuracy. Finally, defect detection is performed on the local three-dimensional feature set and the global feature map to obtain a defect information set. The aforementioned defect information includes defect category and defect location. By deeply integrating global and local features, accurate defect classification and location are achieved, improving the reliability of building correction and ensuring the stability of the building structure. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0012] Figure 1 This is a schematic diagram of an application scenario of a building correction monitoring method based on multimodal perception and digital twin, which is one of the embodiments of this disclosure;

[0013] Figure 2 This is a flowchart of some embodiments of the building correction monitoring method based on multimodal sensing and digital twin according to the present disclosure;

[0014] Figure 3 This is a flowchart of the defect detection method for building correction monitoring based on multimodal perception and digital twin according to this disclosure;

[0015] Figure 4 These are schematic diagrams of some embodiments of a building correction monitoring device based on multimodal sensing and digital twin according to this disclosure;

[0016] Figure 5 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0019] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0020] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0021] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0022] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] Figure 1 This is a schematic diagram of an application scenario of a building correction monitoring method based on multimodal perception and digital twin, which is one of the embodiments of this disclosure.

[0024] exist Figure 1 In the application scenario, firstly, the computing device 101 can acquire multi-source sensing data 103 of the building 102 to be monitored, and construct a differential settlement model 105 based on the multi-source sensing data 103 and a preset building design model 104; perform feature extraction on the preset building feature map 106 to obtain a global feature map 107 and a cognitive uncertainty map 108; perform semantic segmentation on the building structure map 109 of the building 102 to be monitored to obtain a geometric prior weight map 110; perform energy flow analysis on the differential settlement model 105 to generate an energy flow density map 111; construct a composite sampling guidance vector field 112 based on the geometric prior weight map 110, the cognitive uncertainty map 108, and the energy flow density map 111; generate a local three-dimensional feature body 113 based on the composite sampling guidance vector field 112; and perform defect detection on the local three-dimensional feature body 113 and the global feature map 107 to obtain defect information 114.

[0025] It should be noted that the aforementioned computing device 101 can be either hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster composed of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is software, it can be installed in the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here. It should be understood that... Figure 1 The number of computing devices in the system can be arbitrary, depending on the implementation requirements.

[0026] Continue to refer to Figure 2 The diagram illustrates a flow 200 of some embodiments of a building deviation correction monitoring method based on multimodal sensing and digital twins according to this disclosure. This building deviation correction monitoring method based on multimodal sensing and digital twins includes the following steps:

[0027] Step 201: Obtain multi-source sensing data of the building to be monitored, and construct a differential settlement model based on the multi-source sensing data and the preset building design model.

[0028] In some embodiments, the implementing entity of the building correction monitoring method based on multimodal sensing and digital twin (e.g.) Figure 1The computing device 101 shown can acquire multi-source sensing data of the building to be monitored, and construct a differential settlement model based on the multi-source sensing data and a preset building design model. The multi-source sensing data may include: three-dimensional deformation monitoring data, internal force and strain monitoring data, and environmental and geological sensing data. The three-dimensional deformation monitoring data may be data collected by a GNSS receiver deployed on the building to be monitored. The internal force and strain monitoring data may be data collected by strain gauges deployed on the building to be monitored. The environmental and geological sensing data may be data collected by pressure gauges embedded in the foundation. The building design model may be a BIM model of the building to be monitored.

[0029] As an example, the aforementioned three-dimensional deformation monitoring data can be acquired by three or more dual-frequency multi-system GNSS receivers at feature points on the top of the building and surrounding stability reference points. The sampling frequency can be 1Hz, outputting millimeter-level three-dimensional coordinates for calculating the building's real-time settlement ΔZ and tilt rate Δθ. The aforementioned internal force and strain monitoring data can be acquired by fiber optic strain gauges uniformly distributed circumferentially along the cross-section at heights of 0.5m, 1.5m, and 2.5m in the bottom-floor frame columns and shear walls. The sampling frequency is 10Hz, outputting real-time strain values ​​ε. The aforementioned environmental and geological sensing data can be acquired by pore water pressure gauges and earth pressure cells embedded in the foundation. The sampling frequency is 1Hz, outputting groundwater level change ΔW and earth pressure value P_soil.

[0030] In practice, firstly, the aforementioned implementing entity can use the layered superposition method to calculate the vertical load design values ​​of each component at the bottom layer by transferring the dead load, live load, and additional load downwards from the top layer. These vertical load design values ​​are then added to the corresponding components in the building design model. Next, the stress relief method can be used to drill holes and deduce the initial actual values ​​of the vertical loads and initial strains of the components. These initial actual values ​​and initial strains are then added to the corresponding components in the building design model. Finally, the real-time settlement data from the multi-source sensing data can be used as boundary conditions, and the real-time strain values ​​as the model calibration basis to adjust the building design model, resulting in an adjusted finite element numerical calculation model, which serves as the differential settlement model.

[0031] Step 202: Extract features from the preset building feature map to obtain a global feature map and a cognitive uncertainty map.

[0032] In some embodiments, the aforementioned execution entity can extract features from a preset building feature map to obtain a global feature map and a cognitive uncertainty map. The building feature map can be a multi-channel two-dimensional feature map that integrates the GNSS deformation field, BIM model, and initial strain distribution. Specifically, firstly, the BIM model of the building to be monitored can be projected onto a top-down view to obtain a building top view. Next, the real-time settlement and tilt rate measured by a GNSS receiver deployed on the top of the building can be mapped to each pixel of the two-dimensional image using a spatial interpolation algorithm to obtain a deformation field image. Secondly, the real-time strain values ​​measured by fiber optic strain gauges deployed on the key components of the building's lower level can be spatially interpolated and mapped to the two-dimensional image to form a strain distribution image. Finally, the building top view, deformation field image, and strain distribution image are stitched together by channel dimension to obtain the building feature map. The dimensions of the building feature map can be 1024×1024×C, where C is a numerical value representing the number of channels.

[0033] In practice, firstly, the aforementioned execution entity can perform deep feature encoding on a preset building feature map based on a pre-defined encoder to obtain a global feature map. This encoder can be a first deep convolutional network based on a ResNet-50 encoder within a U-Net architecture. The dimensions of the global feature map can be 128×128×512. Secondly, through the decoder of this first deep convolutional network, transposed convolutions and skip connections can be performed on the global feature map, and an initial defect prediction confidence map of the same size as the building feature map is output by a 1×1 convolution and a sigmoid activation function at the end. The pixel values ​​in this initial defect prediction confidence map can be between 0 and 1. The pixel values ​​in this initial defect prediction confidence map can represent the probability of high-risk damage or differential settlement at the corresponding spatial location. Finally, this initial defect prediction confidence map can be defined as a cognitive uncertainty map. Here, the lower the confidence value of a pixel in the initial defect prediction confidence map, the higher the cognitive uncertainty corresponding to that pixel.

[0034] In practice, a common technical challenge in monitoring building deviations is that environmental factors such as strong winds or sudden temperature changes can cause micro-deformation of building materials and fatigue cracking of welds, resulting in significant noise in the monitoring data. This increases the cognitive uncertainty of the convolutional network for feature extraction, thereby reducing the accuracy of the deviation monitoring. Therefore, the following solution is proposed.

[0035] Optionally, the aforementioned execution entity may perform feature extraction on the preset building feature map to obtain a global feature map and a cognitive uncertainty map, and may further include the following steps:

[0036] The first step involves performing deep feature encoding on the preset building feature map using a pre-defined convolutional network to obtain a global feature map. In practice, the execution entity can first perform deep feature encoding on the preset building feature map using a pre-defined encoder to obtain the global feature map. This encoder can be the encoder of the first deep convolutional network in the U-Net architecture based on the ResNet-50 encoder.

[0037] The second step is to generate forward propagation coefficients based on the feature signal-to-noise ratio (SNR) corresponding to the global feature map. In practice, firstly, the number of salient features in the global feature map can be determined using gradient-weighted computational mapping (Grad-CAM). Next, the number of salient features can be subtracted from the total number of features in the global feature map to obtain the number of insignificant features. Finally, the feature SNR corresponding to the global feature map can be determined using the following formula.

[0038] SNR=10×log 10 (Number of significant features / Number of non-significant features).

[0039] SNR represents the feature signal-to-noise ratio.

[0040] Finally, the forward propagation coefficient corresponding to the aforementioned characteristic SNR can be determined according to a preset SNR-forward propagation coefficient mapping table. This mapping table represents the correspondence between characteristic SNRs and forward propagation coefficients. For example, a forward propagation coefficient of 10 corresponds to an SNR ≥ 20dB (low noise); a forward propagation coefficient of 30 corresponds to an SNR ≤ 20dB (medium noise); and a forward propagation coefficient of 50 corresponds to an SNR < 10dB (high noise).

[0041] Third, based on the aforementioned convolutional network and forward propagation coefficients, random forward propagation is performed on the building feature map to generate a prediction confidence map set. In practice, the forward propagation coefficients can be used as the number of passes, the building feature map can be input into the convolutional network, and Monte Carlo Dropout (MC Dropout) in the convolutional network can be kept active. The confidence map output by the convolutional network can be used as the prediction confidence map. The size of the prediction confidence map can be the same as the size of the building feature map.

[0042] As an example, the building feature maps described above can be input into the convolutional network, and during inference, the neurons in the convolutional network can be randomly deactivated, for example, in the `model.train()` mode. The convolutional network can then output prediction confidence maps. Assuming the forward propagation coefficient is 50, 50 inference operations will be performed, resulting in 50 prediction confidence maps.

[0043] Fourth, for each pixel in the building feature map, the statistical variance between the corresponding pixel values ​​in the prediction confidence map set is determined as the pixel variance. In practice, for each pixel (i, j) in the building feature map, the statistical variance between the element values ​​at each prediction confidence map (i, j) position in the prediction confidence map set can be determined as the pixel variance of pixel (i, j).

[0044] The fifth step is to combine the determined pixel variances into a first cognitive uncertainty map. In practice, the determined pixel variances can be combined into a first cognitive uncertainty map based on the position corresponding to each pixel variance.

[0045] Step 6: Input the aforementioned global feature map into a preset evidence head network to obtain a set of normal inverse gamma distribution parameters. This evidence head network may include 1×1 convolutional layers and activation functions (e.g., Softplus). The evidence head network can transform the size of the global feature map to the same size as the building feature map, and the number of channels is transformed to 4. The normal inverse gamma distribution parameters in the aforementioned set include: shape parameters, scale parameters, inverse gamma shape, and inverse gamma scale. Each of the normal inverse gamma distribution parameters in the aforementioned set corresponds one-to-one with a pixel in the building feature map.

[0046] Step 7: Based on the above-mentioned normal inverse gamma distribution parameter set, generate a second cognitive uncertainty map. In practice, for each pixel in the above building feature map, the second cognitive uncertainty corresponding to the pixel can be determined by the following formula.

[0047] P unc (i,j)=β(i,j) / (ν(i,j)·(α(i,j)-1)).

[0048] Among them, P unc (i,j) represents the cognitive uncertainty corresponding to pixel (i,j). β(i,j) represents the inverse gamma scale corresponding to pixel (i,j). ν(i,j) represents the scale parameter corresponding to pixel (i,j). α(i,j) represents the inverse gamma shape corresponding to pixel (i,j).

[0049] Finally, the determined individual second cognitive uncertainties can be merged into a second cognitive uncertainty map according to pixel position.

[0050] Step 8: Generate a cognitive uncertainty map based on the first and second cognitive uncertainty maps described above. In practice, the first and second cognitive uncertainty maps can be added element by element to obtain the cognitive uncertainty map.

[0051] Here, the value of each pixel in the cognitive uncertainty graph represents the "cognitive uncertainty" of the convolutional network's prediction of the pixel. The larger the pixel value, the more ambiguous the convolutional network's prediction of the region corresponding to the pixel, and the more likely it is to contain unknown defect patterns not seen in its training set.

[0052] To improve the accuracy of bias correction monitoring, firstly, a global feature map corresponding to the building feature map is determined using a convolutional network. Next, the number of forward propagations is dynamically adjusted based on the signal-to-noise ratio (SNR) of the global feature map to adapt to noise fluctuations in building monitoring scenarios, reducing redundant computation in high SNR scenarios and improving real-time performance in low SNR scenarios. Then, based on the aforementioned number of forward propagations, the building feature map is input into the convolutional network for random forward propagation to generate a first cognitive uncertainty map. This allows quantification of the "uncertainty" of the prediction results by the convolutional network. However, using random forward propagation to generate the cognitive uncertainty map has the drawback of not being able to reliably distinguish the source of uncertainty for out-of-distribution (OOD) samples. By integrating Deep Evidence Regression (DER) into the feature extraction process, a second cognitive uncertainty map can be directly output, avoiding computational redundancy and calibration failure issues from multiple forward propagations. Fusing the first and second cognitive uncertainty maps improves the robustness of the generated cognitive uncertainty map, reduces noise interference, and thus improves the accuracy of bias correction monitoring.

[0053] Step 203: Perform semantic segmentation on the building structure diagram of the above-mentioned building to be monitored to obtain a geometric prior weight map.

[0054] In some embodiments, the aforementioned execution entity can perform semantic segmentation on the building structure diagram of the building to be monitored to obtain a geometric prior weight map. The building structure diagram can be a two-dimensional image reflecting the spatial distribution of key geometric structures (such as welds, joints, supports, etc.) of the building to be monitored. In practice, firstly, semantic segmentation can be performed on the building structure diagram based on a preset semantic segmentation network to obtain a set of semantic segmentation regions. The semantic segmentation network can be U-Net++. The semantic segmentation regions in the set of semantic segmentation regions can be the regions covered by at least one pixel in the building structure diagram. The semantic segmentation regions in the set of semantic segmentation regions can correspond to region categories. The region categories can include: weld region, joint region, support region, and ordinary structure region. The region categories can correspond to preset region weights. For example, the weights corresponding to weld region, joint region, support region, and ordinary structure region are 0.42, 0.38, 0.16, and 0.04, respectively. Next, the region category corresponding to each pixel in the building structure diagram can be converted into a region weight to obtain a geometric prior weight map.

[0055] Optionally, the execution entity performs semantic segmentation on the building structure diagram of the building to be monitored to obtain a geometric prior weight map, which may include the following steps:

[0056] The first step is to segment the building structure diagram of the building to be monitored, obtaining a set of segmented regions. Each segmented region in this set can be an area covered by at least one pixel in the building structure diagram. Each segmented region in the set corresponds to a region category. The set of segmented regions can include: weld areas, splice seam areas, and support areas. In practice, the building structure diagram of the building to be monitored can be segmented based on the semantic segmentation network described above to obtain the set of segmented regions.

[0057] The second step is to perform the following steps for each segmented region in the above segmented region set:

[0058] The first sub-step involves extracting the morphological skeleton from the segmented region to obtain the morphological centerline. In practice, this can be done first using the Canny edge detection algorithm to perform edge detection on the segmented region, obtaining the edge detection results. Then, a morphological skeletonization algorithm can be used to extract the centerline from the edge detection results, obtaining the morphological centerline. The morphological skeletonization algorithm can be the Zhang-Suen algorithm. The morphological centerline can be a line segment from the building structure diagram.

[0059] The second sub-step involves determining the shortest distance between each pixel in the segmented region and the morphological center line as the pixel center distance, thus obtaining a set of pixel center distances. In practice, for each pixel in the segmented region, the shortest Euclidean distance between the pixel and the morphological center line can be determined as the pixel center distance.

[0060] The third sub-step involves determining the mapping weights corresponding to each pixel center distance in the aforementioned pixel center distance set, based on a preset Gaussian kernel function, thus obtaining a mapping weight set. In practice, this can be achieved by using the Gaussian kernel function to determine the mapping weights corresponding to each pixel center distance in the aforementioned pixel center distance set, as shown in the following formula.

[0061] .

[0062] in, Represents pixels The corresponding mapping weights. d(p) represents the pixel. The distance between the center of pixels. This indicates that 3.0 pixels are being used.

[0063] The third step is to generate a geometric prior weight map based on the obtained mapping weight sets. In practice, firstly, a blank map with the same dimensions as the building structure drawing can be constructed. Then, each mapping weight in the obtained mapping weight sets can be used as the pixel value at the corresponding position in the blank map to obtain the geometric prior weight map.

[0064] In practice, the closer a pixel is to the center line of the geometric structure, the closer its Pgeo value is to 1; the farther a pixel is from the center line, the lower its value becomes (to 0). The geometric prior weight map can characterize the physical prior of the structural importance.

[0065] Step 204: Perform energy flow analysis on the above differential settlement model to generate an energy flow density map.

[0066] In some embodiments, the aforementioned executing entity can perform energy flow analysis on the differential settlement model to generate an energy flow density map. This energy density map can be a two-dimensional image. In practice, firstly, energy flow analysis can be performed on the differential settlement model based on a preset spectral density energy flow analysis technique to generate an energy flow density map. This spectral density energy flow analysis technique can be a method for coordinated control and treatment of differential settlement spectral density energy flow in high-rise buildings.

[0067] Optionally, the aforementioned implementing entity performs energy flow analysis on the differential settlement model to generate an energy flow density map, which may include the following steps:

[0068] The first step is to generate the corresponding perturbation strain signal based on the preset excitation signal and the aforementioned differential settlement model. In practice, firstly, a multi-directional Gaussian white noise excitation signal can be input to the substrate of the differential settlement model. Then, the perturbation strain signal of the differential settlement model in response to the excitation signal can be determined.

[0069] The second step is to determine the real-time perturbation spectral density energy distribution corresponding to the excitation signal and the perturbation strain signal. In practice, firstly, the excitation signal and the perturbation strain signal can be segmented, Hanning windows can be applied, and the cross-spectral density and auto-spectral density between the excitation signal and the perturbation strain signal can be determined to obtain the real-time perturbation spectral density energy distribution corresponding to each spatial location.

[0070] The third step is to generate an energy flux density map based on the real-time perturbation spectral density energy distribution described above. In practice, the energy value at each spatial location can be interpolated and mapped to a two-dimensional raster map aligned with the dimensions of the aforementioned building structure diagram, serving as the energy flux density map.

[0071] Step 205: Construct a composite sampling guidance vector field based on the above geometric prior weight map, the above cognitive uncertainty map, and the above energy flow density map.

[0072] In some embodiments, the execution entity can construct a composite sampling guidance vector field based on the geometric prior weight map, the cognitive uncertainty map, and the energy flow density map. In practice, firstly, the geometric prior weight map, the cognitive uncertainty map, and the energy flow density map can be weighted and fused to obtain a probability heatmap. Secondly, based on a preset image gradient algorithm, the negative gradient field corresponding to the probability heatmap can be determined as the composite sampling guidance vector field. The image gradient algorithm can be the Sobel algorithm.

[0073] Optionally, the aforementioned executing entity constructs a composite sampling guidance vector field based on the aforementioned geometric prior weight map, the aforementioned cognitive uncertainty map, and the aforementioned energy flux density map, which may include the following steps:

[0074] The first step is to perform pixel-level weighted fusion of the aforementioned geometric prior weight map, cognitive uncertainty map, and energy flow density map to generate a probability heatmap. In practice, firstly, the aforementioned geometric prior weight map, cognitive uncertainty map, and energy flow density map can be subjected to min-max normalization to obtain normalized geometric prior weight map, normalized cognitive uncertainty map, and normalized energy flow density map. The value of each pixel in the aforementioned normalized geometric prior weight map, normalized cognitive uncertainty map, and normalized energy flow density map is between 0 and 1. Next, the aforementioned normalized geometric prior weight map, normalized cognitive uncertainty map, and normalized energy flow density map are performed pixel-level weighted fusion to generate a probability heatmap, as shown in the following formula:

[0075] S(p)=γ·[Penergy(p)·(1+β·Punc(p))]+(1-γ)·Pgeo(p).

[0076] Where β represents the uncertainty modulation intensity, which can be 1.2; γ represents the balance coefficient between energy flow and geometric prior, which can be 0.6; Penergy(p) represents the element value of pixel p in the normalized energy flow density map; and Punc(p) represents the element value of pixel p in the normalized cognitive uncertainty map.

[0077] In practice, weighted fusion can enable regions with high energy density, low prediction confidence of convolutional networks, and weak geometry to achieve higher significance.

[0078] The second step is to determine the negative gradient field corresponding to the aforementioned probability heatmap as the basic driving vector field. This basic driving vector field can be a two-dimensional vector field with the same size as the probability heatmap. In practice, a preset image gradient algorithm can be used to determine the negative gradient field corresponding to the probability heatmap as the basic driving vector field.

[0079] The third step is to generate a geometric tangential vector field based on the structure tensor corresponding to the aforementioned geometric prior weight map. This geometric tangential vector field can be a two-dimensional vector field with the same size as the aforementioned probability heatmap. In practice, firstly, the structure tensor of the aforementioned geometric prior weight map can be determined. This structure tensor can have the same size as the aforementioned probability heatmap. Each element in the structure tensor can include a texture matrix. The texture matrix can be expressed as follows:

[0080] J(p) = ▽P geo (p)(▽P geo (p)) T .

[0081] Where J(p) represents the texture matrix of pixel p. ▽P geo (p) represents the gradient vector of pixel p. T represents the transpose of the matrix.

[0082] Next, for the texture matrix corresponding to each element in the aforementioned structure tensor, eigenvalue decomposition can be performed on the texture matrix to obtain an eigenvalue decomposition result set. The eigenvalue decomposition result set can include eigenvalues ​​and eigenvectors. Then, the eigenvector corresponding to the eigenvalue with the smallest value can be determined as the geometric tangential force. Finally, the obtained geometric tangential forces can be combined according to their corresponding spatial positions to form a geometric tangential vector field.

[0083] The fourth step is to determine the negative gradient field corresponding to the aforementioned cognitive uncertainty map as the uncertain attraction vector field. This uncertain attraction vector field can be a two-dimensional vector field with the same size as the aforementioned probability heatmap. In practice, the negative gradient field corresponding to the aforementioned cognitive uncertainty map can be determined as the uncertain attraction vector field based on the aforementioned image gradient algorithm.

[0084] The fifth step involves weighted fusion of the aforementioned basic driving vector field, geometric tangential vector field, and uncertain attraction vector field to obtain a composite sampling guidance vector field. In practice, the aforementioned basic driving vector field, geometric tangential vector field, and uncertain attraction vector field can be element-wise weighted and added together to obtain the composite sampling guidance vector field.

[0085] Step 206: Generate a local three-dimensional feature set based on the composite sampling guided vector field described above.

[0086] In some embodiments, the aforementioned execution entity may generate a local three-dimensional feature set based on the aforementioned composite sampling guided vector field.

[0087] Optionally, the execution entity generates a local three-dimensional feature set based on the composite sampling guided vector field, which may include the following steps:

[0088] The first step is to identify pixels in the probability heatmap whose values ​​are greater than a preset threshold as seed points, thus obtaining a seed point set. The threshold value can be a preset value, such as 0.85. No specific limitation is made here.

[0089] The second step is to perform the following steps for each seed point in the above seed point set:

[0090] The first sub-step involves numerically integrating the composite sampling guide vector field based on the aforementioned seed points to obtain a sampling point sequence. In practice, firstly, for each seed point in the seed point set, this seed point can be used as a starting point, and numerical integration can be performed along the direction of the composite sampling guide vector field using the fourth-order Runge-Kutta method (RK4) to obtain a sampling guide streamline. This sampling guide streamline can be a continuous, smooth line. Next, sampling points can be placed along the sampling guide streamline at a preset physical step size to obtain a sampling point sequence. This physical step size can be a distance value, for example, 10 centimeters.

[0091] The second sub-step involves controlling a sampling robot to perform a local tomographic scan of the building to be monitored at each sampling point in the aforementioned sampling point sequence, thereby obtaining a sequence of two-dimensional projection images. The sampling robot can be a robotic arm or robot carrying high-precision detection equipment (such as an industrial CT scanner or an ultrasonic phased array probe). In practice, firstly, for each sampling point in the aforementioned sampling point sequence, the executing entity can control the sampling robot to move to that sampling point. Then, the X-ray source and detector included in the sampling robot are controlled to perform a local tomographic scan of the area under test, obtaining a sequence of two-dimensional projection images. For example, within a range of ±45°, a rotational scan with a step size of 1.5° is performed to acquire a set of 61 high-resolution two-dimensional projection images.

[0092] The third sub-step involves performing 3D reconstruction on the aforementioned 2D projected image sequence based on a preset 3D reconstruction algorithm to obtain local 3D feature volumes. These 3D reconstruction algorithms may include, but are not limited to, filtered back projection algorithms (Feldkamp-Davis-Kress, FDK) and algebraic reconstruction techniques (SART). The aforementioned local 3D feature volumes can be high-resolution 3D voxel sets representing local regions.

[0093] Step 207: Perform defect detection on the above-mentioned local three-dimensional feature set and the above-mentioned global feature map to obtain a defect information set.

[0094] In some embodiments, the execution entity can perform defect detection on the local 3D feature set and the global feature map to obtain a defect information set. The defect information in the defect information set includes: defect category and defect location. The defect category can include: no defect, microcrack, through crack, and shear crack caused by settlement. The defect location can be three-dimensional coordinates. In practice, based on a preset defect detection model, defect detection can be performed on each local 3D feature in the local 3D feature set and the global feature map to generate defect information and obtain a defect information set. The defect detection model can include a 3D convolutional network, a 2D convolutional network, a feature fusion layer, and an output layer. The 3D convolutional network can be a 3DCNN. The 2D convolutional network can be a ResNet. The feature fusion layer can concatenate the features output by the 3D convolutional network with the features output by the 2D convolutional network. The output layer can be used to perform prediction operations. The output layer can include a fully connected layer and an activation function (e.g., Softmax). The defect detection model can be trained using a pre-built dataset. The data in the above dataset may include: local 3D feature volumes, global feature maps, defect category ground truth, and defect location ground truth.

[0095] Optionally, the execution entity performs defect detection on the aforementioned local three-dimensional feature set and the aforementioned global feature map to obtain a defect information set, which may include the following steps:

[0096] The first step involves fusing each local 3D feature volume in the aforementioned local 3D feature volume set with the global feature map across scales to obtain enhanced defect features. In practice, the local 3D feature volumes are first input into a 3D convolutional encoder, which outputs a query vector. Then, local feature blocks are extracted from the global feature map centered on the spatial coordinates corresponding to the query vector. Next, these local feature blocks are input into two independent projection networks to generate key and value vectors, respectively. These projection networks can be fully connected layers. Then, scaled dot product attention is performed on the query vector, key vector, and value vector to obtain attention features. Finally, the attention feature map is residually connected to the original feature vectors of the corresponding center points in the global feature map to obtain enhanced defect features.

[0097] Step 2: Perform defect detection on each of the obtained enhanced defect features to generate defect information, obtaining a defect information set. In practice, the above enhanced defect features can be input into a defect parameter decoder. The probability distribution of the defect category is output through the classification branch, and the three-dimensional spatial position and three-dimensional bounding box size of the defect are output through the regression branch. The above defect parameter decoder can include a classification branch and a regression branch. The above classification branch can include a fully connected layer (MLP) and an activation function (e.g., Softmax). The above regression branch can include a fully connected layer (MLP) and an activation function (e.g., ReLu). In practice, the flowchart of performing defect detection on the local three-dimensional feature volume and the above global feature map in the above optional step can be as Figure 3 shown.

[0098] Optionally, after the above "Step 207", the above execution entity can further perform the following steps:

[0099] Step 1: For each defect information in the above defect information set, perform the following steps:

[0100] The first sub-step: Generate defect level information according to the above defect information and a preset dynamic threshold. Among them, the above dynamic threshold can be a preset range. For example, (90, 95). The above defect category can correspond to a defect score. For example, no defect, microcrack, through crack, and shear crack caused by settlement can correspond to 40, 65, 92, and 99 respectively. The above defect level information can include: "attention level", "warning level", "severe level".

[0101] As an example, the above dynamic threshold can be defined based on the quantile of historical data. When the defect score belongs to the corresponding dynamic range, the corresponding defect level information is assigned. For example, when 50 < St < 90, the "attention level" is assigned. When 90 < St < 95, the "warning level" is assigned. When St ≥ 95, the "severe level" is assigned. St represents the defect score.

[0102] The second sub-step: In response to determining that the above defect level information meets the preset correction condition, control the hydraulic servo system corresponding to the defect position included in the above defect information to adjust the pile sealing pressure value, and obtain the real-time axial force value. Among them, the above correction condition can be that the defect level information is the severe level. In practice, when the above defect level information is the severe level, the pile sealing pressure value of the anchor pile under the corresponding area of the hydraulic servo system corresponding to the defect position can be controlled. At the same time, continuously monitor the axial force of the vertical member corresponding to the above defect position as the real-time axial force value.

[0103] The third sub-step involves controlling the hydraulic servo system to stop in response to determining that the real-time axial force value is equal to the preset design value. Each spatial location of the building to be monitored can correspond to a preset design value. This design value characterizes the rated axial force value of the anchor pile at that spatial location. In practice, when the real-time axial force value equals the design value, the executing entity can control the hydraulic servo system to stop operating.

[0104] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a building correction monitoring device based on multimodal sensing and digital twins. These device embodiments are similar to... Figure 2 Corresponding to the method embodiments shown, this building correction monitoring device based on multimodal sensing and digital twins can be specifically applied to various electronic devices.

[0105] like Figure 4 As shown, a building correction monitoring device 400 based on multimodal perception and digital twin in some embodiments includes: an acquisition unit 401, a feature extraction unit 402, a semantic segmentation unit 403, an energy flow analysis unit 404, a construction unit 405, a generation unit 406, and a defect detection unit 407. The system comprises the following components: an acquisition unit 401, configured to acquire multi-source sensing data of the building to be monitored, and to construct a differential settlement model based on the multi-source sensing data and a preset building design model; a feature extraction unit 402, configured to extract features from a preset building feature map to obtain a global feature map and a cognitive uncertainty map; a semantic segmentation unit 403, configured to perform semantic segmentation on the building structure map of the building to be monitored to obtain a geometric prior weight map; an energy flow analysis unit 404, configured to perform energy flow analysis on the differential settlement model to generate an energy flow density map; a construction unit 405, configured to construct a composite sampling guide vector field based on the geometric prior weight map, the cognitive uncertainty map, and the energy flow density map; a generation unit 406, configured to generate a local three-dimensional feature set based on the composite sampling guide vector field; and a defect detection unit 407, configured to perform defect detection on the local three-dimensional feature set and the global feature map to obtain a defect information set, wherein the defect information in the defect information set includes defect category and defect location.

[0106] It is understandable that the various units and references recorded in the building correction monitoring device 400 based on multimodal perception and digital twins... Figure 2 The steps in the described method correspond accordingly. Therefore, the operations, features, and beneficial effects described above for the method also apply to the building correction monitoring device 400 based on multimodal sensing and digital twins and the units contained therein, and will not be repeated here.

[0107] The following is for reference. Figure 5 It shows a schematic diagram of the structure of an electronic device (e.g., a computing device) 500 suitable for implementing some embodiments of the present disclosure. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0108] like Figure 5 As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory 502 or a program loaded from a storage device 508 into a random access memory 503. The random access memory 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, the read-only memory 502, and the random access memory 503 are interconnected via a bus 504. An input / output interface 505 is also connected to the bus 504.

[0109] Typically, the following devices can be connected to the input / output interface 505: input devices 506 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 508 including, for example, magnetic tape, hard disk, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 5 Each box shown can represent a device or multiple devices as needed.

[0110] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a read-only memory 502. When the computer program is executed by the processing device 501, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0111] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0112] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0113] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: acquire multi-source sensing data of the building to be monitored, and construct a differential settlement model based on the multi-source sensing data and a preset building design model; extract features from a preset building feature map to obtain a global feature map and a cognitive uncertainty map; perform semantic segmentation on the building structure map of the building to be monitored to obtain a geometric prior weight map; perform energy flow analysis on the differential settlement model to generate an energy flow density map; construct a composite sampling guidance vector field based on the geometric prior weight map, the cognitive uncertainty map, and the energy flow density map; generate a local three-dimensional feature set based on the composite sampling guidance vector field; and perform defect detection on the local three-dimensional feature set and the global feature map to obtain a defect information set, wherein the defect information in the defect information set includes defect category and defect location.

[0114] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0116] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0117] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A building deviation monitoring method based on multi-modal perception and digital twinning, characterized in that, include: Acquire multi-source sensing data of the building to be monitored, and construct a differential settlement model based on the multi-source sensing data and a preset building design model; Feature extraction is performed on the preset building feature map to obtain a global feature map and a cognitive uncertainty map; Semantic segmentation is performed on the building structure diagram of the building to be monitored to obtain a geometric prior weight map; Energy flow analysis was performed on the differential settlement model to generate an energy flow density map; A composite sampling guidance vector field is constructed based on the geometric prior weight map, the cognitive uncertainty map, and the energy flow density map. Based on the composite sampling guided vector field, a local three-dimensional feature set is generated; Defect detection is performed on the local three-dimensional feature set and the global feature map to obtain a defect information set, wherein the defect information in the defect information set includes defect category and defect location.

2. The method of claim 1, wherein, The method further includes: For each defect in the defect information set, perform the following steps: Based on the defect information and the preset dynamic threshold, defect level information is generated; In response to determining that the defect level information meets the preset correction conditions, the hydraulic servo system corresponding to the defect location included in the defect information is controlled to adjust the sealing pressure value and obtain the real-time axial force value. In response to determining that the real-time axial force value is equal to the preset design value, the hydraulic servo system is controlled to stop.

3. The method of claim 1, wherein, The semantic segmentation of the building structure diagram of the building to be monitored to obtain a geometric prior weight map includes: The building structure diagram of the building to be monitored is segmented to obtain a set of segmented regions, wherein the set of segmented regions includes: weld region, splice region, and support region; For each segmented region in the segmented region set, perform the following steps: Morphological skeleton extraction is performed on the segmented region to obtain the morphological center line; The nearest distance between each pixel in the segmented region and the morphological center line is determined as the pixel center distance, thus obtaining the pixel center distance set; Based on a preset Gaussian kernel function, the mapping weight corresponding to each pixel center distance in the pixel center distance set is determined to obtain a mapping weight set. Based on the obtained sets of mapping weights, a geometric prior weight graph is generated.

4. The method of claim 1, wherein, The step of performing energy flow analysis on the differential settlement model to generate an energy flow density map includes: Based on the preset excitation signal and the differential settlement model, a corresponding disturbance strain signal is generated; Determine the real-time perturbation spectral density energy distribution corresponding to the excitation signal and the perturbation strain signal; An energy flux density map is generated based on the real-time perturbation spectral density energy distribution.

5. The method of claim 1, wherein, The step of constructing a composite sampling guidance vector field based on the geometric prior weight map, the cognitive uncertainty map, and the energy flux density map includes: The geometric prior weight map, the cognitive uncertainty map, and the energy flow density map are fused at the pixel level to generate a probability heatmap. The negative gradient field corresponding to the probability heatmap is determined as the basic driving vector field; Based on the structural tensor corresponding to the geometric prior weight map, a geometric tangential vector field is generated; The negative gradient field corresponding to the cognitive uncertainty map is determined as the uncertain attraction vector field; The fundamental driving vector field, the geometric tangential vector field, and the uncertain attraction vector field are weighted and fused to obtain a composite sampling guiding vector field.

6. The method according to claim 5, characterized in that, The step of generating a local three-dimensional feature set based on the composite sampling guided vector field includes: Pixels whose pixel values ​​in the probability heatmap are greater than a preset threshold value are identified as seed points, and a seed point set is obtained. For each seed point in the seed point set, perform the following steps: Based on the seed points, the composite sampling guide vector field is numerically integrated to obtain a sampling point sequence; For each sampling point in the sampling point sequence, the sampling robot is controlled to perform a local tomographic scan of the building to be monitored at the sampling point to obtain a two-dimensional projection image sequence. Based on a preset 3D reconstruction algorithm, the 2D projected image sequence is reconstructed into 3D to obtain local 3D feature volumes.

7. The method according to claim 1, characterized in that, The defect detection process, which involves performing defect detection on the local 3D feature set and the global feature map to obtain a defect information set, includes: For each local three-dimensional feature volume in the set of local three-dimensional feature volumes, the local three-dimensional feature volume is fused with the global feature map across scales to obtain enhanced defect features; Each enhanced defect feature obtained is subjected to defect detection to generate defect information, resulting in a defect information set.

8. A building deviation correction monitoring device based on multimodal sensing and digital twin, characterized in that, include: The acquisition unit is configured to acquire multi-source sensing data of the building to be monitored, and to construct a differential settlement model based on the multi-source sensing data and a preset building design model. The feature extraction unit is configured to extract features from a preset building feature map to obtain a global feature map and a cognitive uncertainty map; A semantic segmentation unit is configured to perform semantic segmentation on the building structure diagram of the building to be monitored to obtain a geometric prior weight map; An energy flow analysis unit is configured to perform energy flow analysis on the differential settlement model and generate an energy flow density map. The construction unit is configured to construct a composite sampling guidance vector field based on the geometric prior weight map, the cognitive uncertainty map, and the energy flux density map; The generation unit is configured to generate a local three-dimensional feature set based on the composite sampling guide vector field; The defect detection unit is configured to perform defect detection on the local three-dimensional feature set and the global feature map to obtain a defect information set, wherein the defect information in the defect information set includes defect category and defect location.

9. An electronic device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that, It stores a computer program thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images